Add optimisers example. README, requirements, and python version of the exercise.

Author: jatkinson1000

examples/n_Optimisers/README.md

@@ -0,0 +1,56 @@
+# Example n - Optimisers
+
+**This example is currently under development.** Eventually, it will demonstrate
+the use of optimisers in FTorch by leveraging PyTorch's optim module.
+
+By exposing optimisers in Fortran, FTorch will be able to compute optimisation
+steps to update models as part of a training process.
+
+## Description
+
+A Python demo is copied from the PyTorch documentation as `optimisers.py`, which
+shows how to use an optimiser in PyTorch.
+
+The demo will be replicated in Fortran as `optimisers.f90`, to show how to do the
+same thing using FTorch.
+
+## Dependencies
+
+To run this example requires:
+
+- CMake
+- Fortran compiler
+- FTorch (installed as described in main package)
+- Python 3
+
+## Running
+
+To run this example install FTorch as described in the main documentation.
+Then from this directory create a virtual environment and install the necessary
+Python modules:
+```
+python3 -m venv venv
+source venv/bin/activate
+pip install -r requirements.txt
+```
+
+Run the Python version of the demo with
+```
+python3 optimisers.py
+```
+This trains a tensor to scale, elementwise, a vector of ones to the vector `[1, 2, 3, 4]`.
+It uses the torch SGD optimiser to adjust the values of the scaling tensor at each step,
+outputting values of interest to screen in the form:
+```console
+========================
+Epoch: 0
+        Output:
+                tensor([1., 1., 1., 1.], grad_fn=<MulBackward0>)
+        loss:
+                3.5
+        tensor gradient:
+                tensor([ 0.0000, -0.5000, -1.0000, -1.5000])
+        tensor:
+                tensor([1.0000, 1.5000, 2.0000, 2.5000], requires_grad=True)
+...
+```

examples/n_Optimisers/optimisers.py

@@ -0,0 +1,55 @@
+"""Optimisers demo."""
+
+import torch
+
+# We define:
+#  - the input as as a vector of ones,
+#  - the target as a vector where each element is the index value,
+#  - a tensor to transform from input to target by elementwise multiplication
+#    initialised as a vector of ones
+# This is a contrived example, but provides a simple demo of optimiser functionality
+input_vec = torch.ones(4)
+target_vec = torch.tensor([1.0, 2.0, 3.0, 4.0])
+scaling_tensor = torch.ones(4, requires_grad=True)
+
+# Set the optimiser as torch's stochastic gradient descent (SGD)
+# The parameters to tune will be the values of `tensor`, and we also set a learning rate
+# Since this is a simple elemetwise example we can get away with a large learning rate
+optimizer = torch.optim.SGD([scaling_tensor], lr=1.0)
+
+# Training loop
+# Run n_iter times printing every n_print steps
+n_iter = 15
+n_print = 1
+for epoch in range(n_iter + 1):
+    # Zero any previously stored gradients ready for a new iteration
+    optimizer.zero_grad()
+
+    # Forward pass: multiply the input of ones by the tensor (elementwise)
+    output = input_vec * scaling_tensor
+
+    # Create a loss tensor as computed mean square error (MSE) between target and input
+    # Then perform backward step on loss to propogate gradients using autograd
+    #
+    # We could use the following 2 lines to do this by explicitly specifying a
+    # gradient of ones to start the process:
+    # loss = ((output - target) ** 2) / 4.0
+    # loss.backward(gradient=torch.ones(4))
+    #
+    # However, we can avoid explicitly passing an initial gradient and instead do this
+    # implicitly by aggregating the loss vector into a scalar value:
+    loss = ((output - target_vec) ** 2).mean()
+    loss.backward()
+
+    # Step the optimiser to update the values in `tensor`
+    optimizer.step()
+
+    if (epoch) % n_print == 0:
+        print(f"========================")
+        print(f"Epoch: {epoch}")
+        print(f"\tOutput:\n\t\t{output}")
+        print(f"\tloss:\n\t\t{loss}")
+        print(f"\ttensor gradient:\n\t\t{scaling_tensor.grad}")
+        print(f"\tscaling_tensor:\n\t\t{scaling_tensor}")
+
+print("Training complete.")

examples/n_Optimisers/requirements.txt

@@ -0,0 +1,2 @@
+torch
+numpy