Skip to content

Commit ec650f5

Browse files
mhjensenmhjensen
committed
update of p2
1 parent 2ada3ce commit ec650f5

File tree

11 files changed

+1048
-744
lines changed

11 files changed

+1048
-744
lines changed

‎doc/LectureNotes/_build/.doctrees/environment.pickle‎

1.37 KB
Binary file not shown.

‎doc/LectureNotes/_build/.doctrees/project2.doctree‎

13.5 KB
Binary file not shown.

‎doc/LectureNotes/_build/.doctrees/week42.doctree‎

464 Bytes
Binary file not shown.

‎doc/LectureNotes/_build/html/_sources/project2.ipynb‎

Lines changed: 110 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
"cells": [
33
{
44
"cell_type": "markdown",
5-
"id": "061af572",
5+
"id": "96e577ca",
66
"metadata": {
77
"editable": true
88
},
@@ -14,7 +14,7 @@
1414
},
1515
{
1616
"cell_type": "markdown",
17-
"id": "22578683",
17+
"id": "067c02b9",
1818
"metadata": {
1919
"editable": true
2020
},
@@ -27,7 +27,7 @@
2727
},
2828
{
2929
"cell_type": "markdown",
30-
"id": "61fb162f",
30+
"id": "01f9fedd",
3131
"metadata": {
3232
"editable": true
3333
},
@@ -58,7 +58,7 @@
5858
},
5959
{
6060
"cell_type": "markdown",
61-
"id": "104c69e1",
61+
"id": "9f8e4871",
6262
"metadata": {
6363
"editable": true
6464
},
@@ -104,7 +104,7 @@
104104
},
105105
{
106106
"cell_type": "markdown",
107-
"id": "0d2c42e3",
107+
"id": "460cc6ea",
108108
"metadata": {
109109
"editable": true
110110
},
@@ -121,7 +121,7 @@
121121
"\n",
122122
"* Regression (fitting a continuous function). In this part you will need to bring back your results from project 1 and compare these with what you get from your Neural Network code to be developed here. The data sets could be\n",
123123
"\n",
124-
" * The simple one-dimensional function Runge function from project 1, that is $f(x) = \\frac{1}{1+25x^2}$. We recommend using a simpler function when developing your neural network code for regression problems. Feel however free to discuss and study other functions, such as the the two-dimensional Runge function $f(x,y)=\\left[(10x - 5)^2 + (10y - 5)^2 + 1 \\right]^{-1}$, or even more complicated two-dimensional functions (see the supplementary material of <https://www.nature.com/articles/s41467-025-61362-4> for an extensive list of two-dimensional functions). \n",
124+
" * The simple one-dimensional function Runge function from project 1, that is $f(x) = \\frac{1}{1+25x^2}$. We recommend using a simpler function when developing your neural network code for regression problems. Feel however free to discuss and study other functions, such as the two-dimensional Runge function $f(x,y)=\\left[(10x - 5)^2 + (10y - 5)^2 + 1 \\right]^{-1}$, or even more complicated two-dimensional functions (see the supplementary material of <https://www.nature.com/articles/s41467-025-61362-4> for an extensive list of two-dimensional functions). \n",
125125
"\n",
126126
"* Classification.\n",
127127
"\n",
@@ -132,7 +132,7 @@
132132
},
133133
{
134134
"cell_type": "markdown",
135-
"id": "d8baab67",
135+
"id": "d62a07ef",
136136
"metadata": {
137137
"editable": true
138138
},
@@ -162,7 +162,7 @@
162162
},
163163
{
164164
"cell_type": "markdown",
165-
"id": "87e7ed71",
165+
"id": "9cd8b8ac",
166166
"metadata": {
167167
"editable": true
168168
},
@@ -189,7 +189,7 @@
189189
},
190190
{
191191
"cell_type": "markdown",
192-
"id": "5a26b6ad",
192+
"id": "5931b155",
193193
"metadata": {
194194
"editable": true
195195
},
@@ -205,7 +205,7 @@
205205
},
206206
{
207207
"cell_type": "markdown",
208-
"id": "096fe6c4",
208+
"id": "b273fc8a",
209209
"metadata": {
210210
"editable": true
211211
},
@@ -217,7 +217,7 @@
217217
},
218218
{
219219
"cell_type": "markdown",
220-
"id": "fd986596",
220+
"id": "e13db1ec",
221221
"metadata": {
222222
"editable": true
223223
},
@@ -252,7 +252,7 @@
252252
},
253253
{
254254
"cell_type": "markdown",
255-
"id": "e853d4b6",
255+
"id": "4f864e31",
256256
"metadata": {
257257
"editable": true
258258
},
@@ -270,7 +270,7 @@
270270
},
271271
{
272272
"cell_type": "markdown",
273-
"id": "fc2d413b",
273+
"id": "c9faeafd",
274274
"metadata": {
275275
"editable": true
276276
},
@@ -285,7 +285,7 @@
285285
},
286286
{
287287
"cell_type": "markdown",
288-
"id": "e6821051",
288+
"id": "d865c22b",
289289
"metadata": {
290290
"editable": true
291291
},
@@ -302,7 +302,7 @@
302302
},
303303
{
304304
"cell_type": "markdown",
305-
"id": "cba72d68",
305+
"id": "5270af8f",
306306
"metadata": {
307307
"editable": true
308308
},
@@ -328,7 +328,7 @@
328328
{
329329
"cell_type": "code",
330330
"execution_count": 1,
331-
"id": "e16fb528",
331+
"id": "4e0e1fea",
332332
"metadata": {
333333
"collapsed": false,
334334
"editable": true
@@ -347,7 +347,7 @@
347347
},
348348
{
349349
"cell_type": "markdown",
350-
"id": "73599f42",
350+
"id": "8fe85677",
351351
"metadata": {
352352
"editable": true
353353
},
@@ -358,7 +358,7 @@
358358
{
359359
"cell_type": "code",
360360
"execution_count": 2,
361-
"id": "f1a639ef",
361+
"id": "b28318b2",
362362
"metadata": {
363363
"collapsed": false,
364364
"editable": true
@@ -370,7 +370,7 @@
370370
},
371371
{
372372
"cell_type": "markdown",
373-
"id": "90fb7b41",
373+
"id": "97e02c71",
374374
"metadata": {
375375
"editable": true
376376
},
@@ -381,7 +381,7 @@
381381
{
382382
"cell_type": "code",
383383
"execution_count": 3,
384-
"id": "424af629",
384+
"id": "88af355c",
385385
"metadata": {
386386
"collapsed": false,
387387
"editable": true
@@ -394,7 +394,7 @@
394394
},
395395
{
396396
"cell_type": "markdown",
397-
"id": "3c006080",
397+
"id": "d1f8f0ed",
398398
"metadata": {
399399
"editable": true
400400
},
@@ -407,7 +407,7 @@
407407
},
408408
{
409409
"cell_type": "markdown",
410-
"id": "a18ddd54",
410+
"id": "554b3a48",
411411
"metadata": {
412412
"editable": true
413413
},
@@ -419,7 +419,7 @@
419419
},
420420
{
421421
"cell_type": "markdown",
422-
"id": "1a1afaf9",
422+
"id": "77bfdd5c",
423423
"metadata": {
424424
"editable": true
425425
},
@@ -434,15 +434,15 @@
434434
"code for classification and pertinent results against a similar code using **Scikit-Learn** or **tensorflow/keras** or **pytorch**.\n",
435435
"\n",
436436
"If you have time, you can use the functionality of **scikit-learn** and compare your neural network results with those from Logistic regression. This is optional.\n",
437-
"The weblink here <https://medium.com/ai-in-plain-english/comparison-between-logistic-regression-and-neural-networks-in-classifying-digits-dc5e85cd93c3>compares logistic regression and FFNN using the so-called MNIST data set. You may find several useful hints and ideas from this article. Your neural network code can implement the equivalent of logistic regression by simply setting the number of hidden layers to zero. \n",
437+
"The weblink here <https://medium.com/ai-in-plain-english/comparison-between-logistic-regression-and-neural-networks-in-classifying-digits-dc5e85cd93c3>compares logistic regression and FFNN using the so-called MNIST data set. You may find several useful hints and ideas from this article. Your neural network code can implement the equivalent of logistic regression by simply setting the number of hidden layers to zero and keeping just the input and the output layers. \n",
438438
"\n",
439439
"If you wish to compare with say Logisti Regression from **scikit-learn**, the following code uses the above data set"
440440
]
441441
},
442442
{
443443
"cell_type": "code",
444444
"execution_count": 4,
445-
"id": "3c37cbaf",
445+
"id": "eaa9e72e",
446446
"metadata": {
447447
"collapsed": false,
448448
"editable": true
@@ -464,7 +464,7 @@
464464
},
465465
{
466466
"cell_type": "markdown",
467-
"id": "106b9303",
467+
"id": "c7ba883e",
468468
"metadata": {
469469
"editable": true
470470
},
@@ -480,7 +480,88 @@
480480
},
481481
{
482482
"cell_type": "markdown",
483-
"id": "55da0d7f",
483+
"id": "595be693",
484+
"metadata": {
485+
"editable": true
486+
},
487+
"source": [
488+
"## Summary of methods to implement and analyze\n",
489+
"\n",
490+
"**Required Implementation:**\n",
491+
"1. Reuse the regression code and results from project 1, these will act as a benchmark for seeing how suited a neural network is for this regression task.\n",
492+
"\n",
493+
"2. Implement a neural network with\n",
494+
"\n",
495+
" * A flexible number of layers\n",
496+
"\n",
497+
" * A flexible number of nodes in each layer\n",
498+
"\n",
499+
" * A changeable activation function in each layer (Sigmoid, ReLU, LeakyReLU, as well as Linear and Softmax)\n",
500+
"\n",
501+
" * A changeable cost function, which will be set to MSE for regression and cross-entropy for multiple-classification\n",
502+
"\n",
503+
" * An optional L1 or L2 norm of the weights and biases in the cost function (only used for computing gradients, not interpretable metrics)\n",
504+
"\n",
505+
"3. Implement the back-propagation algorithm to compute the gradient of your neural network\n",
506+
"\n",
507+
"4. Reuse the implementation of Plain and Stochastic Gradient Descent from Project 1 (and adapt the code to work with the your neural network)\n",
508+
"\n",
509+
" * With no optimization algorithm\n",
510+
"\n",
511+
" * With RMS Prop\n",
512+
"\n",
513+
" * With ADAM\n",
514+
"\n",
515+
"5. Implement scaling and train-test splitting of your data, preferably using sklearn\n",
516+
"\n",
517+
"6. Implement and compute metrics like the MSE and Accuracy"
518+
]
519+
},
520+
{
521+
"cell_type": "markdown",
522+
"id": "35138b41",
523+
"metadata": {
524+
"editable": true
525+
},
526+
"source": [
527+
"### Required Analysis:\n",
528+
"\n",
529+
"1. Briefly show and argue for the advantages and disadvantages of the methods from Project 1.\n",
530+
"\n",
531+
"2. Explore and show the impact of changing the number of layers, nodes per layer, choice of activation function, and inclusion of L1 and L2 norms. Present only the most interesting results from this exploration. 2D Heatmaps will be good for this: Start with finding a well performing set of hyper-parameters, then change two at a time in a range that shows good and bad performance.\n",
532+
"\n",
533+
"3. Show and argue for the advantages and disadvantages of using a neural network for regression on your data\n",
534+
"\n",
535+
"4. Show and argue for the advantages and disadvantages of using a neural network for classification on your data\n",
536+
"\n",
537+
"5. Show and argue for the advantages and disadvantages of the different gradient methods and learning rates when training the neural network"
538+
]
539+
},
540+
{
541+
"cell_type": "markdown",
542+
"id": "b18bea03",
543+
"metadata": {
544+
"editable": true
545+
},
546+
"source": [
547+
"### Optional (Note that you should include at least two of these in the report):\n",
548+
"\n",
549+
"1. Implement Logistic Regression as simple classification model case (equivalent to a Neural Network with just the output layer)\n",
550+
"\n",
551+
"2. Compute the gradient of the neural network with autograd, to show that it gives the same result as your hand-written backpropagation.\n",
552+
"\n",
553+
"3. Compare your results with results from using a machine-learning library like pytorch (https://docs.pytorch.org/tutorials/beginner/basics/buildmodel_tutorial.html)\n",
554+
"\n",
555+
"4. Use a more complex classification dataset instead, like the fashion MNIST (see <https://www.kaggle.com/datasets/zalando-research/fashionmnist>)\n",
556+
"\n",
557+
"5. Use a more complex regression dataset instead, like the two-dimensional Runge function $f(x,y)=\\left[(10x - 5)^2 + (10y - 5)^2 + 1 \\right]^{-1}$, or even more complicated two-dimensional functions (see the supplementary material of <https://www.nature.com/articles/s41467-025-61362-4> for an extensive list of two-dimensional functions). \n",
558+
"\n",
559+
"6. Compute and interpret a confusion matrix of your best classification model (see <https://www.researchgate.net/figure/Confusion-matrix-of-MNIST-and-F-MNIST-embeddings_fig5_349758607>)"
560+
]
561+
},
562+
{
563+
"cell_type": "markdown",
564+
"id": "580d8424",
484565
"metadata": {
485566
"editable": true
486567
},
@@ -496,7 +577,7 @@
496577
},
497578
{
498579
"cell_type": "markdown",
499-
"id": "d3731e2c",
580+
"id": "96f5c67e",
500581
"metadata": {
501582
"editable": true
502583
},
@@ -527,7 +608,7 @@
527608
},
528609
{
529610
"cell_type": "markdown",
530-
"id": "6c7c5340",
611+
"id": "d1bc28ba",
531612
"metadata": {
532613
"editable": true
533614
},

0 commit comments

Comments
 (0)