updated week 48

Author: mhjensen

doc/LectureNotes/_build/.doctrees/environment.pickle

@@ -527,7 +527,7 @@ <h2> Contents </h2>
 <!-- dom:TITLE: Week 48: Gradient boosting  and summary of course --><section class="tex2jax_ignore mathjax_ignore" id="week-48-gradient-boosting-and-summary-of-course">
 <h1>Week 48: Gradient boosting  and summary of course<a class="headerlink" href="#week-48-gradient-boosting-and-summary-of-course" title="Link to this heading">#</a></h1>
 <p><strong>Morten Hjorth-Jensen</strong>, Department of Physics and Center for Computing in Science Education, University of Oslo, Norway</p>
-<p>Date: <strong>Nov 24, 2024</strong></p>
+<p>Date: <strong>Nov 25, 2024</strong></p>
 <p>Copyright 1999-2024, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license</p>
 <section id="overview-of-week-48">
 <h2>Overview of week 48<a class="headerlink" href="#overview-of-week-48" title="Link to this heading">#</a></h2>
@@ -542,13 +542,13 @@ <h2>Lecture Monday, November 25<a class="headerlink" href="#lecture-monday-novem
 </ol>
 <p>a. These lecture notes at <a class="github reference external" href="https://github.com/CompPhysics/MachineLearning/blob/master/doc/pub/week48/ipynb/week48.ipynb">CompPhysics/MachineLearning</a></p>
 <p>b. See also lecture notes from week 47 at <a class="github reference external" href="https://github.com/CompPhysics/MachineLearning/blob/master/doc/pub/week47/ipynb/week47.ipynb">CompPhysics/MachineLearning</a>. The lecture on Monday starts with a repetition on AdaBoost before we move over to gradient boosting with examples</p>
-<!-- o Video of lecture at <https://youtu.be/RIHzmLv05DA> -->
-<!-- o Whiteboard notes at <https://github.com/CompPhysics/MachineLearning/blob/master/doc/HandWrittenNotes/2024/NotesNovember25.pdf> -->
-<p>c. Video on Decision trees <a class="reference external" href="https://www.youtube.com/watch?v=RmajweUFKvM&amp;amp;ab_channel=Simplilearn">https://www.youtube.com/watch?v=RmajweUFKvM&amp;ab_channel=Simplilearn</a></p>
-<p>d. Video on boosting methods <a class="reference external" href="https://www.youtube.com/watch?v=wPqtzj5VZus&amp;amp;ab_channel=H2O.ai">https://www.youtube.com/watch?v=wPqtzj5VZus&amp;ab_channel=H2O.ai</a></p>
-<p>e. Video on AdaBoost <a class="reference external" href="https://www.youtube.com/watch?v=LsK-xG1cLYA">https://www.youtube.com/watch?v=LsK-xG1cLYA</a></p>
-<p>f. Video on Gradient boost, part 1, parts 2-4 follow thereafter <a class="reference external" href="https://www.youtube.com/watch?v=3CC4N4z3GJc">https://www.youtube.com/watch?v=3CC4N4z3GJc</a></p>
-<p>g. Decision Trees: Rashcka et al chapter 3 pages 86-98, and chapter 7 on Ensemble methods, Voting and Bagging and Gradient Boosting. See also lecture from STK-IN4300, lecture 7 at <a class="reference external" href="https://www.uio.no/studier/emner/matnat/math/STK-IN4300/h20/slides/lecture_7.pdf">https://www.uio.no/studier/emner/matnat/math/STK-IN4300/h20/slides/lecture_7.pdf</a>.</p>
+<p>c. Video of lecture at <a class="reference external" href="https://youtu.be/iTaRdAPQnDA">https://youtu.be/iTaRdAPQnDA</a></p>
+<p>d. Whiteboard notes at <a class="github reference external" href="https://github.com/CompPhysics/MachineLearning/blob/master/doc/HandWrittenNotes/2024/NotesNovember25.pdf">CompPhysics/MachineLearning</a></p>
+<p>e. Video on Decision trees <a class="reference external" href="https://www.youtube.com/watch?v=RmajweUFKvM&amp;amp;ab_channel=Simplilearn">https://www.youtube.com/watch?v=RmajweUFKvM&amp;ab_channel=Simplilearn</a></p>
+<p>f. Video on boosting methods <a class="reference external" href="https://www.youtube.com/watch?v=wPqtzj5VZus&amp;amp;ab_channel=H2O.ai">https://www.youtube.com/watch?v=wPqtzj5VZus&amp;ab_channel=H2O.ai</a></p>
+<p>g. Video on AdaBoost <a class="reference external" href="https://www.youtube.com/watch?v=LsK-xG1cLYA">https://www.youtube.com/watch?v=LsK-xG1cLYA</a></p>
+<p>h. Video on Gradient boost, part 1, parts 2-4 follow thereafter <a class="reference external" href="https://www.youtube.com/watch?v=3CC4N4z3GJc">https://www.youtube.com/watch?v=3CC4N4z3GJc</a></p>
+<p>i. Decision Trees: Rashcka et al chapter 3 pages 86-98, and chapter 7 on Ensemble methods, Voting and Bagging and Gradient Boosting. See also lecture from STK-IN4300, lecture 7 at <a class="reference external" href="https://www.uio.no/studier/emner/matnat/math/STK-IN4300/h20/slides/lecture_7.pdf">https://www.uio.no/studier/emner/matnat/math/STK-IN4300/h20/slides/lecture_7.pdf</a>.</p>
 </section>
 <section id="lab-sessions">
 <h2>Lab sessions<a class="headerlink" href="#lab-sessions" title="Link to this heading">#</a></h2>
@@ -651,17 +651,17 @@ <h2>Random Forests Compared with other Methods on the Cancer Data<a class="heade
 (143, 30)
 Test set accuracy Logistic Regression with scaled data: 0.96
 Test set accuracy SVM with scaled data: 0.96
-Test set accuracy with Decision Trees and scaled data: 0.91
+Test set accuracy with Decision Trees and scaled data: 0.92
 </pre></div>
 </div>
-<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[1.         0.8        0.93333333 1.         1.         0.92857143
+<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[1.         0.73333333 0.93333333 1.         1.         0.92857143
  1.         0.92857143 0.92857143 0.92857143]
-Test set accuracy with Random Forests and scaled data: 0.98
+Test set accuracy with Random Forests and scaled data: 0.97
 </pre></div>
 </div>
-<img alt="_images/936367d3bdcae10aafd2cc903d30ce54287b55ddddaa7af46f455a620a3745cd.png" src="_images/936367d3bdcae10aafd2cc903d30ce54287b55ddddaa7af46f455a620a3745cd.png" />
-<img alt="_images/be8d5df8bb940da757ef8fd6eac65ebe0352641500f781f4284c949ce274e1ee.png" src="_images/be8d5df8bb940da757ef8fd6eac65ebe0352641500f781f4284c949ce274e1ee.png" />
-<img alt="_images/8f696a60652d0003039dd9a563eb80367f1d574ca15b61c7a3f9757b19083d26.png" src="_images/8f696a60652d0003039dd9a563eb80367f1d574ca15b61c7a3f9757b19083d26.png" />
+<img alt="_images/90075505602c3f17740e87e303ddff0ae0ff2ec0245679468ad8fb7cb2ba3b3a.png" src="_images/90075505602c3f17740e87e303ddff0ae0ff2ec0245679468ad8fb7cb2ba3b3a.png" />
+<img alt="_images/a318a67ffc1f6a60c2418c7f612568dd9b06752eb57ba2c4643dcc6e7d1a01dc.png" src="_images/a318a67ffc1f6a60c2418c7f612568dd9b06752eb57ba2c4643dcc6e7d1a01dc.png" />
+<img alt="_images/1d150add40cbaa91d293348b988a5fbd2e04c68c41f11a70ebf78ac5686e7e4a.png" src="_images/1d150add40cbaa91d293348b988a5fbd2e04c68c41f11a70ebf78ac5686e7e4a.png" />
 </div>
 </div>
 <p>Recall that the cumulative gains curve shows the percentage of the
@@ -1173,30 +1173,30 @@ <h2>Gradient Boosting, Examples of Regression<a class="headerlink" href="#gradie
 </div>
 <div class="cell_output docutils container">
 <div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Max depth: 1
-Error: 0.4203129333425336
-Bias^2: 0.21226966048908316
-Var: 0.20804327285345042
-0.4203129333425336 &gt;= 0.21226966048908316 + 0.20804327285345042 = 0.4203129333425336
+Error: 0.4010613825254484
+Bias^2: 0.2079593417034804
+Var: 0.19310204082196794
+0.4010613825254484 &gt;= 0.2079593417034804 + 0.19310204082196794 = 0.4010613825254483
 Max depth: 2
-Error: 0.40767639731018696
-Bias^2: 0.21200998139721822
-Var: 0.19566641591296877
-0.40767639731018696 &gt;= 0.21200998139721822 + 0.19566641591296877 = 0.407676397310187
+Error: 0.4250776117755916
+Bias^2: 0.2080984218270197
+Var: 0.21697918994857185
+0.4250776117755916 &gt;= 0.2080984218270197 + 0.21697918994857185 = 0.42507761177559156
 Max depth: 3
-Error: 0.4076774836661818
-Bias^2: 0.2120099429256955
-Var: 0.19566754074048626
-0.4076774836661818 &gt;= 0.2120099429256955 + 0.19566754074048626 = 0.40767748366618173
+Error: 0.4250796355306808
+Bias^2: 0.2080985447081304
+Var: 0.21698109082255032
+0.4250796355306808 &gt;= 0.2080985447081304 + 0.21698109082255032 = 0.42507963553068073
 Max depth: 4
-Error: 0.4076774836661818
-Bias^2: 0.2120099429256955
-Var: 0.19566754074048626
-0.4076774836661818 &gt;= 0.2120099429256955 + 0.19566754074048626 = 0.40767748366618173
+Error: 0.4250796355306808
+Bias^2: 0.2080985447081304
+Var: 0.21698109082255038
+0.4250796355306808 &gt;= 0.2080985447081304 + 0.21698109082255038 = 0.4250796355306808
 Max depth: 5
-Error: 0.4076774836661816
-Bias^2: 0.2120099429256955
-Var: 0.1956675407404862
-0.4076774836661816 &gt;= 0.2120099429256955 + 0.1956675407404862 = 0.40767748366618173
+Error: 0.42507963553068073
+Bias^2: 0.2080985447081304
+Var: 0.21698109082255032
+0.42507963553068073 &gt;= 0.2080985447081304 + 0.21698109082255032 = 0.42507963553068073
 </pre></div>
 </div>
 <div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/ensemble/_gb.py:424: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
@@ -1211,7 +1211,7 @@ <h2>Gradient Boosting, Examples of Regression<a class="headerlink" href="#gradie
   y = column_or_1d(y, warn=True)
 </pre></div>
 </div>
-<img alt="_images/82abcfe355d89bb3d7dd4eb6e9ffe48d1f3f3511015d80687fbe0db1b8fd5999.png" src="_images/82abcfe355d89bb3d7dd4eb6e9ffe48d1f3f3511015d80687fbe0db1b8fd5999.png" />
+<img alt="_images/801d16873af4f485d9c398ac11811a7d9c4f25dd40c995cf7565bd81af8071c5.png" src="_images/801d16873af4f485d9c398ac11811a7d9c4f25dd40c995cf7565bd81af8071c5.png" />
 </div>
 </div>
 </section>
@@ -1264,13 +1264,13 @@ <h2>Gradient Boosting, Classification Example<a class="headerlink" href="#gradie
 (143, 30)
 </pre></div>
 </div>
-<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[0.93333333 0.93333333 0.93333333 0.92857143 1.         0.92857143
+<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[0.93333333 0.93333333 0.86666667 1.         1.         0.92857143
  1.         0.92857143 0.85714286 0.92857143]
-Test set accuracy with Gradient boosting and scaled data: 0.99
+Test set accuracy with Gradient boosting and scaled data: 0.97
 </pre></div>
 </div>
-<img alt="_images/e0c9e5cfc32bfe482c04b2091d5d98aa80212c0eef5e22892d16f73e93be8afc.png" src="_images/e0c9e5cfc32bfe482c04b2091d5d98aa80212c0eef5e22892d16f73e93be8afc.png" />
-<img alt="_images/796ded1719cb864b630c37e7692028ee18f0df7fb5a0ed2de15817b0c2f36c0c.png" src="_images/796ded1719cb864b630c37e7692028ee18f0df7fb5a0ed2de15817b0c2f36c0c.png" />
+<img alt="_images/90075505602c3f17740e87e303ddff0ae0ff2ec0245679468ad8fb7cb2ba3b3a.png" src="_images/90075505602c3f17740e87e303ddff0ae0ff2ec0245679468ad8fb7cb2ba3b3a.png" />
+<img alt="_images/de0019c6a0f8206c4c2b0f29eedfececf3b0ad5a07a733217a9ed5b8e4cae808.png" src="_images/de0019c6a0f8206c4c2b0f29eedfececf3b0ad5a07a733217a9ed5b8e4cae808.png" />
 <img alt="_images/45972a93ed8e1f6ed66fe9c322a65b549b39dd2c80e16d5081151b2bc713b669.png" src="_images/45972a93ed8e1f6ed66fe9c322a65b549b39dd2c80e16d5081151b2bc713b669.png" />
 </div>
 </div>