You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: ch8/bg/README.md
+3-5
Original file line number
Diff line number
Diff line change
@@ -48,23 +48,21 @@ After initial training on specific pairs of stimuli, we then tested on all diffe
48
48
49
49
**Figure 1:** Data from [Frank, Seeberger & O'Reilly (2004)](#references), showing that Parkinson's Disease (PD) patients OFF their medications were more likely to learn to avoid the B stimulus that was only rewarded 20% of the time, while PD patients ON their meds learned more to choose the A stimulus that was rewarded 80% of the time. Age-matched control Seniors were more balanced in learning across both cases. These results make sense in terms of PD OFF having low dopamine and favoring D2 dopamine-dip based learning, while PD ON has elevated dopamine from the meds that also "fill in the dips", producing a bias toward D1 burst learning and away from D2 dip learning.
50
50
51
-
* Do `Run` (which will Train) and click on the `Phase` / `ActAvg` variable in the Network -- this shows the running-average activations over many trials, so you can see the effects of learning on the general activations of units in the network. You can click the `+` button next to the `Min` field at the top of the Network view (or `-` on the Max), to zoom the value display range, to see it better. You can also see a corresponding pattern in the weights into the Go and NoGo units.
52
-
53
-
You can also click on the `Weights` tab to see the Matrix Go weights. The patterns should stabilize after the 30 epochs of training. You can also `Step Trial` for a few trials with the `Act` variable shown, to see how the network is performing on specific trials -- it should correspond with what you observed with the `ActAvg` averages.
51
+
* Click on the `Weights` tab to see the Matrix Go weights. Each row represents the weights from one input unit. Do `Run`, which will Train the network. (It's best to view the `Weights` tab during the `Run` to speed things up and prevent crashing.) The weight patterns should stabilize after the 30 epochs of training. Click the `Network` tab and click on the `Phase` -> `ActAvg` variable in the Network. This shows the running-average activations over many trials, so you can see the effects of learning on the general activations of units in the network. You can click the `+` button next to the `Min` field at the top of the Network view (or `-` on the Max), to zoom the value display range, to see it better. You can also see a corresponding pattern in the weights into the Go and NoGo units. `Step Trial` for a few trials with the `Act` variable shown, to see how the network is performing on specific trials -- it should correspond with what you observed with the `ActAvg` averages.
54
52
55
53
> **Question 8.1:** Describe the patterns of activation that emerge in the `MatrixGo`, `MatrixNoGo`, and `PFCoutD` layers over learning. Briefly explain why these patterns emerged given the probabilities of dopamine bursts and dips associated with each of the different stimuli / actions (a-f). (Note that in this simulation, the reward probabilities are highest for A, next highest for B and so forth -- unlike the labels in the empirical task where B was the least rewarded action).
56
54
57
55
You should have observed that the model learned a sensible action valuation representation given the relative dopamine outcomes associated with these actions, similar to the participants in the probabilistic selection task, who were reliably able to choose the more consistently rewarded stimuli over those that were less frequently rewarded. You should also have noticed that while the matrix units encode a more continuous representation of the reward probabilities, the net output of the system reflected a threshold-like behavior that chooses any action that is has more good than bad outcomes, while avoiding those with the opposite profile.
58
56
59
57
# Simulating Parkinson's Disease and Dopamine Medications
60
58
61
-
* To simulate the reduction in dopamine present in unmedicated Parkinson's disease (PD), set the `Burst da gain` value to .25 instead of 1 -- corresponding to the roughly 75% damage of DA neurons in PD, and thus weakening the effects of dopamine bursts, while leaving the dips alone (i.e., DA levels are not prevented from decreasing to zero in PD). `Init` and `Train` the model again.
59
+
* To simulate the reduction in dopamine present in unmedicated Parkinson's disease (PD), set the `Burst da gain` value to .25 instead of 1 -- corresponding to the roughly 75% damage of DA neurons in PD, and thus weakening the effects of dopamine bursts, while leaving the dips alone (i.e., DA levels are not prevented from decreasing to zero in PD). Click on the `Weights` tab, and `Init` and `Run` the model again.
62
60
63
61
> **Question 8.2:** How do the `ActAvg` results from this model with `Burst da gain`=.25 compare to that of the "intact" network from before, with `Burst da gain`=1, in the MatrixGo and NoGo pathways, and the PFCOutD output layer? How does this correspond with the results from PD patients OFF meds, as shown in Figure 1? Recall that the PFCOutD layer reflects the net impact of the learning on action valuation, so units that have high `ActAvg` correspond to those that the system would deem rewarding on average -- you should notice a difference in how rewarding an action needs to be before the system will reliably select it.
64
62
65
63
Next, we can simulate effects of DA medication given to PD patients -- for example levodopa increases the synthesis of dopamine. In addition to increasing dopamine availability, medications also continually and directly stimulate dopamine D2 receptors (so-called D2 agonists), which has the effect of blunting the impact of any dips in dopamine (i.e., even when dopamine levels go down, the drugs will continue to occupy D2 receptors and prevent NoGo units from getting excited and learning).
66
64
67
-
* Set `Burst da gain` back to 1 to reflect an increase in dopamine availability, and `Dip da gain` to .25 (or lower for more extreme effects) to simulate the effects of D2 agonists in continuing to stimulate D2 receptors and hence blocking the effects of dips. `Init` and `Train` the model again.
65
+
* Set `Burst da gain` back to 1 to reflect an increase in dopamine availability, and `Dip da gain` to .25 (or lower for more extreme effects) to simulate the effects of D2 agonists in continuing to stimulate D2 receptors and hence blocking the effects of dips. Click on the `Weights` tab, `Init` and `Train` the model again.
68
66
69
67
> **Question 8.3:** Now how do the `ActAvg` results compare with both previous runs? How does this correspond with the results from PD patients ON meds, as shown in Figure 1?
0 commit comments