Skip to content

few fixes for hw2 #144

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
klensy wants to merge 2 commits into
base: 25f_ml_trainings_4
Choose a base branch
from
Open

few fixes for hw2 #144

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6b55e51
few fixes for hw2
klensy Sep 17, 2025
c8d2255
fix signatures for task1, now they synced to the ones in template_cro…
klensy Sep 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions homeworks/hw02_cross_entropy/01_crossentropy_method.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -493,7 +493,7 @@
"elite_states = [1, 2, 3, 4, 2, 0, 2, 3, 1]\n",
"elite_actions = [0, 2, 4, 3, 2, 0, 1, 3, 3]\n",
"\n",
"new_policy = update_policy(elite_states, elite_actions)\n",
"new_policy = update_policy(elite_states, elite_actions, n_states, n_actions)\n",
"\n",
"assert np.isfinite(new_policy).all(\n",
"), \"Your new policy contains NaNs or +-inf. Make sure you don't divide by zero.\"\n",
Expand Down Expand Up @@ -587,13 +587,13 @@
"\n",
"for i in range(100):\n",
"\n",
" %time sessions = [generate_session(policy) for _ in range(n_sessions)]\n",
" %time sessions = [generate_session(env, policy) for _ in range(n_sessions)]\n",
"\n",
" states_batch, actions_batch, rewards_batch = zip(*sessions)\n",
"\n",
" elite_states, elite_actions = select_elites(states_batch, actions_batch, rewards_batch, percentile)\n",
"\n",
" new_policy = update_policy(elite_states, elite_actions)\n",
" new_policy = update_policy(elite_states, elite_actions, n_states, n_actions)\n",
"\n",
" policy = learning_rate*new_policy + (1-learning_rate)*policy\n",
"\n",
Expand Down Expand Up @@ -622,9 +622,9 @@
"\n",
"In this section you will train a neural network policy for continuous state space game\n",
"\n",
"You can find full description of the environment [here](https://www.gymlibrary.dev/environments/classic_control/cart_pole/).\n",
"You can find full description of the environment [here](https://gymnasium.farama.org/environments/classic_control/cart_pole/).\n",
"\n",
"![CartPole-v0\"](https://www.gymlibrary.dev/_images/cart_pole.gif)\n",
"![CartPole-v0\"](https://gymnasium.farama.org/_images/cart_pole.gif)\n",
"\n",
"So here's how it works:"
]
Expand Down Expand Up @@ -684,7 +684,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Update the generate_session function and beat the game!"
"Update the generate_session_cartpole function and beat the game!"
]
},
{
Expand Down Expand Up @@ -772,7 +772,7 @@
"sessions_to_send = []\n",
"for session in sessions:\n",
" observations = [x.tolist() for x in session[0]]\n",
" actions = [x.item() for x in session[1]]\n",
" actions = [x.item() if hasattr(x, 'item') else int(x) for x in session[1]]\n",
" sessions_to_send.append((observations, actions))\n",
"\n",
"import json\n",
Expand Down