Greencrab month env #21
Conversation
|
Awesome work Jim! Monthly env looks good! :) |
|
Nice work @jiangjingzhi2003 , this is exciting. I think the natural next step would be to run this through the existing pipeline comparing RL solutions to the best fixed-action-strategy solution like you were doing before. Do you want to take a go at that? Chances are this will depend on the details of the parameters. Usually with a new environment, the first thing we find is that even for the 'fixed-action' case, the action is often on the boundary of action space (i.e. set no traps ever, set maximum traps), and some adjusting of costs and benefits is required such that the control problem does not reduce to one of these limiting approximations, but it helps build our intuition for the model. |
Got it. I will try to get a constant action for the new environment. Also, do we need a "simplified version" for monthly environment like we did for normal greenCrabEnv. |
|
Ah you know I was thinking about that the other day! I think it would even
make sense to make those changes in the same monthly environment rather
than making a new "simplified" environment. (I dont think my original
decision of making a 'normal' and a 'simplified' environment was the best
for the project long-term haha.)
…On Sun 1. Dec 2024 at 12:40, jim jiang ***@***.***> wrote:
Nice work @jiangjingzhi2003 <https://github.com/jiangjingzhi2003> , this
is exciting. I think the natural next step would be to run this through the
existing pipeline comparing RL solutions to the best fixed-action-strategy
solution like you were doing before. Do you want to take a go at that?
Chances are this will depend on the details of the parameters. Usually
with a new environment, the first thing we find is that even for the
'fixed-action' case, the action is often on the boundary of action space
(i.e. set no traps ever, set maximum traps), and some adjusting of costs
and benefits is required such that the control problem does not reduce to
one of these limiting approximations, but it helps build our intuition for
the model.
Got it. I will try to get a constant action for the new environment. Also,
do we need a "simplified version" for monthly environment like we did for
normal greenCrabEnv.
—
Reply to this email directly, view it on GitHub
<#21 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AIFQIMLCUBRVXDYUXZTCZSD2DNJ25AVCNFSM6AAAAABSOVU4H2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMJQGIYDEMZRGQ>
.
You are receiving this because your review was requested.Message ID:
***@***.***>
|
Got it. Yeah, I was thinking about it as well and couldn't come up with a even more simplified version for the monthly environment. |
|
Yeah totally agreed! Maybe the only thing it's missing is having [-1, +1]
valued actions and observations, which we can modify within that same env!
…On Sun 1. Dec 2024 at 13:09, jim jiang ***@***.***> wrote:
Ah you know I was thinking about that the other day! I think it would even
make sense to make those changes in the same monthly environment rather
than making a new "simplified" environment. (I dont think my original
decision of making a 'normal' and a 'simplified' environment was the best
for the project long-term haha.)
… <#m_6930814394684895075_>
On Sun 1. Dec 2024 at 12:40, jim jiang *@*.*> wrote: Nice work
@jiangjingzhi2003 <https://github.com/jiangjingzhi2003>
https://github.com/jiangjingzhi2003 <https://github.com/jiangjingzhi2003> ,
this is exciting. I think the natural next step would be to run this
through the existing pipeline comparing RL solutions to the best
fixed-action-strategy solution like you were doing before. Do you want to
take a go at that? Chances are this will depend on the details of the
parameters. Usually with a new environment, the first thing we find is that
even for the 'fixed-action' case, the action is often on the boundary of
action space (i.e. set no traps ever, set maximum traps), and some
adjusting of costs and benefits is required such that the control problem
does not reduce to one of these limiting approximations, but it helps build
our intuition for the model. Got it. I will try to get a constant action
for the new environment. Also, do we need a "simplified version" for
monthly environment like we did for normal greenCrabEnv. — Reply to this
email directly, view it on GitHub <#21 (comment)
<#21 (comment)>>,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIFQIMLCUBRVXDYUXZTCZSD2DNJ25AVCNFSM6AAAAABSOVU4H2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMJQGIYDEMZRGQ
<https://github.com/notifications/unsubscribe-auth/AIFQIMLCUBRVXDYUXZTCZSD2DNJ25AVCNFSM6AAAAABSOVU4H2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMJQGIYDEMZRGQ>
. You are receiving this because your review was requested.Message ID: @.*
>
Got it. Yeah, I was thinking about it as well and couldn't come up with a
even more simplified version for the monthly environment.
—
Reply to this email directly, view it on GitHub
<#21 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AIFQIMPTUZRZHTHZGAV53R32DNNIJAVCNFSM6AAAAABSOVU4H2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMJQGIYTEMZQGM>
.
You are receiving this because your review was requested.Message ID:
***@***.***>
|
First version of greencrab month environment, mainly change the step function.