Using a saved model with HistoryWrapper

[masterkey2000]

Hello,

I have trained a tf-agent and now I want to test it in backtesting.py. I used the HistoryWrapper to wrap 3 days of data together:

my_history_length = 3

environment = wrappers.HistoryWrapper(MyTradeEnv(df, start_data, end_data), history_length=my_history_length)

The df looks like this:

        Open     High      Low    Close  Volume
0    1.21223  1.21246  1.21215  1.21227  132.32
1    1.21223  1.21226  1.21206  1.21215  200.66
2    1.21215  1.21222  1.21201  1.21204  165.59
3    1.21204  1.21217  1.21204  1.21207   32.89
4    1.21208  1.21208  1.21197  1.21201  135.67
..       ...      ...      ...      ...     ...
994  1.20428  1.20486  1.20424  1.20481  525.03
995  1.20481  1.20495  1.20464  1.20472  467.26
996  1.20471  1.20508  1.20466  1.20503  364.49
997  1.20503  1.20555  1.20500  1.20545  567.99
998  1.20546  1.20547  1.20514  1.20535  716.76

My time_step_spec.observation looks like this:

time_step_spec.observation: 
    {'price': BoundedArraySpec(shape=(3, 5), dtype=dtype('float32'), name='obs_price', minimum=-3.4028234663852886e+38,    maximum=3.4028234663852886e+38), 
     'pos': BoundedArraySpec(shape=(3, 2), dtype=dtype('float32'), name='obs_pos', minimum=0.0, maximum=1.0)}

Now I am trying to apply the strategy. It is not clear to me in which form I have to do this. My "price_data"-array and my "pos"-array have the form:

self.price_data = np.array([[0,0,0,0,0],[0,0,0,0,0],[0,0,0,0,0]], dtype='float32')
self.pos = np.array([[0, 0],[0, 0],[0, 0]], dtype='float32')

The Error message I get is:

ValueError: Could not find matching concrete function to call loaded from the SavedModel. Got:
  Positional arguments (2 total):
    * TimeStep(
{'discount': <tf.Tensor 'time_step_2:0' shape=(1,) dtype=float32>,
 'observation': {'pos': <tf.Tensor 'time_step_3:0' shape=(1, 3, 2) dtype=float32>,
                 'price': <tf.Tensor 'time_step_4:0' shape=(1, 3, 5) dtype=float32>},
 'reward': <tf.Tensor 'time_step_1:0' shape=(1,) dtype=float32>,
 'step_type': <tf.Tensor 'time_step:0' shape=(1,) dtype=int32>})
    * ()
  Keyword arguments: {}

 Expected these arguments to match one of the following 2 option(s):

Option 1:
  Positional arguments (2 total):
    * TimeStep(step_type=TensorSpec(shape=(None,), dtype=tf.int32, name='step_type'), 
    * reward=TensorSpec(shape=(None,), dtype=tf.float32, name='reward'), 
    * discount=TensorSpec(shape=(None,), dtype=tf.float32, name='discount'), 
    * observation=
    *     {'pos': TensorSpec(shape=(None, 2), dtype=tf.float32, name='observation/pos'), 
    *      'price': TensorSpec(shape=(None, 5), dtype=tf.float32, name='observation/price')})
    * ()
  Keyword arguments: {}

Option 2:
  Positional arguments (2 total):
    * TimeStep(step_type=TensorSpec(shape=(None,), dtype=tf.int32, name='time_step/step_type'), 
    * reward=TensorSpec(shape=(None,), dtype=tf.float32, name='time_step/reward'), 
    * discount=TensorSpec(shape=(None,), dtype=tf.float32, name='time_step/discount'), 
    * observation=
    *     {'price': TensorSpec(shape=(None, 5), dtype=tf.float32, name='time_step/observation/price'), 
    *      'pos': TensorSpec(shape=(None, 2), dtype=tf.float32, name='time_step/observation/pos')})
    * ()
  Keyword arguments: {}

I can't figure out where the error is here. Can someone please help me further.

[Kimtaehung123]

I understand the error, but yet don't find the solution. Your input is 'observation': {'pos': <tf.Tensor 'time_step_3:0' shape=(1, 3, 2) dtype=float32>, 'price': <tf.Tensor 'time_step_4:0' shape=(1, 3, 5) dtype=float32>},but model requires observation= * {'price': TensorSpec(shape=(None, 5), dtype=tf.float32, name='time_step/observation/price'), * 'pos': TensorSpec(shape=(None, 2), dtype=tf.float32, name='time_step/observation/pos')} , that is your input has one more dimension than needed, check second dimension in your input comes from where, this dimension should be removed.