Attempting to use uninitialized value conv2d_1/kernel

[ShaniGam]

Whenever I try to start the training I get the error:
FailedPreconditionError (see above for traceback): Attempting to use uninitialized value conv2d_1/kernel

[ckleban]

+1. I get the same thing. I assume it's because I'm using the new tensorflow and perhaps this code uses depracated items?

Traceback (most recent call last):
File "async_dqn.py", line 310, in
tf.app.run()
File "/home/ckleban/.local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "async_dqn.py", line 307, in main
train(session, graph_ops, num_actions, saver)
File "async_dqn.py", line 235, in train
session.run(graph_ops["reset_target_network_params"])
File "/home/ckleban/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 789, in run
run_metadata_ptr)
File "/home/ckleban/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 997, in _run
feed_dict_string, options, run_metadata)
File "/home/ckleban/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1132, in _do_run
target_list, options, run_metadata)
File "/home/ckleban/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1152, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value conv2d_2/bias
[[Node: conv2d_2/bias/read = IdentityT=DT_FLOAT, _class=["loc:@conv2d_2/bias"], _device="/job:localhost/replica:0/task:0/cpu:0"]]
Caused by op u'conv2d_2/bias/read', defined at:
File "async_dqn.py", line 310, in
tf.app.run()
File "/home/ckleban/.local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "async_dqn.py", line 301, in main
graph_ops = build_graph(num_actions)
File "async_dqn.py", line 173, in build_graph
s, q_network = build_network(num_actions=num_actions, agent_history_length=FLAGS.agent_history_length, resized_width=FLAGS.resized_width, resized_height=FLAGS.resized_height
)
File "/home/ckleban/gym/async-rl/model.py", line 11, in build_network
model = Convolution2D(nb_filter=32, nb_row=4, nb_col=4, subsample=(2,2), activation='relu', border_mode='same')(model)
File "/home/ckleban/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 569, in call
self.build(input_shapes[0])
File "/home/ckleban/.local/lib/python2.7/site-packages/keras/layers/convolutional.py", line 140, in build
constraint=self.bias_constraint)
File "/home/ckleban/.local/lib/python2.7/site-packages/keras/legacy/interfaces.py", line 88, in wrapper
return func(*args, **kwargs)
File "/home/ckleban/.local/lib/python2.7/site-packages/keras/engine/topology.py", line 391, in add_weight
weight = K.variable(initializer(shape), dtype=dtype, name=name)
File "/home/ckleban/.local/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py", line 320, in variable
v = tf.Variable(value, dtype=_convert_string_dtype(dtype), name=name)
File "/home/ckleban/.local/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 200, in init
expected_shape=expected_shape)
File "/home/ckleban/.local/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 319, in _init_from_args
self._snapshot = array_ops.identity(self._variable, name="read")
File "/home/ckleban/.local/lib/python2.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 1303, in identity
result = _op_def_lib.apply_op("Identity", input=input, name=name)
File "/home/ckleban/.local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
op_def=op_def)
File "/home/ckleban/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2506, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/home/ckleban/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1269, in init
self._traceback = _extract_stack()
FailedPreconditionError (see above for traceback): Attempting to use uninitialized value conv2d_2/bias
[[Node: conv2d_2/bias/read = IdentityT=DT_FLOAT, _class=["loc:@conv2d_2/bias"], _device="/job:localhost/replica:0/task:0/cpu:0"]]

[yukang2017]

@ShaniGam @ckleban
Try this one.
async_dqn.txt

[GVDubrovskiy]

I had the same error. But using the file from @yukang2017 made it work. I didn't wait until the end, though (stopped with Progress 0.02)

Thanks for sharing, @yukang2017 !

[enragedginger]

@yukang2017 can you submit a pull request with that change?

[Mageswaran1989]

@yukang2017 When I used your code, it still shows the error, however the training starts.

Does the error message have any impact on the game GUI? I am not aseeing any movements in the Game UI? Is it normal while training?

Thanks!

[yukang2017]

@enragedginger Sorry, the code is not written by me. I found it from a blog.... So I don't have right to do this with other's code.

@Mageswaran1989 In my opinion, the problem is caused by the version of python or tensorflow. What is your version?

[viluon]

I'm on Arch, latest Python 2 and deps, same issue as @Mageswaran1989

[iNomaD]

@enragedginger doesn't work for me. Still get an error

`/home/Denis/atari/async-rl/model.py:10: UserWarning: Update your `Conv2D` call to the Keras 2 API: `Conv2D(activation="relu", strides=(4, 4), filters=16, padding="same", kernel_size=(8, 8))`
  model = Convolution2D(nb_filter=16, nb_row=8, nb_col=8, subsample=(4,4), activation='relu', border_mode='same')(inputs)
/home/Denis/atari/async-rl/model.py:11: UserWarning: Update your `Conv2D` call to the Keras 2 API: `Conv2D(activation="relu", strides=(2, 2), filters=32, padding="same", kernel_size=(4, 4))`
  model = Convolution2D(nb_filter=32, nb_row=4, nb_col=4, subsample=(2,2), activation='relu', border_mode='same')(model)
/home/Denis/atari/async-rl/model.py:13: UserWarning: Update your `Dense` call to the Keras 2 API: `Dense(units=256, activation="relu")`
  model = Dense(output_dim=256, activation='relu')(model)
/home/Denis/atari/async-rl/model.py:14: UserWarning: Update your `Dense` call to the Keras 2 API: `Dense(units=3, activation="linear")`
  q_values = Dense(output_dim=num_actions, activation='linear')(model)
/home/Denis/atari/async-rl/model.py:15: UserWarning: Update your `Model` call to the Keras 2 API: `Model(inputs=Tensor("in..., outputs=Tensor("de...)`
  m = Model(input=inputs, output=q_values)
WARNING:tensorflow:From /home/Denis/anaconda3/envs/intel35/lib/python3.5/site-packages/tensorflow/python/util/tf_should_use.py:175: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02.
Instructions for updating:
Use `tf.global_variables_initializer` instead.
Doing workaround for pong or breakout
Starting thread  0 with final epsilon  0.1
Doing workaround for pong or breakout
Starting thread  1 with final epsilon  0.01
Doing workaround for pong or breakout
Starting thread  2 with final epsilon  0.1
Doing workaround for pong or breakout
Starting thread  3 with final epsilon  0.5
Doing workaround for pong or breakout
Starting thread  4 with final epsilon  0.01
Doing workaround for pong or breakout
Starting thread  5 with final epsilon  0.01
Doing workaround for pong or breakout
Starting thread  6 with final epsilon  0.01
Doing workaround for pong or breakout
Starting thread  7 with final epsilon  0.1
Traceback (most recent call last):
  File "/home/Denis/anaconda3/envs/intel35/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1327, in _do_call
    return fn(*args)
  File "/home/Denis/anaconda3/envs/intel35/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1306, in _run_fn
    status, run_metadata)
  File "/home/Denis/anaconda3/envs/intel35/lib/python3.5/contextlib.py", line 66, in __exit__
    next(self.gen)
  File "/home/Denis/anaconda3/envs/intel35/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
    pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value Variable_2
         [[Node: Variable_2/read = _MklIdentity[T=DT_FLOAT, _kernel="MklOp", _device="/job:localhost/replica:0/task:0/cpu:0"](Variable_2, DMT/_2)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "async_dqn.py", line 331, in <module>
    tf.app.run()
  File "/home/Denis/anaconda3/envs/intel35/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 48, in run
    _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  File "async_dqn.py", line 328, in main
    train(session, graph_ops, num_actions, saver)
  File "async_dqn.py", line 280, in train
    summary_str = session.run(summary_op)
  File "/home/Denis/anaconda3/envs/intel35/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 895, in run
    run_metadata_ptr)
  File "/home/Denis/anaconda3/envs/intel35/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1124, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/Denis/anaconda3/envs/intel35/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1321, in _do_run
    options, run_metadata)
  File "/home/Denis/anaconda3/envs/intel35/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1340, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value Variable_2
         [[Node: Variable_2/read = _MklIdentity[T=DT_FLOAT, _kernel="MklOp", _device="/job:localhost/replica:0/task:0/cpu:0"](Variable_2, DMT/_2)]]

Caused by op 'Variable_2/read', defined at:
  File "async_dqn.py", line 331, in <module>
    tf.app.run()
  File "/home/Denis/anaconda3/envs/intel35/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 48, in run
    _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  File "async_dqn.py", line 328, in main
    train(session, graph_ops, num_actions, saver)
  File "async_dqn.py", line 256, in train
    summary_ops = setup_summaries()
  File "async_dqn.py", line 223, in setup_summaries
    logged_epsilon = tf.Variable(0.)
  File "/home/Denis/anaconda3/envs/intel35/lib/python3.5/site-packages/tensorflow/python/ops/variables.py", line 199, in __init__
    expected_shape=expected_shape)
  File "/home/Denis/anaconda3/envs/intel35/lib/python3.5/site-packages/tensorflow/python/ops/variables.py", line 330, in _init_from_args
    self._snapshot = array_ops.identity(self._variable, name="read")
  File "/home/Denis/anaconda3/envs/intel35/lib/python3.5/site-packages/tensorflow/python/ops/gen_array_ops.py", line 1400, in identity
    result = _op_def_lib.apply_op("Identity", input=input, name=name)
  File "/home/Denis/anaconda3/envs/intel35/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
    op_def=op_def)
  File "/home/Denis/anaconda3/envs/intel35/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2630, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/home/Denis/anaconda3/envs/intel35/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1204, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

FailedPreconditionError (see above for traceback): Attempting to use uninitialized value Variable_2
         [[Node: Variable_2/read = _MklIdentity[T=DT_FLOAT, _kernel="MklOp", _device="/job:localhost/replica:0/task:0/cpu:0"](Variable_2, DMT/_2)]]

[xu-song]

@iNomaD @Mageswaran1989 try this one #26