Carla-Simulato-Dueling-DDQN

Dueling Double Deep Q-Network (Dueling DDQN)

In the dueling variant of the DQN, we incorporate an intermediate layer in the Q-Network to estimate both the state value and the state-dependent advantage function. After reformulation (see ref), it turns out we can express the estimated Q-Value as the state value, to which we add the advantage estimate and subtract its mean. This factorization of state-independent and state-dependent values helps disentangling learning across actions and yields better results.

Trained a model to overtake front vehicle in carla environment taking image as input. State is formed by stacking last 4 images.

Output video link: https://drive.google.com/file/d/1q-IK11GlPLRgP1JlCBtKgLT0Z2U96_GB/view?usp=sharing

Requirements

The follow packages are required, you can install them with pip3 install [package] opencv-python
gym
gym[atari]
tensorflow
keras
scipy