Hi bamattsson,

[shr-2311]

Hi Bamattsson,
Thanks for the MrRNN code. It was very useful.
I am currently training the model.

How to we test the model with sample dialogues and generate the response? It would be of great help if you could guide me with this.

Thanks in advance

[nayansinghal]

Hi,

We are trying to run your code but loss is neither increasing nor decreasing at all. We have tried a bunch of scenarios:

1. Remove the gradient clipping and using optimizer.minimize() by using learning rate = 100. Since we are not using gradient clipping and learning rate is too high, loss should shoot up but it's neither increasing and decreasing.

Would it be possible for you to point out the problem ?It would be of great help if you could guide me with this.

[bamattsson]

Hi,

Sorry for late response on this. @shr-2311 fun that you've tried it out and that it worked. I have only implemented the MrRNN in training mode and not in generating mode.

So you would have to implement that part. If you don't know how to start on that I would suggest you to look for some other generating decoder on GitHub (for a seq2seq model for example) and try to combine the two codes. Shouldn't be too difficult.

[bamattsson]

@nayansinghal okey, hmm it worked last time I run it. Did you follow the README when running it?

If you want to debug it I've implemented a test data set for that purpose which can be used instead for the full ubuntu data set. Start with that one is my suggestion :)

[nayansinghal]

It worked. The problem was we were using latest version of tensorflow and python and that's why loss was not decreasing.

[bamattsson]

@tysa13 fun that you found this piece of code :)

The code in this repository is only built for training mode, and not generative mode. However, if I wanted to extend it for generative mode I would've done the following steps:

• 1. Update the code to TF 1.4 and make sure that it still runs properly (the same decreasing loss as before). You'll probably have to change a few functions, but shouldn't be too difficult.
• 2. Change the _build_decoders function to use tf.conrib.seq2seq, and the dynamic_decode function. This guide explains the basics. Make first sure that it works in training mode (by using TrainingHelper), and gives the same results (decreasing loss) as before. This shouldn't be difficult.
• 3. When it works in training mode, add inference mode as well (by using GreedyEmbeddingHelper). This will probably be the most difficult step since you need to first train it in training mode, and then reload the graph in inference mode. There should be some example code out there you can get some help from.

Good luck!

[kells94]

@bamattsson ,thanks for being awesome ;)

[bamattsson]

@kells94 Thank you! Great if you managed to use it 😄