char-rnn-tensorflow | Training | Machine Learning library

Inspired from Andrej Karpathy Char-RNN, There is a Tensorflow implementation of char-rnn sherjilozair/char-rnn-tensorflow: Multi-layer Recurrent Neural Networks (LSTM, RNN) for character-level language models in Python using Tensorflow. I want to implement bidirectional character level language model from this code. I change the model.py and wrote a simple code:

I'm having a trained rnn that I try to use on mobile. Problem is, when I use toco to convert my .pb file to .tflite it fails with the following error message:

I've been trying to look at RNN examples documentation and roll my very own simple RNN for sequence-to-sequence by using the tiny shakespeare corpus with outputs shifted by one character. I'm using sherjilozair's fantastic utils.py to load the data (https://github.com/sherjilozair/char-rnn-tensorflow/blob/master/utils.py) but my training run looks like this...

loading preprocessed files ('epoch ', 0, 'loss ', 930.27938270568848) ('epoch ', 1, 'loss ', 912.94828796386719) ('epoch ', 2, 'loss ', 902.99976110458374) ('epoch ', 3, 'loss ', 902.90720677375793) ('epoch ', 4, 'loss ', 902.87029957771301) ('epoch ', 5, 'loss ', 902.84992623329163) ('epoch ', 6, 'loss ', 902.83739829063416) ('epoch ', 7, 'loss ', 902.82908940315247) ('epoch ', 8, 'loss ', 902.82331037521362) ('epoch ', 9, 'loss ', 902.81916546821594) ('epoch ', 10, 'loss ', 902.81605243682861) ('epoch ', 11, 'loss ', 902.81366014480591)

I was expecting a much sharper dropoff, and even after 1000 epochs it's still around the same. I think there's something wrong with my code, but I can't see what. I've pasted the code below, if anyone could have a quick look over and see if anything stands out as odd I'd be very grateful, thank you.

We are working with Tensorflow. We have a very large dataset of words, brands and so on. We are trying to generate suffixes for words. For instance, give 'real' may give 'realtor', 'really', 'realestate', ... In our dataset, each word is separated by \n.

We have tried several things so far, but it does not seem to produce adequate results. It seems to me that this is a fairly simple task that should be solved

The basic idea we are working on is adapting char-rnn-tensorflow. We have simply changed the sampling to sample (with weighted random probabilities) until a carriage return is sampled. We kept the original implementation, with rnn_decoder and sequence_loss_by_example and used rnn as model.

So far, we haven't obtained good samples, almost random ones and sometimes suffixes but without any relation to the beginning of our word (real -> realstation). It is possible that it comes from the fact that this implementation do not separate words, so we would need something more simple ?

What technologies would you use for such a task ?