Running the model
You have Python and Tensorflow installed, you have a folder on your computer that contains the neural network and jour project folder with some training data. Great, lets’ get started.
Open a Terminal window
And navigate to the folder containing the model (e.g.
$ cd ~/path/to/your-folder/tensorflow).
The program will first ask you to type in the name of your current project folder. After that, it will ask you wether you want to train or play the model.
All .txt files from the input folder will be opened and used for training. During the process, you can see how the network improves its predictions.
By default, training will run for 500 epochs, but you don’t have to wait that long, you can quit anytime by typing
Unfortunately it is not possible to continue training from a existing checkpoint. So asure yourself if you really want to stop the training half way. But you can pause the training by switching your computer into stand by mode.
The preview sequence, loss and accuracy calculations as well as regularly generated text blocks give you an impression on how the training progresses.
After every 3rd batch, a checkpoint file of the current progress is saved to
my-new-project/checkpoints. They are named usgin the following pattern
YYYYMMDD-HHMMSS-(number of training sequences). Every checkpoint consists of 3 files:
.data-00000-of-00001 as well as ther is a
checkpoints file containing a list of all checkpoints. You should not rename, move or partially delete checkpoints files if you plan to use any of them.
Your Computer will get hot and use a lot of power, so remove it from any fabric enclosure and attatch to power supply.
Let it work 🏋️♀️🏋️♂️
Depending on your input, we can easily let this run for 1-2 hours. During that time, let’s learn a little bit more about why it is interesting to do all that and what’s is happening behind the scenes.
If you open rnn.py in your text editor, the file starts with an so called dictionairy of values that you can change to adapt your models behaviour to your specific project.
sequence_length: 30 The string length of a training sequence If you are training on poetry, where rhyme and the length of lines is really importat, increase a little bit, e.g. 40-50. batch_size: 200 Training sequences inside one batch (200) The size of one batch is then sequence_length*batch_size, which has to be notably lower than the amount of text input that you provide. In other words, bring more text or decrease batch_size. validation: True Wether validation is switched on. Slows down training process epochs: 500 Number of training epochs.
output_length: 10000 Length of text to be produced when playing top_n: 3 Number of possibilities that are involved in the prediction. 1 = only the highest scoring possibility makes it, danger of repeating input 2 or 3 = allows for some variation 10 = might become rubbish or non-language again This value is used for text generation during training and play }