AdamW | https : //arxiv.org/abs/1711.05101 | Machine Learning library

The import should be:

As you are importing from tqdm.notebook it means that you're using Jupyter notebook, right? If not you have to do from tqdm import tqdm.

I simplified your example code to make it really minimal, like this:

Your code is designed for an older version of the transformers library:

AttributeError: 'str' object has no attribute 'dim' in pytorch

As such you will need to either downgrade to version 3.0.0, or adapt the code to deal with the new-format output of bert.

Can you explain a little more about what you're trying to do with a model from every epoch/some number of steps? I think it already archives the model every time it gets a new best score, so I'm wondering what you want to do that can't be accomplished with that.

Edit:

It looks like AllenNLP already saves a model every epoch, but it only keeps a maximum of 2 by default. I believe you can change that by adding a checkpointer to your training config, e.g.:

Please try adding

I am also trying to do the same. The error occurs because the textaugment function t.random_swap() is supposed to work on Python string objects.

In your code, the function is taking in a Tensor with dtype=string. As of now, tensor objects do not have the same methods as Python strings. Hence, the error code.

Nb. tensorflow_text has some additional APIs to work with such tensors of string types. Albeit, it is limited at the moment to tokenization, checking upper or lower case etc. A long winded workaround is to use the py_function wrapper but this reduces performance. Cheers and hope this helps. I opted not to use textaugment in the end in my use case.

Nbb. tf.strings APIs have a bit more functionalities, such as regex replace etc but it is not complicated enough for your use case of augmentation. Would be helpful to see what others come up with, or if there are future updates to either TF or textaugment.

what does BatchNorm1d do mathematically?
try and write down the equation for the case of batch_size=1 and you'll understand why pytorch is angry with you.

How to solve it?
It is simple: BatchNorm has two "modes of operation": one is for training where it estimates the current batch's mean and variance (this is why you must have batch_size>1 for training).
The other "mode" is for evaluation: it uses accumulated mean and variance to normalize new inputs without re-estimating the mean and variance. In this mode there is no problem processing samples one by one.

When evaluating your model use model.eval() before and model.train() after.

The multiple parsers solution, as you are finding, can be error-prone. I see two alternatives:

Something like this: