coronavirus | Folding @ home COVID-19 efforts | Theme library

I have a tab delimited file that looks like this:

I am running a flask app that effectively scrapes data from https://coronavirus.data.gov.uk/ api and I have a question about formatting (trying not to be repetitive)

So the code in question takes in arguments and parameters for the data and is as follows:

I am currently trying to crawl headlines of the news articles from https://7news.com.au/news/coronavirus-sa.

After I found all headlines are under h2 classes, I wrote following code:

this code bellow :

I am writing a program in python to have a user input multiple websites then request and scrape those websites for their titles and output it. However, when the program surpasses 8 websites the program crashes every time. I am not sure if it is a memory problem, but I have been looking all over and can't find any one who has had the same problem. The code is below (I added 9 lists so all you have to do is copy and paste the code to see the issue).

I have a dataframe that has a weird format that I am having difficulty formatting it to a desired format. I just need the columns first_name, last_name, domain, Email, Verification and status but am not sure how to remove it when it is in this format.

I’m currently trying to train a Spanish to English model using yaml scripts. My data set is pretty big but just for starters, I’m trying to get a 10,000 training set and 1000-2000 validation set working well first. However, after trying for days, I think I need help considering that my validation accuracy goes down the more I train while my training accuracy goes up.

My data comes from the ES-EN coronavirus commentary data set from ModelFront found here https://console.modelfront.com/#/evaluations/5e86e34597c1790017d4050a. I found the parallel sentences to be pretty accurate. And I’m using the first 10,000 parallel lines from the dataset, skipping sentences that contain any digits. I then take the next 1000 or 2000 for my validation set and the next 1000 for my test set, only containing sentences without numbers. Upon looking at the data, it looks clean and the sentences are lined up with each other in the respective lines.

I then use sentencepiece to build a vocabulary model. Using the spm_train command, I feed in my English and Spanish training set, comma separated in the argument, and output a single esen.model. In addition, I chose to use unigrams and a vocab size of 16000

As for my yaml configuration file: here is what I specify

My source and target training data (the 10,000 I extracted for English and Spanish with “sentencepiece” in the transforms [])

My source and target validation data (2,000 for English and Spanish with “sentencepiece” in the transforms [])

My vocab model esen.model for both my Src and target vocab model

Encoder: rnn Decoder: rnn Type: LSTM Layers: 2 bidir: true

Optim: Adam Learning rate: 0.001

Training steps: 5000 Valid steps: 1000

Other logging data.

Upon starting the training with onmt_translate, my training accuracy starts off at 7.65 and goes into the low 70s by the time 5000 steps are over. But, in that time frame, my validation accuracy goes from 24 to 19.

I then use bleu to score my test set, which gets a BP of ~0.67.

I noticed that after trying sgd with a learning rate of 1, my validation accuracy kept increasing, but the perplexity started going back up at the end.

I’m wondering if I’m doing anything wrong that would make my validation accuracy go down while my training accuracy goes up? Do I just need to train more? Can anyone recommend anything else to improve this model? I’ve been staring at it for a few days. Anything is appreciated. Thanks.

!spm_train --input=data/spanish_train,data/english_train --model_prefix=data/esen --character_coverage=1 --vocab_size=16000 --model_type=unigram