scrape-twitter | Scrape any twitter page using Selenium and BeautifulSoup | Scraper library

To remove the b's, you'd want to do something like:

str_tweet = tweet_text.decode('utf-8')

To get rid of the hyperlinks at the end you could do something like this, which is quick and dirty:

only_tweet = str_tweet.split('https://')[0]

And then of course change your write statement to point to the new variable. This will result in output like:

'Van crash in south-east Iran kills 28 Afghan nationals'

instead of

b'Van crash in south-east Iran kills 28 Afghan nationalshttps://bbc.in/2qcsg9P\xc2\xa0'