Vrdoljak-Asn4-try2 | hw4 # s3 bucket # http

Vrdoljak-Asn4-try2 is a Python library. Vrdoljak-Asn4-try2 has no bugs, it has no vulnerabilities and it has low support. However Vrdoljak-Asn4-try2 build file is not available. You can download it from GitHub.

hw4 #s3 bucket: ###design architecture: my data collection script generates all relevant combinations of hashtags (37 total) and days (30 total), then generates a url based search for each, for a total of 1,110 search queries. then, it proceeds to simulate scrolling to the bottom of the page, using the selenium library’s firefox webdriver, 2,500 times per search. this gives a potential total of 2,775,000 pages of tweets. (pages, as displayed in a browser, not api “pages.”). the actual number was lower due to web-based rate limits and several hashtags having next to no tweets except on select days (non-english speaking, or at least tweeting, countries whereas this search was limited to only english language tweets.) the final number (still going, as i write this) should be around 400k. given a number of limitations, including needing my computer at times for isvc, i wrote my script to stored completed hashtag/date combinations into a separate file (called done.txt), and check if completed on each pass. this allows me to stop (intentionally or otherwise) my process, and continue where i left off. additionally, i make ample use of try/except methods to avoid corrupting any files with multiple potential ungraceful shutdowns. to add another layer of resilience to the process, i prep the data to write to a file, and only briefly open, write, then immediately close the file to avoid a corrupted 400mb csv