docnow | A Twitter data collection and appraisal application
kandi X-RAY | docnow Summary
kandi X-RAY | docnow Summary
The web is a big and rapidly changing place, so it can be challenging to discover what resources related to a particular event or topic are in need of archiving. Appraisal is an umbrella term for the many processes by which archivists identify records of enduring value for preservation in an archive. DocNow is an appraisal tool for the social web that uses Twitter. DocNow allows archivists to tap into conversations in Twitter to help them discover what web resources for collection and preservation. It also connects archivists with content creators in order to make the process of archving web content more collaborative and consentful. The purpose of DocNow is to help ensure ethical practices in web archiving by building conversations between archivists and the communities they are documenting.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of docnow
docnow Key Features
docnow Examples and Code Snippets
Community Discussions
Trending Discussions on docnow
QUESTION
I am using the command line tool twarc to download Twitter data as a csv. I have set up my twarc commands and they successfully execute on the command line without issue. Example command:
twarc dosomething > outputfile.jsonl
While I would like to carry out a collection process over an extended period of time, the output files become a bit too large (10+GB) after running for more than a day.
I would like to run a bash script that executes the twarc command, runs until the output file reaches a certain limit, and then starts a new file.
These questions are related...
...although I've had little luck with the translation.
Could anyone provide some insight on setting up a basic bash script to execute a command, wait until a file grows to X size, and then start again on a new file? Could take it from there...
...ANSWER
Answered 2020-Oct-02 at 20:16The tool you're looking for is aptly named split
:
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install docnow
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page