kimuraframework | modern web scraping framework written in Ruby | Crawler library

Kimurai requires Ruby version >= 2.5.0. Supported platforms: Linux and Mac OS X.
If your system doesn't have appropriate Ruby version, install it:
Install Kimurai gem: $ gem install kimurai
Install browsers with webdrivers:
You can automatically setup required environment for Kimurai on the remote server (currently there is only Ubuntu Server 18.04 support) using $ kimurai setup command. setup will perform installation of: latest Ruby with Rbenv, browsers with webdrivers and in additional databases clients (only clients) for MySQL, Postgres and MongoDB (so you can connect to a remote database from ruby). To perform remote server setup, Ansible is required on the desktop machine (to install: Ubuntu: $ sudo apt install ansible, Mac OS X: $ brew install ansible). It's recommended to use regular user to setup the server, not root. To create a new user, login to the server $ ssh root@your_server_ip, type $ adduser username to create a user, and $ gpasswd -a username sudo to add new user to a sudo group.
--ask-sudo pass this option to ask sudo (user) password for system-wide installation of packages (apt install)
--ssh-key-path path/to/private_key authorization on the server using private ssh key. You can omit it if required key already added to keychain on your desktop (Ansible uses SSH agent forwarding)
--ask-auth-pass authorization on the server using user password, alternative option to --ssh-key-path.
-p port_number custom port for ssh connection (-p 2222)
--repo-url provide custom repo url (--repo-url git@bitbucket.org:username/repo_name.git), otherwise current origin/master will be taken (output from $ git remote get-url origin)
--repo-key-path if git repository is private, authorization is required to pull the code on the remote server. Use this option to provide a private repository SSH key. You can omit it if required key already added to keychain on your desktop (same like with --ssh-key-path option)