ComicsApi_Bot | A comics crawler built with Scrapy

ComicsApi_Bot is a Python library. ComicsApi_Bot has no bugs, it has no vulnerabilities, it has build file available and it has low support. You can download it from GitHub.

my love for comics:especially marvel comics goes way back to the days of archie comics and the websites which provide them are bloated with 1001 ads. so i figured, why not create crawl the site and come up with an api instead and then later make an app. if you are not acquainted with scrapy please go through the [scrapy] first. scrapy is written with twisted meaning it is asynchronous hence its speed. integration of scrapy with your favorite python wsgi web framework is quite a headache. a common traceback encountered is reactornotrestartable, which stems from the underlying twisted framework. this occurs because, unlike asyncio or tornado, twisted’s eventloop/reactor cannot be restarted once stopped (the reason is a bit out of scope). so it becomes apparent that the trick to integrating scrapy and wsgi frameworks involves being able to tame twisted by either: a) embedding a crawler in a wsgi app and run it using twisted’s twist web wsgi server b) embed a crawler in a wsgi app and run it any wsgi server (example: gunicorn, uwsgi,