spash | classic way of doing simple operations

So a disclaimer that I wasn't able to do this with Scrapy.

I'm not sure what information you're requiring about the articles but here's a couple of things to think about when scraping dynamic content driven websites.

1. How much is the website driven by javascript ?
2. Is there an API I can re-engineering HTTP requests instead of automating browser activity? 2.1) If so, do I need headers, parameters and cookies to mimic that request ?
3. Pre-rendering the page with splash
4. Last resort using selenium with scrapy
5. Using the selenium module directly in your scripts.

The reason's for going in this order is that with each one as a potential solution you're increasing the probability that your scraper is brittle and that efficiency of the scraper gets slower and slower.

Most efficient solution is to seek out an API.

Checking the website you can see it's entirely driven by javascript which increases the chances of it making AJAX requests to an API end-point. Using chrome dev tools you can see there are 5 requests made to an API https://dimsum.eu-gb.containers.appdomain.cloud/api/scholar/search

I often use the requests package to fiddle around with the API end point first. So in doing so I figured out that it really only needs the headers and your query. I assume you were looking at reading comprehension as a search so i've used that as an Example.

I will do a CURL copy of the request found in network tools and copy that into curl.trillworks.com, which converts the headers and etc into nice format.

For some reason, it's absolutely necessary to pass null in the data string to this API. However there is no null equivilent in passing a dictionary in python which is the way to be able to pass parameters in Scrapy (using the meta or cb_kwargs). I'd be interested to see others work on this to get it working in Scrapy. I may be missing something about passing the parameters in the request.

I faced same error before but not in ionic framework. The error is the same and maybe these steps can solve your error too:

• Go to "Product > Scheme > Edit Scheme > Diagnostics" and disable "Thread Sanitizer" and "Guard Malloc"
• "Product > Clean Build Folder"
• Add camera and library usage descriptions to your info.plist (if
  neccessary)
• Rebuild project
• Run

Can you remove */routers from your dependencies?

And then run npm install/yarn and expo start -c