advanced-web-scraping-tutorial | Zipru scraper developed in the Advanced Web | Scraper library

You can't achieve what you want because Scrapy uses asynchronous processing.

In theory you could use approach partially suggested in comment by @Paulo Scardine, i.e. raise an exception in parse_lookup. For it to be useful, you would then have to code your spider middleware and handle this exception in process_spider_exception method to log back in and retry failed requests.

But I think better and simpler approach would be to do the same once you detect the need to login, i.e. in parse_lookup. Not sure exactly how CONCURRENT_REQUESTS_PER_DOMAIN works, but setting this to 1 might let you process one request at time and so there should be no failing requests as you always log back in when you need to.