tor-privoxy | Setup rotating proxies with Tor and Privoxy | Proxy library

In short:

To get the latest list of available packages, use the update command.

it is similar to the Debian apt-get update that you do before apt-get install my_package.

from https://wiki.alpinelinux.org/wiki/Alpine_Linux_package_management#Update_the_Package_list:

extract

Update the Package list

Remote repositories change as packages are added and upgraded. To get the latest list of available packages, use the update command. The command downloads the APKINDEX.tar.gz from each repository and stores it in the local cache, typically /var/cache/apk/, /var/lib/apk/ or /etc/apk/cache/.

apk update

Tip: If using remote repositories, it is a good idea to do an update just before doing an add or upgrade command. That way you know you are using the latest software available.

I have solved my problem in Ubuntu 18.04 using the suggestion given by link

Following the structure of the Aquarium project as suggested by paul trmbrth, I found that it is essential to name the .ini file default.ini, not proxy.ini (otherwise it doesn't get 'picked up' automatically). I managed to get the scraper to work in this way (cf. my self-answer to How to use Scrapy with both Splash and Tor over Privoxy in Docker Compose).

Following the Aquarium template project (https://github.com/TeamHG-Memex/aquarium), I found that the trick is to make Splash use Tor, not the spider directly.

My adapted project has the following structure:

This isn't really enough for a proper answer, but I can't comment, so...

Things I would check:

1. You are using Tor Browser and say when you click the download button it gives you a download choice box. For me on chrome, it opens "https://www.apkmirror.com/apk/google-inc/youtube/youtube-12-19-56-release/youtube-12-19-56-android-apk-download/download/" (note the /download/ at the end) where you wait for a few seconds. Maybe try scraping this?
   1. If what you mainly want is the download, you can try scraping through the elements of said /download/ page, or possibly even autogenerate the links yourself based on the post ID, for example:

We know that the class of the page is:

And therefore the postid is 215041.

So then we can use the link found on the download page here

To steal it directly from https://www.apkmirror.com/wp-content/themes/APKMirror/download.php?id=215041

But... if we try this with another link, it fails, giving us 403 Forbidden. So likely there is something going on with a cookie or a referrer. I noticed _gid was the only cookie that changed, but that doesn't mean it's the culprit.

So maybe you will need some middleware for that.