vpn | OpenVPN docker compose yml | Continuous Deployment library

I'm doing some scraping, but as I'm parsing approximately 4000 URL's, the website eventually detects my IP and blocks me every 20 iterations.

I've written a bunch of Sys.sleep(5) and a tryCatch so I'm not blocked too soon.

I use a VPN but I have to manually disconnect and reconnect it every now and then to change my IP. That's not a suitable solution with such a scraper supposed to run all night long.

I think rotating a proxy should do the job.

Here's my current code (a part of it at least) :

I'm running the code below as part of a Celery task.

When trying to install the python package datatable, I get the following error:

I am using Spring Boot 2.4.4 and Spring Data Cassandra dependency to connect to the Cassandra database. During the application startup, I am getting a DriverTimeout error (I am using VPN).

I have gone through all the Stack Overflow questions similar to this and none of them worked for me. I have cross-posted the same question on the Spring Boot official page here.

I used below configuration properties below -

I need to secure/encrypt exchanged data between app and database and I've thought on:

1. SSL/TLS

2. VPN (Wireguard)

What option do you suggest for PROD env?

AWS Transfer Family supports integration with AD Connector (https://docs.aws.amazon.com/directoryservice/latest/admin-guide/ad_connector_app_compatibility.html). As far as I understand, connectors are deployed in vpn-linked subnets that allows them to proxy calls to an on-premise Active Directory.

What exactly happens (what resources are created/updated under the hood) when I select AD connector as the authenticator for AWS Transfer? I'm specifically curious as to what changes are made in VPC to allow this integration.

I have AZ VM window which installed MSSQL I created cluster in AZ K8S, then create the pod with image - application run with embedded tomcat using MSSQL connect to private IP of the above VM. The container in the pod start with error: can not connect to that private IP of MSSQL I can access to that private IP from my local machine (using VPN), so is there any way/config to make pod can connect to that VM using private IP ? since it's same infrastructure, I dont why it cannot connect (I am newbie with Azure) Thanks alot

I am using below command for mount file on windows azure vm :

After setting up my device with Burpsuite. Im able to capture and intercept request from the mobile browser (Chrome) and only "some request" from my application. I wonder why are some request can't be capture in the HTTP history?

**Note that after setting up the proxy in wifi, my device is showing LIMITED CONNECTION. However I can still have access to internet when surfing website in Mobile Chrome, but some apps (Youtube, Facebook) is totally losing internet access.

Is that the reason why some of my request can't be capture?

Setup works including:

1. IP:proxy
2. cacert installation (VPN & APPS) + (WIFI)

Some info

1. Im using 8082 proxy
2. Android 10

After some debugging, I found out that my application is returning Network Error while firing some requests (those which failed to capture). Though this could be the reasoning of CACERT / proxy set, but I still expect the http request to be appear in burp? Why is not appearing?