selenium-python | Selenium Python Bindings Documentation | Functional Testing library

this is a somewhat discussed problem, however I haven't find a solution in other posts.

I am creating an automation to submit parcels, using Selenium on Python, for this website: https://www.mondospedizioni.com

To select Country and City, I need to add an input on a text field. The text field is actually a search field, that opens a dropdown menu upon adding the key.

I fail to select the option and close the dropdown menu, thus filling the "Country"/"City" field and moving fwd.

The div I refer to (Country):

I need to automate an OTP capture for my selenium-python automation scripts, Now the otp is getting updated in the google sheet and I am fetching the same using google API using the following code. Now My script runs every hour, so in that case, I will have new OTP which is getting updated every hour. How to get the latest updated data from google sheet using python?

"otp" is my sheet name. and I am using the "range="otp!a1" for getting the value of first row and first column. So during the second run it should be "range="otp!a2"for getting the value of second row and first column and so on...

Now my goal is to capture the latest value during each execution.

I am writing a webscraping script that automatically logs into my Email account and sends a message.

I have written the code to the point where the browser clicks the "new Message" button. After that a new window (or frame?) opens where I can type in recipient address, subject and message.

I came across this problem while automating task with Selenium.

What was the code execution process?

So first of all there're two functions inside seleniumtest.py : getGateway() and restartRouter().

Let's focus on restartRouter() function as it contains code related to selenium, and getGateway() function was just function that I used in restartRouter() just like this: driver.get(getGateway()).

First of all, what was happening is following:

1. Start browser and open provided url;

2. Find elements username and password

3. Enter username and password;

4. Find Login element

5. Click Login;

6. Find another button

7. Click on another button

   ...

What is the problem?

So, the problem start occuring on step 6, just after clicking Login button (See code execution process, step 6).

After it clicks on login button, it loads new page (it doesn't open it in new tab), and after that it didn't want to work.

What have you tried to do to solve the problem

First thing that came up to mine mind was to use driver.wait(5) just after clicking on login, which didn't work. I've also tried to check if any other element on that page will result in action .click()

After that, I've tried Waits - SELENIUM WAITS ; which also didn't work. I tried two possible options: driver.implicitly_wait(10) and WebDriverWait(driver,10).until(EC.presence_of_element_located((By.XPATH,"//*[@id=\"mmManagDiag\"]")). It also gave me errors how my USB Device isn't working while using explicit waits.

None of these worked.

What worked for me?

After working around it came up to my mind to try with time.sleep(3) and suddenly it worked.

My final question So, I am wondering why Selenium couldn't find any element on that page before I used time.sleep()?

And isn't selenium waits and time.sleep() almost the same thing?

Code

I've taken a few coding subjects in uni and am trying to analyse tennis statistics by learning selenium which is completely new to me.

The page I'm using is here (https://www.atptour.com/en/scores/results-archive?year=2021) and I'm followinig a guide from this website here (https://www.scrapingbee.com/blog/selenium-python/ , https://www.scrapingbee.com/blog/practical-xpath-for-web-scraping/). The particular problem I'm having is in the second guide website under the subtitle "E-commerce product data extraction".

My Goal is to loop through the tournaments and extract the links located with the 'Results' button, but I'm having trouble as my program is just giving me an emptylist.

Background
I am trying to automate a website data input; however, I have not found a way to send the value to the field "___33_xsde_6h_input__data" for the portion of the HTML attached.
I have seen that Selenium is unable to write directly to hidden fields per the question 1, the question 2, question 3 and question 4. However I have not been able to find the specific solution for my scenario.
Code Attempted
HTML

I'm trying to start a selenium application (with python) via webdriver.Remote(), like they recomends on documentation, but when I run the code below, I get the following error message:

I am trying the code from the following URL and it seems to be working for Opera- Drive Opera with selenium python

I'm trying to scrape my Instagram DMs (direct messages). However, I don't know how to automate scrolling down the column to get the additional usernames that messaged me. I've tried this, as well as the scrolling code listed here.

However, neither work because Instagram's DM page is broken into columns. The full page itself does not need to be scrolled. Just the first column. For reference, here's a screenshot of what the Instagram DM page looks like. I blocked out my username and the usernames of people who DMed me for privacy reasons

Does anyone have any ideas on how I can modify the script that scrolls down the page to accommodate the column size?

Thank you for taking the time to read my question and help in any way you can.