selectorgadget | Go go CSS / DOM inspection

I am trying to webscrape this website.

The content I need is available after clicking on each title. I can get the content I want if I do this for example (I am using SelectorGadget):

I'm using rvest to scrape some links from the magazine 'The Hustle'. I've used this code

I am new to Python and I am trying to webscrape this website. What I am trying to do is to get just dates and articles' titles from this website. I follow a procedure I found on SO which is as follows:

I am trying to webscrape the table from this following page (https://www.coya.com/bike/fahrrad-index-2019), namely the values the bike index for 50 german cities (if u click "Alle Ergebnisse +", you ll see all 50 cities.

I need especially some columns ("Bewertung spezielle Radwege & Qualität der Radwege", "Investitionen & QUalität der Infrastruktur", "Bewertung der Infrastruktur", "Fahrradsharing-Score", "Autofreier Tag", "Critical-Mass-Fahrrad-aktionen, "Event-Score).

This is what I tried:

i need to extract values for 80 cities from this page:

https://deutschland-studie-senioren-familie.zdf.de/senioren/

Unfortunately, the URL does not include the names of the cities, instead it has endings as follows: "district/05754.

If it would have names, I would have used:

I would like to extract the values in the table on the top right side of this Webpage:

https://www.timeanddate.de/wetter/deutschland/karlsruhe/klima

(Wärmster Monat : VALUE, Kältester Monat: VALUE, Jahresniederschlag: VALUE)

Unfortunately, if I use html_nodes("Selectorgadgets result for the specific value"), I receive the values for the table on the top of the link:

https://www.timeanddate.de/stadt/info/deutschland/karlsruhe

(The webpages are similar, if you click "Uhrzeit/Übersicht" on the top bar, you access the second page and table, if you click "Wetter" --> "Klima", you access the first page/table (the one I want to extract values from!)

I am trying to get a text from a webpage. To simplify my question, let me use @RonakShah's Stackoverflow account as an example to extract the reputation value. With 'SelectorGadget' showing "div, div", I used the following code:

We are trying to parse href attributes from the DOM of a job website. We want to get an href for each job.

We usually use CSS paths and pass those to Selenium's find_elements_by_css method.

Unfortunately, we've noticed that the browser plugin SelectorGadget had trouble providing us with a CSS path. We proceeded to use a CSS path using Google Chrome (ctrl+shift+c). Chrome could extract a path, but neither Selenium nor BeautifulSoup can work with those paths.

After many failed attempts to extract the elements using different classes and tags, we believe something is entirely wrong with either our approach or the website. We hypothesize that the desired elements are impossible to parse by Selenium and BeautifulSoup for whatever reason? Could the iframe tags in the DOM be a source of error (see this SO question)? What makes the parsing fail here, and is there a way to get around this problem? A website-related problem source would also explain why the SelectorGadget was unable to get a path in the first place. Our conclusion would be to use regular expressions to extract the href attributes that we need. This would only be a last resort solution.

For German-speakers, please note that there is a spelling error in the target elements:

No luck with BeautifulSoup:

I am attempting to scrape the this URL to get the names of the top 50 soundcloud artists in Canada.

Using SelectorGadget, I selected the artists names and it told me the path is '.sc-link-light'.

My first attempt was as follows: