FBRef | Scrapes data from the FBRef website

When I am scraping a table from a website, it is missing the bottom 5 rows of data and I do not know how to pull them. I am using a combination of BeautifulSoup and Selenium. I thought that they were not loading, so I tried scrolling to the bottom with Selenium, but that still did not work.

Code trials:

Using: Python in Google Collab Thanks in Advance: I have run this code on other data I have scraped FBREF, so I am unsure why it's happening now. The only difference is the way I scraped it. The first time I scraped it: url_link = 'https://fbref.com/en/comps/Big5/gca/players/Big-5-European-Leagues-Stats'

The second time I scraped it:

url = 'https://fbref.com/en/comps/22/stats/Major-League-Soccer-Stats'

html_content = requests.get(url).text.replace('', '')

df = pd.read_html(html_content)

I then convert the data from object to float so I can do a calculation, after I have pulled it into my dataframe:

dfstandard['90s'] = dfstandard['90s'].astype(float) dfstandard['Gls'] = dfstandard['Gls'].astype(float)

I look and it shows they are both floats:

10 90s 743 non-null float64

11 Gls 743 non-null float64

But when I run the code that as worked previously:

dfstandard['Gls'] = dfstandard['Gls'] / dfstandard['90s']

I get the error message "TypeError: '<' not supported between instances of 'str' and 'int'"

I am fairly new to scraping, I'm stuck and don't know what to do next.

The full error message is below:

I am scraping https://fbref.com/en/squads/12192a4c/Greuther-Furth-Stats with Beautiful Soup and Selenium which worked fine until suddenly some special chars are not displayed properly anymore. Here's a screenshot how it's displayed now:

I am using:

• Chromium (Version 96.0.4664.110 (Official Build) for Linux Mint (64-bit))
• Chromedriver for Chrome 96 from https://chromedriver.chromium.org/downloads

Any idea how to solve it? I already cleared cache in Chromium.

https://fbref.com/en/squads/0cdc4311/Augsburg-Stats provides buttons to transform a table to csv, which I would like to scrape. I click the buttons like

Trying to click a button with Selenium, but I keep getting an error:

NoSuchElementException: Message: no such element: Unable to locate element: {"method":"link text","selector":"AGREE"}

Here's the button I am trying to click.

I assume, the popup is loaded lazy. I found some sources, but could not make it work. Here's my code

I created a function to scrape the fbref.com website. Sometimes in this website or in others that I'm trying to scrape I receive a timeout error. I read about it and it is suggested to include a sys.sleep between the requisitions or a purrr:slowly. I tried to include inside the map but I could not. How can I include a 10 seconds gap between each requisition inside the map (It would be 7 requisitions and 6 intervals of 10 seconds). Thanks in advance and if I do not include something please inform me!

I am trying to get 3 tables from a particular website but only the first two are showing up. I have even tried get the data using BeautifulSoup but the third seems to be hidden somehow. Is there something I am missing?

I'm using the function importhtml() on my Google Sheets : =IMPORTHTML("https://fbref.com/fr/comps/13/Statistiques-Ligue-1";"table";3)

The data are imported but some data are displayed "01.08" and the value is a date. The other values are ok if they contains big number like 1.93. How it's possible to change that and have only numbers and not displayed that value as a date ? I try to change the format of the cell but the value became a number like 44455.

This is a screen of what I have

How can I have the value as a number so to display 1.08 and not 01.08 ( for Google SHeets this is a date ) Thanks a lot in advance

I am trying to create a list of all football teams/links from any one of a number of tables within the base URL: https://fbref.com/en/comps/10/stats/Championship-Stats

I would then use the link from the href to scrape each individual team's data. The href is embedded within the th tag as per below