BeautifulSoup | python框架BeautifulSoup的应用，结合Requests抓取了极客学院网站上所有课程的基本信息

maybe you guys here can help. i’m trying to get a token in a script on a website with python beautiful soup but i’m stuck at one part. the request i make is

I'm trying to use BS4 to parse through the HTML for an about page on a youtube channel so I can scrape the number of channel views. Below is the code to scrape the channel views (located in the 'yt-formatted-string') and also the whole right column of the page. Both lines of code return either an empty list and a "None" value for the findAll() and find() functions, respectively.

I read another thread saying I may be receiving an empty list or "None" value because the page is accessing an API to get the total channel views to count and the values aren't actually in the HTML I'm parsing.

I know I could access much of this info through the Youtube API, but I want to iterate this code over multiple channels that are not my own. Moreover, I want to understand how to use BS4 to its full extent so I can replicate this process on an Instagram page or Facebook page.

Should I be using a different library that isn't BS4? Is what I'm looking to accomplish even possible?

My CODE

I am writing a program in python to have a user input multiple websites then request and scrape those websites for their titles and output it. However, when the program surpasses 8 websites the program crashes every time. I am not sure if it is a memory problem, but I have been looking all over and can't find any one who has had the same problem. The code is below (I added 9 lists so all you have to do is copy and paste the code to see the issue).

I am trying to scrape data of a match played between United and Sheffield United yesterday night in the premier league from understat.com. My goal is to fetch "shots per game". If you see understat.com, it has a match id for all the matches and I am using that match id to scrape the data using BS4 and requests. I have successfully located the class and got the raw data that I need to fetch in JSON format but it's giving me an error like "json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)". Below is my code:

I have a folder with several hundreds of .txt files that contain HTML code. All the file names and file paths are stored in a .csv file. I would like to convert the HTML code in each of the .txt file into plain text and save the file again.

I read that html2text is a python script that would fit my needs.

Could you help how I would need to proceed?

main.py

I am translating Xliff file using BeautifulSoup and googletrans packages. I managed to extract all strings and translate them and managed to replace strings by creating new tag with a translations, e.g.

The project: for a list of meta-data of wordpress-plugins: - approx 50 plugins are of interest! but the challenge is: i want to fetch meta-data of all the existing plugins. What i subsequently want to filter out after the fetch is - those plugins that have the newest timestamp - that are updated (most) recently. It is all aobut acutality... so the base-url to start is this:

I need some help in trying to web scrape laptop prices, ratings and products from Flipkart to a CSV file with BeautifulSoup, Selenium and Pandas. The problem is that I am getting an error AttributeError: 'NoneType' object has no attribute 'text' when I try to append the scraped items into an empty list.

I have the following code and the code scrapes some data from websites like Redbubble. and sometimes I scrape a lot of data and I want to know the real-time progress in the code... I tried progressbar module but I didn't get what I want....