fantasy-football | Fantasy Football Analysis

User Experience
I am recent engineering (Not C.S.) graduate with basic proficiency in MATLAB. I have no prior experience with Python/Jupyter. I have scoured SO and google for help but cannot find a similar issue. The code for this project is based on the following article: https://medium.com/@shahrezanjum/using-python-to-automate-fantasy-football-stats-in-madden-ff9020fc2d2d

Motivation
Madden is a NFL video game. In franchise mode, players can cooperatively play as different teams in the same league. Madden has the ability to output player statistics for this league as CSV files. CSV files are separate, and are organized in folders by week and by team. As such, this output format requires modification in order to perform data analysis. See Madden output structure here

Problem Statement
The objective is to concatenate these CSVs into a single CSV file to facilitate data analysis. Madden CSV column orders are not identical. The code I have so far has two issues: 1)The values for the first column "defCatchAllowed" is missing ONLY for the first data frame. 2)The values for the column "fullName" is missing values for every data frame after the first.

Code Strategy
Unlike the code in the link, I see 3 objectives for the code:

1. Find all CSV files for a given week.
2. Fill in blank cells with a value of zero.
3. Concatenate CSV files. (Concat can sort columns so different col orders for df's is ok.)
   

Here is the code that I have so far:

-Create DFs from CSV (starting with just 3 df, will add all teams when code works)

I am developing news app and I have converted elapsed time from now to that date but when I run code I am getting following exception in my adapter class

java.time.format.DateTimeParseException: Text '09/10/2019' could not be parsed at index 0 could not be parsed, unparsed text found at index 19

below my Adapter class

I'm currently trying to web scrape the 2018 fantasy football player rankings from espn website and import that information into a csv file. Currently my program is able to successfully scrape but it only grabs the first element for each class tag I search through. I used the soup.find_all('') method but that still doesn't seem to get the entire table. Here is my code.

I am having issues using the Yahoo Fantasy API. Using examples here and here, I have developed a pretty hacky solution to get my connection set.

First, I create an oauth1 token with the following code:

I am working with the following URL: http://www.espn.com/blog/stephania-bell/post/_/id/3563/key-fantasy-football-injury-updates-for-week-4-2

I am trying to extract the name of the blog as (stephania-bell).

I have implemented following function to extract the expected value from URL:

So I'm pulling statistics of NFL players. The table only shows max 50 rows, so I have to filter it down to make sure I don't miss any stats, which means I'm iterating through the pages to collect all the data by Season, by Position, by Team, by Week.

I figured out how the url changes to cycle through these, but the iteration process takes so long, and was thinking: we're able to open multiple webpages at one time, couldn't I be able to run these processes parallel, where each process simultaneously collects the data from each page, stores it in its temp_df, then merge them all at the end...instead of collecting one url, by one url, then merge, then next url, then merge, then next,......at a time. Meaning this iterates through 6,144 times (if I'm not iterating through the positions), but with the positions, over 36,000 iteration through.

But I'm stuck on how to implement it, or if it's even possible.

Here's the code I'm currently using. I eliminated the cycle through position to just give an idea of how its working, which for quarterbacks, the p = 2.

So it starts at season 2005 = 1, team 1 = 1, week 1 =0, then iterates all those to the last season 2016 = 12, team 32 = 33, and week 16 = 17: