DOCSEARCHbyAlgolia
Submit Docs

vaastav/fantasy-premier-league

repository·master·Indexed 23 days ago

https://github.com/vaastav/fantasy-premier-league

A library and dataset of comprehensive Fantasy Premier League (FPL) statistics. It provides player overview stats, gameweek-specific data, historical performance records, and advanced metrics including the ICT Index and expected stats (xG, xA, xGC). The repository includes CSV files for players, teams, and fixtures, as well as advanced data from understat.com. It also provides a teams_scraper.py script for downloading specific team data and guidance on handling lookahead bias in xP columns for ML developers.

Tokens
3.6K
Snippets
7
Records
15
Agent score
33%

What's inside vaastav-fantasy-premier-league

  1. Interpret FPL performance metrics

    master

    The dataset includes several proprietary and standard metrics:

    ICT Index (Proprietary):

    • Influence: Impact on single matches via goals, assists, and other contributions.
    • Creativity: Chance creation through passing and crossing.
    • Threat: Scoring threat based on shots and positions.

    BPS (Bonus Points System):

    • A scoring system used to determine bonus points (1-3) awarded to top performers in each match.

    Expected Stats:

    • xG (Expected Goals): Quality of chances created.
    • xA (Expected Assists): Passes that led to shots.
    • xGI (Expected Goal Involvements): xG + xA combined.
    • xGC (Expected Goals Conceded): Expected goals conceded based on shots faced.

  2. Handle potential lookahead bias in the xP column

    master

    The xP (Expected Points) column in gws/merged_gw.csv is sourced from the FPL bootstrap-static API's ep_this field. Because the scraper runs after each gameweek ends, the xP value may contain information that was not available to managers before the deadline (post-match updates).

    Warning for ML Developers: Using xP unshifted as a feature to predict same-GW total_points can cause severe lookahead bias. To mitigate this, you should either:

    1. Apply .shift(1) within each element group.
    2. Exclude the xP column entirely.

  3. Understand the FPL Data Structure

    master

    The data folder contains historical and current season data organized by season. The directory structure is as follows:

    • season/cleaned_players.csv: Overview statistics for the entire season.
    • season/gws/gw_number.csv: Gameweek-specific statistics for a particular season.
    • season/gws/merged_gws.csv: A single file containing GW-by-GW statistics for every player in that season.
    • season/players/player_name/gws.csv: GW-by-GW statistics for a specific player.
    • season/players/player_name/history.csv: Statistics regarding a specific player's performance in prior seasons.

  4. Understand the ICT Index and Expected Stats (xG)

    master

    The dataset includes proprietary and advanced performance metrics:

    ICT Index: Measures performance through three components:

    • influence: Impact on matches.
    • creativity: Chance creation.
    • threat: Goal threat.

    Expected Stats (xG/xA):

    • expected_goals: xG (Expected goals).
    • expected_assists: xA (Expected assists).
    • expected_goal_involvements: Combined xG + xA.
    • expected_goals_conceded: xGC (Expected goals conceded).

  7. Access FPL data directly in Python using pandas

    master

    You can programmatically access the CSV files in this repository by using the raw GitHub URL with the pandas library. This allows you to bypass the GitHub UI and load data directly into a DataFrame.

    import pandas as pd

# URL of the CSV file (example)
url = "https://raw.githubusercontent.com/vaastav/Fantasy-Premier-League/master/data/2023-24/gws/merged_gw.csv"

# Read the CSV file into a pandas DataFrame
df = pd.read_csv(url)

  8. Reference the cleaned_players.csv data dictionary

    master

    The cleaned_players.csv file is a simplified version of players_raw.csv containing only key columns for easier analysis. It includes essential metrics like names, goals, assists, total points, minutes, ICT scores, and a calculated value_per_m field.

    | Column | Type | Description |
|--------|------|-------------|
| `first_name` | str | Player's first name |
| `second_name` | str | Player's surname |
| `goals_scored` | int | Goals scored this season |
| `assists` | int | Assists this season |
| `total_points` | int | Total FPL points |
| `minutes` | int | Minutes played |
| `goals_conceded` | int | Goals conceded |
| `creativity` | str | Creativity ICT score |
| `influence` | str | Influence ICT score |
| `threat` | str | Threat ICT score |
| `bonus` | int | Bonus points |
| `bps` | int | BPS total |
| `ict_index` | str | Combined ICT index |
| `clean_sheets` | int | Clean sheets |
| `red_cards` | int | Red cards |
| `yellow_cards` | int | Yellow cards |
| `selected_by_percent` | str | Ownership percentage |
| `now_cost` | int | Current price (in £0.1m) |
| `element_type` | str | Position (GK/DEF/MID/FWD) |
| `value_per_m` | float | Points per £1m (total_points / (now_cost/10)) |

  10. Understand teams.csv data structure

    master

    The teams.csv file contains information about the 20 Premier League teams.

    Key Fields:

    • id: Team ID (1-20).
    • name: Full team name.
    • short_name: 3-letter abbreviation.
    • strength: Overall team strength rating.
    • strength_attack_home / strength_defence_home: Attacking and defensive strength for home games.
    • strength_attack_away / strength_defence_away: Attacking and defensive strength for away games.
    | Column | Type | Description |
|--------|------|-------------|
| `id` | int | Team ID (1-20) |
| `code` | int | Internal team code |
| `name` | str | Full team name |
| `short_name` | str | 3-letter abbreviation |
| `strength` | int | Overall team strength rating |
| `strength_overall_home` | int | Home strength |
| `strength_overall_away` | int | Away strength |
| `strength_attack_home` | int | Home attacking strength |
| `strength_attack_away` | int | Away attacking strength |
| `strength_defence_home` | int | Home defensive strength |
| `strength_defence_away` | int | Away defensive strength |

  11. Understand Understat advanced data

    master

    Advanced expected goals data from understat.com is available in the understat/ directory.

    Files:

    • understat_player.csv: Overview of all EPL players with Understat metrics.
    • Player Match Files (<player_name>_<id>.csv): Match-by-match Understat data for individual players.
    • id_dict.csv: Mapping between Understat IDs and FPL IDs.

    Key Understat Metrics:

    • xG: Expected goals.
    • xA: Expected assists.
    • xGChain: Involvement in goal-scoring chains.
    • xGBuildup: Contribution to build-up play.
    • npxG: Non-penalty expected goals.
    | Column | Type | Description |
|--------|------|-------------|
| `id` | int | Understat player ID |
| `player_name` | str | Player name |
| `games` | int | Games played |
| `time` | int | Minutes played |
| `goals` | int | Goals scored |
| `xG` | float | Expected goals |
| `assists` | int | Expected assists |
| `shots` | int | Total shots |
| `key_passes` | int | Key passes |
| `yellow_cards` | int | Yellow cards |
| `red_cards` | int | Red cards |
| `position` | str | Position(s) played |
| `team_title` | str | Team name |
| `npg` | int | Non-penalty goals |
| `npxG` | float | Non-penalty xG |
| `xGChain` | float | xG Chain (involvement in goal-scoring chains) |
| `xGBuildup` | float | xG Buildup (contribution to build-up play) |

  12. Understand Gameweek (gw*.csv) data structure

    master

    Gameweek files contain individual performance data for all players for a specific round. The data is categorized into Player Info, Match Context, Performance, ICT & Expected Stats, Additional GW Stats, Defensive Stats, and Manager Points (for 2024-25+).

    Key Fields:

    • element: Player FPL ID.
    • total_points: FPL points scored in this GW.
    • xP: Expected points (scraped from FPL's ep_this field).
    • value: Player price in £0.1m units.

    Important Caveat on xP: xP is scraped after each gameweek has ended. Because FPL's update cadence is undocumented, these values may reflect post-match information rather than the pre-match predictions seen by managers. If using xP for Machine Learning, treat it as potentially post-match by applying shift(1) within each element group or dropping the column entirely.

    | Column | Type | Description |
|--------|------|-------------|
| `name` | str | Player's full name |
| `position` | str | Position (GK/DEF/MID/FWD) |
| `team` | str | Team name |
| `element` | int | Player FPL ID |