gdeltdoc Python Client

All search queries are constructed using a gdeltdoc.Filters object. You must provide either a date range (start_date and end_date) or a timespan.

Note on Dates: start_date and end_date can be strings in YYYY-MM-DD format or UTC datetime objects. Using datetime allows for second-level granularity. The API officially supports the most recent 3 months of articles; older requests are not guaranteed to return data.

from gdeltdoc import Filters, near, repeat

f = Filters(
    start_date = "2020-05-01",
    end_date = "2020-05-02",
    num_records = 250,
    keyword = "climate change",
    domain = ["bbc.co.uk", "nytimes.com"],
    country = ["UK", "US"],
    theme = "GENERAL_HEALTH",
    near = near(10, "airline", "carbon"),
    repeat = repeat(5, "planet")
)

Install the GDELT 2.0 Doc API client via pip from PyPI.

pip install gdeltdoc

To use the client, instantiate GdeltDoc and create a Filters object to define your search criteria. You can then use article_search(filters) to get a list of articles or timeline_search(mode, filters) to get temporal data.

from gdeltdoc import GdeltDoc, Filters

f = Filters(
    keyword = "climate change",
    start_date = "2020-05-10",
    end_date = "2020-05-11"
)

gd = GdeltDoc()

# Search for articles matching the filters
articles = gd.article_search(f)

# Get a timeline of the number of articles matching the filters
timeline = gd.timeline_search("timelinevol", f)

The article_search(filters) method returns a pandas DataFrame containing news articles matching the provided Filters.

DataFrame Columns:

• url
• url_mobile
• title
• seendate
• socialimage
• domain
• language
• sourcecountry

The timeline_search(mode, filters) method allows you to retrieve temporal data using one of five available modes:

• timelinevol: A timeline of the volume of news coverage as a percentage of total GDELT monitored articles.
• timelinevolraw: Similar to timelinevol, but returns the actual number of articles and a total instead of a percentage.
• timelinelang: Breaks down the total articles by published language. Each language is a separate column in the resulting DataFrame.
• timelinesourcecountry: Breaks down the total articles by the country of publication. Each country is a separate column in the resulting DataFrame.
• timelinetone: A timeline of the average tone of the news coverage (refer to GDELT documentation for tone metric details).

Use these helper functions to construct complex text requirements within a Filters object.

• near(distance, word1, word2, ...): Finds articles where words are within a certain distance of each other.
• multi_near(list_of_tuples, method="AND"): Allows multiple proximity restrictions.
• repeat(count, word): Finds articles where a word is repeated at least count times.
• multi_repeat(list_of_tuples, method="AND"): Allows multiple repetition restrictions.

# Examples of near and repeat usage
near_filter = near(5, "airline", "climate")

multi_near_filter = multi_near([
    (5, "airline", "crisis"), 
    (10, "airline", "climate", "change")
], method="AND")

repeat_filter = repeat(3, "environment")

multi_repeat_filter = multi_repeat([("airline"), ("airport")], "AND")