DOCSEARCHbyAlgolia
Submit Docs

dateparser

repository·master·Indexed 25 days ago

https://github.com/scrapinghub/dateparser

A Python library designed to parse human-readable dates from HTML pages and strings in many different languages and formats, including relative dates, timestamps, and non-Gregorian calendars. It provides a primary parse() function to convert strings into datetime objects, a Settings class for configuring parsing behavior, and specialized tools like FreshnessDateDataParser for relative dates and a CLI for managing cached models.

Tokens
6.1K
Snippets
11
Records
51
Agent score
84%

What's inside dateparser

  1. Use langdetect for language detection

    master

    To improve language detection accuracy, you can use the langdetect library by passing its wrapper to the detect_languages_function parameter in dateparser.parse().

    Note: For short strings, language detection may fail. It is highly recommended to use detect_languages_function in conjunction with DEFAULT_LANGUAGES to improve results.

    pip install dateparser[langdetect]
    from dateparser.custom_language_detection.langdetect import detect_languages
import dateparser

dateparser.parse('12/12/12', detect_languages_function=detect_languages)

  3. Configure locales and regions in dateparser

    master

    You can specify the language and region for date parsing using two different methods:

    1. Using languages and region arguments: Pass a list of language codes to languages and a specific region subtag to region. For example, to use the en-IN locale, use languages=['en'] and region='IN'. To use fr-CA, use languages=['fr'] and region='CA'.

    2. Using the locales argument: If you already have the full locale code (e.g., 'en-IN'), you can pass it directly as a list to the locales argument.

  5. Configure date parsing behavior with settings

    master

    You can customize how dateparser parses strings by passing a dictionary to the settings argument in dateparser.parse() or the DateDataParser constructor.

    Note: Since version 1.0.0, providing a setting with an invalid value will raise a SettingValidationError.

  8. Use DateDataParser for efficient language detection

    master

    When parsing multiple dates from the same source, use dateparser.date.DateDataParser instead of the standard dateparser.parse function. While dateparser.parse attempts to detect the language for every call, DateDataParser caches detected languages and prioritizes them for subsequent strings, making it significantly more efficient.

    You can initialize DateDataParser with a specific list of known languages to further optimize performance and accuracy.

  9. Create language data templates for dateparser

    master

    When adding support for a new language, you must follow a specific template structure. The template is keyed by the ISO-639-1 two-letter language code (e.g., en for English).

    Key configuration fields include:

    • name: The full name of the language.
    • no_word_spacing: A boolean set to True for languages that do not use spaces between words.
    • skip: A list of words to ignore during parsing.
    • pertain: A list of words that pertain to specific date contexts.
    • monday through sunday: Lists containing the [name, abbreviation] for each day of the week.
    • january through december: Lists containing the [name, abbreviation] for each month.
    • Time units (year, month, week, day, hour, minute, second): Lists containing the [name, abbreviation] for each unit.
    • ago: A list of words that represent the concept of "ago".
    • simplifications: A list of mappings to handle complex phrases or regex replacements (e.g., mapping day before yesterday to 2 days ago).

  11. Handle Incomplete or Partial Dates

    master

    Use these settings to define how dateparser fills in missing date components:

    • PREFER_DAY_OF_MONTH: (String, default 'current') When the day is missing, use 'first' or 'last' to pick the start or end of the month.
    • PREFER_MONTH_OF_YEAR: (String, default 'current') When the month is missing, use 'first' or 'last' to pick the start or end of the year.
    • PREFER_DATES_FROM: (String, default 'current_period') When parts are missing, choose whether to prefer 'past' or 'future' relative to the current date.
    • RELATIVE_BASE: (datetime) Sets a specific datetime object as the base for interpreting relative strings (like 'tomorrow') instead of the current system time.
    • STRICT_PARSING: (Boolean, default False) If True, returns None if any of day, month, or year are missing.
    • REQUIRE_PARTS: (List of strings, default []) Ensures the result contains specific parts. Valid parts are 'day', 'month', and 'year'. Example: ['day', 'month'].

  12. Configure Date Order and Locale Preferences

    master

    Control how ambiguous dates (like 02-03-2016) are interpreted using these settings:

    • DATE_ORDER: Specifies the expected order of year, month, and day. Uses characters M, D, or Y. Defaults to MDY. Example: 'DMY' for Day-Month-Year.
    • PREFER_LOCALE_DATE_ORDER: (Boolean, default True) If True, dateparser uses the default order associated with the detected language (e.g., DMY for French). If you want to force your custom DATE_ORDER even when a language is detected, set this to False.