FinanceDatabase

The financedatabase project uses specific compression techniques to optimize data access. Because users must download data files to access the database, the primary metric for choosing a compression method is file size (to minimize download time), balanced against local read speeds.

While testing included csv, pickle, and hdf, the project has selected CSV BZ2 as the standard compression format. This choice provides efficient loading speeds similar to Pickle (xz) while avoiding the security vulnerabilities associated with loading Pickle files.

You can convert a selection from the Finance Database directly into a FinanceToolkit object using the .to_toolkit() method. This requires an API key from FinancialModelingPrep. Once converted, you can access historical data and automatically calculate over 60 financial ratios.

import financedatabase as fd

API_KEY = "YOUR_FINANCIAL_MODELING_PREP_API_KEY"

equities = fd.Equities()

# 1. Select companies
dutch_insurance_companies = equities.select(
    country='Netherlands',
    industry='Insurance',
    market='Euronext Amsterdam',
)

# 2. Convert to toolkit
toolkit = dutch_insurance_companies.to_toolkit(api_key=API_KEY)

# 3. Perform analysis
toolkit.get_historical_data()
toolkit.ratios.collect_all_ratios()

To install the FinanceDatabase, use pip to install the package. It is recommended to use the -U flag to ensure you are using the latest version.

pip install financedatabase -U

Once installed, you can import the library in your Python scripts using the financedatabase alias.

import financedatabase as fd

This notebook provides a methodology for testing different data compression formats (CSV, Pickle, HDF) to determine the best balance between file size, write time, and read time. This is particularly useful when data must be downloaded remotely, as file size directly impacts download latency.

Key Findings:

• Pickle (xz) provides the most efficient loading times.
• CSV BZ2 is recommended as a safer alternative to Pickle to avoid potential security vulnerabilities associated with unpickling data, while maintaining similar loading performance.

If standard categorization doesn't meet your needs, use the .search() method. This allows you to filter any column using custom string matching. By default, searches are case-insensitive, but you can set case_sensitive=True. You can also filter the symbol column specifically using the index parameter.

# Search for instruments with specific keywords in summary or industry_group
equities.search(
    summary=["Robotics", "Education"],
    industry_group="Equipment",
    market='Frankfurt',
    index=".F"
)

To see the valid values for each column without loading the full database, use show_options(). You can call this on the package level to see all global options, or on a specific class instance to see options filtered by your current selection.

# Get all available options for the equities database
fd.show_options("equities")

# Get options filtered by a specific parameter (e.g., only options available in the Netherlands)
equities = fd.Equities()
equities.show_options(country='Netherlands')

# Get options for a specific selection (e.g., only industries within the Financials sector in the Netherlands)
equities.show_options(
    selection='industry',
    sector='Financials',
    country='Netherlands'
)

Use the Equities class to query the equities database. It is recommended to save the class instance to a variable to avoid reloading the data files on every query. You can use .select() to filter results by various parameters like country, industry, market, or exchange.

import financedatabase as fd

equities = fd.Equities()

# Basic selection
equities.select()

# Filtered selection
equities.select(
    country='Netherlands',
    industry='Insurance',
    market='Euronext Amsterdam',
)

# Selection using lists for multiple values
equities.select(
    country=['Netherlands', 'United States'],
    industry='Insurance',
    market=['Euronext Amsterdam', 'New York Stock Exchange']
)

The HDF (Hierarchical Data Format) can be used via pandas for efficient data storage. It requires a key to identify the dataset within the file.

Example: Saving and loading HDF data

# Saving to HDF
df.to_hdf('filename.h5', key='key', mode='w')

# Loading from HDF
pd.read_hdf('filename.h5', key='key', mode='r')

#---saving---
result_save = %timeit -n5 -r5 -o df.to_hdf(filename + '.h5', \
                                           key='key', \
                                           mode='w')
#---get the size of file---
filesize = os.path.getsize(filename + '.h5') / 1024**2
#---load---
result_read = %timeit -n5 -r5 -o pd.read_hdf(filename + '.h5', \
                                             key='key', \
                                             mode='r')