csv2ofx

repository·main·Indexed 42 days ago

https://github.com/reubano/csv2ofx

A Python-based tool and library for converting CSV transaction files into OFX (Open Financial Exchange) and QIF (Quicken Interchange Format) files, facilitating data import into financial accounting software like GnuCash or Moneydance. Version 0.34.2 provides both a CLI and a programmatic API for custom mapping and transaction processing.

Tokens
7.7K
Snippets
26
Records
28
Agent score
52%

What's inside csv2ofx

  1. Create a custom mapping for csv2ofx

    main

    To handle CSV files with non-standard field names, you must create a mapping object. A mapping is a dictionary where keys are OFX/QIF attributes and values are functions (or callables) that extract the value from a CSV record (a dictionary representing a row).

    Required Attributes:

    • account: Transaction account
    • date: Transaction date
    • amount: Transaction amount

    Optional Attributes:

    • desc: Transaction description
    • payee: Transaction payee
    • notes: Transaction notes
    • check_num: Check or transaction number
    • id: Transaction ID
    • bank: Bank name
    • account: Transaction account type (e.g., checking, savings)
    • account_id: Transaction account ID
    • type: Transaction type (CREDIT or DEBIT)
    • balance: Account balance
    • class: Transaction class

    Example Mapping (using itemgetter):

    from operator import itemgetter
    
    mapping = {
        'bank': 'BetterBank',
        'account': itemgetter('Account'),
        'date': itemgetter('Date'),
        'amount': itemgetter('Amount')
    }

    Example Mapping (using complex logic):

    mapping = {
        'account': lambda r: r['Details'].split(':')[0],
        'date': lambda r: '%s/%s/%s' % (r['Month'], r['Day'], r['Year']),
        'amount': lambda r: r['Amount'] * 2,
        'first_row': 1,
        'last_row': 10,
        'filter': lambda r: float(r['Amount']) > 10,
    }

    New mappings should be placed in the csv2ofx/mappings folder.

    from operator import itemgetter
    
    mapping = {
        'bank': 'BetterBank',
        'account': itemgetter('Account'),
        'date': itemgetter('Date'),
        'amount': itemgetter('Amount')}
  2. Use csv2ofx as a Python library (OFX)

    main

    To use csv2ofx within a Python script to generate OFX files, follow this workflow:

    1. Initialize an OFX object with a mapping.
    2. Read the CSV records (e.g., using meza.io.read_csv).
    3. Generate groups, transactions, and clean them using the OFX instance methods.
    4. Use utils.gen_data to prepare the data.
    5. Chain the header, body, and footer to create the final content stream.
    import itertools as it
    
    from meza.io import read_csv, IterStringIO
    from csv2ofx import utils
    from csv2ofx.ofx import OFX
    from csv2ofx.mappings.default import mapping
    
    ofx = OFX(mapping)
    records = read_csv('path/to/file.csv', has_header=True)
    groups = ofx.gen_groups(records)
    trxns = ofx.gen_trxns(groups)
    cleaned_trxns = ofx.clean_trxns(trxns)
    data = utils.gen_data(cleaned_trxns)
    content = it.chain([ofx.header(), ofx.gen_body(data), ofx.footer()])
    
    for line in IterStringIO(content):
        print(line)
  3. Use csv2ofx as a Python library (QIF)

    main

    To use csv2ofx within a Python script to generate QIF files, follow this workflow:

    1. Initialize a QIF object with a mapping.
    2. Read the CSV records.
    3. Generate groups, transactions, and clean them using the QIF instance methods.
    4. Use utils.gen_data to prepare the data.
    5. Chain the body and footer to create the final content stream.
    import itertools as it
    
    from tabutils.io import read_csv, IterStringIO
    from csv2ofx import utils
    from csv2ofx.qif import QIF
    from csv2ofx.mappings.default import mapping
    
    qif = QIF(mapping)
    records = read_csv('path/to/file.csv', has_header=True)
    groups = qif.gen_groups(records)
    trxns = qif.gen_trxns(groups)
    cleaned_trxns = qif.clean_trxns(trxns)
    data = utils.gen_data(cleaned_trxns)
    content = it.chain([qif.gen_body(data), qif.footer()])
    
    for line in IterStringIO(content):
        print(line)
  4. Configure CSV parsing and value attributes

    main

    When defining mappings or using the library, you can control how the CSV data is interpreted using these attributes:

    attributedescriptiondefault value
    has_headerwhether the CSV has a header rowTrue
    custom_headerheader row to use if not provided in CSVNone
    is_splitif the CSV contains split (double entry) transactionsFalse
    currencythe currency ISO codeUSD
    delimiterthe CSV field delimiter,
    date_fmtcustom QIF date output format%m/%d/%y
    dayfirstinterpret first value in ambiguous dates as the dayFalse
    parse_fmttransaction date parsing formatNone
    first_rowthe first row to process (zero based)0
    last_rowthe last row to process (zero based)inf
    first_colthe first column to process (zero based)0
    filterfunction to keep transactions (returns bool)None
  5. Use the csv2ofx CLI

    main

    The csv2ofx command line tool converts CSV files to OFX or QIF.

    Basic Usage: csv2ofx <source_csv> <dest_file>

    Common CLI Flags:

    • -q, --qif: Enables QIF output instead of the default OFX.
    • -m, --mapping <NAME>: Specifies the account mapping to use (e.g., yoodlee).
    • -s, --start <DATE>: Sets the start date.
    • -e, --end <DATE>: Sets the end date.
    • -x, --custom <FILE_PATH>: Path to a custom mapping file.
    • -L, --list-mappings: Lists available mappings.
    • -o, --overwrite: Overwrites the destination file if it exists.
    • -v, --verbose: Enables verbose output.
    # Normal usage
    csv2ofx file.csv file.ofx
    
    # QIF output
    csv2ofx -q file.csv
    
    # Specify date range with QIF output
    csv2ofx -s '-1 year' -e yesterday -q file.csv
    
    # Use specific mapping
    csv2ofx -m yoodlee file.csv
  6. Generate OFX account start and end blocks

    main

    These methods manage the lifecycle of an account statement in the OFX file.

    account_start() Generates the <STMTRS> and <BANKACCTFROM> blocks.

    • Required Kwargs: currency, bank_id, account_id, account_type (one of CHECKING, SAVINGS, MONEYMRKT, CREDITLINE).
    • Note: If ms_money is enabled, dates are automatically adjusted to Noon GMT (e.g., YYYYMMDD120000) to ensure compatibility.

    account_end() Generates the <LEDGERBAL> and closing tags.

    • Required Kwargs: date (datetime), balance (number).
    • Automatic Balance Detection: If balance is not provided in kwargs, the class attempts to infer the ending balance from the transaction history by analyzing date and balance order (ascending vs descending).
    import datetime
    from csv2ofx.ofx import OFX
    
    ofx = OFX()
    
    # Account Start
    start_block = ofx.account_start(
        currency='USD', 
        bank_id='1', 
        account_id='1', 
        account_type='CHECKING', 
        start=datetime.datetime(2012, 1, 1), 
        end=datetime.datetime(2012, 2, 1)
    )
    
    # Account End
    end_block = ofx.account_end(
        date=datetime.datetime(2012, 1, 15), 
        balance=150.00
    )
  7. Retrieve attributes with get()

    main

    The get(name, trxn=None, default=None) method is used to retrieve transaction attributes. It is highly flexible: it can return a static attribute of the Content instance, a value from a mapping function applied to the transaction, or a value from the mapping dictionary itself.

    • If name refers to a normal attribute on Content, it returns that attribute.
    • If name refers to a mapping function, it calls that function with trxn as an argument.
    • If the attribute is not found, it returns the default value.
    import datetime
    from csv2ofx.mappings.mint import mapping
    
    trxn = {'Transaction Type': 'DEBIT', 'Amount': 1000.00}
    content = Content(mapping)
    
    # Get a normal attribute
    start_date = content.get('start')
    
    # Get a mapping function value (requires the transaction dict)
    amount = content.get('amount', trxn)
    
    # Get a mapping attribute
    has_header = content.get('has_header')
  8. Generate QIF groups with `gen_groups()`

    main

    A generator method that yields groups of records, useful for organizing transactions by account or ID.

    Arguments:

    • records (iterable): The records to group.
    • chunksize (int, optional): Size of chunks to process.
    from csv2ofx.qif import QIF
    
    qif = QIF()
    # records is an iterable of transaction dictionaries
    for group in qif.gen_groups(records):
        # process group
        pass
  9. Load a custom mapping module

    main

    If the built-in mappings are insufficient, you can provide a path to a custom Python module using the -x or --custom flag. This module must define a mapping attribute. You can also use the load_custom_module function programmatically to load a mapping file.

    from csv2ofx.main import load_custom_module
    
    # Load a custom mapping from a file path
    mod = load_custom_module("path/to/your_custom_mapping.py")
    mapping = mod.mapping
  10. Initialize the OFX class

    main

    The OFX class is used to generate OFX content from transaction data. It inherits from Content and requires a mapping dictionary (typically from csv2ofx.mappings) to interpret bank-specific CSV formats.

    Constructor Arguments:

    • mapping (dict): A bank mapper dictionary.
    • def_type (str, optional): Default account type.
    • start (date, optional): Date from which to begin including transactions.
    • end (date, optional): Date from which to exclude transactions.

    Account Types: Supported types include CHECKING, SAVINGS, MONEYMRKT, and CREDITLINE.

    from csv2ofx.mappings.mint import mapping
    from csv2ofx.ofx import OFX
    
    # Initialize with a mapping
    ofx = OFX(mapping)
  11. Handle OFX transfers and splits

    main

    For transactions involving transfers between accounts, use the following methods:

    • transfer(**kwargs): Generates the <INTRARS> block. Requires account_type, currency, id, amount, bank_id, and account_id.
    • split_content(**kwargs): Generates the <BANKACCTTO> block. If split_account is provided in kwargs, it uses the split account details; otherwise, it uses the primary account details.
    • transfer_end(**kwargs): Generates the closing <XFERINFO> and <INTRARS> tags. Requires date (datetime).

    Note: OFX does not support more than 2 splits per transaction. gen_body will raise a TypeError if a group contains more than 2 splits.

    import datetime
    from csv2ofx.ofx import OFX
    
    ofx = OFX()
    kwargs = {
        'currency': 'USD', 
        'date': datetime.datetime(2012, 1, 15), 
        'bank_id': 1, 
        'account_id': 1, 
        'account_type': 'CHECKING', 
        'amount': 100, 
        'id': 'jbaevf',
        'split_account': 'Checking',
        'split_account_id': 2,
        'split_account_type': 'CHECKING'
    }
    
    transfer_start = ofx.transfer(**kwargs)
    split_part = ofx.split_content(**kwargs)
    transfer_end = ofx.transfer_end(date=kwargs['date'])