DOCSEARCHbyAlgolia
Submit Docs

Microdict

repository·main·Indexed 42 days ago

https://github.com/touqir14/microdict

A high-performance, typed Python hash table library implemented in C. Microdict uses native types instead of PyObjects to provide faster lookups and significantly lower memory consumption (up to 7x less) than standard Python dictionaries. It supports various dtypes including i32, i64, and str, and provides a Pythonic interface for data manipulation and iteration.

Tokens
1.2K
Snippets
7
Records
7
Agent score
3%

What's inside Microdict

  2. Build Microdict from source

    main

    Since Microdict is built using C extensions, you need the Python C API header files. To build and install from a cloned repository, use setup.py install.

    Requirements:

    • Linux/macOS: GCC 7+
    • Windows: Visual C++ 14+
    • Architecture: 64-bit system is recommended for best performance.
    python setup.py install

  4. Use Microdict methods for data manipulation

    main

    Microdict provides a Pythonic interface for common dictionary operations:

    • Set/Update: Use d[key] = value or d.update(other_dict_or_mdict).
    • Access: Use d[key] to get a value or d.pop(key) to remove and return a value. pop() raises a KeyError if the key is missing.
    • Check existence: Use key in d.
    • Clear: d.clear() empties the dictionary. d.clear([key1, key2]) removes specific keys provided in a list.
    • Copy: d.copy() returns a deep copy of the Microdict.
    • Convert: d.to_Pydict() returns a standard Python dictionary containing all items.
    from microdict import mdict

d = mdict.create("i32:i32")
d[1] = 10
d[2] = 20

# Update with a standard python dict
d.update({3: 30, 4: 40})

# Remove specific keys
d.clear([1, 2])

# Convert to standard dict
pydict = d.to_Pydict()

  5. Iterate over Microdict items, keys, and values

    main

    Microdict supports two ways to access elements: for memory efficiency (iteration) and for convenience (list creation).

    For memory-efficient iteration (Recommended): Use these methods in for loops to avoid creating large intermediate list objects.

    • d.items(): Iterates over (key, value) pairs.
    • d.keys(): Iterates over keys.
    • d.values(): Iterates over values.

    For creating Python lists: Use these if you specifically need a list object. Note that these are generally slower and more memory-intensive than direct iteration.

    • d.get_items(): Returns a list of (key, value) tuples.
    • d.get_keys(): Returns a list of keys.
    • d.get_values(): Returns a list of values.
    from microdict import mdict
d = mdict.create("i32:i32")
# ... populate d ...

# Efficient iteration
for k, v in d.items():
    print(k, v)

# Creating lists
keys_list = d.get_keys()
values_list = d.get_values()
items_list = d.get_items()

  6. Create a Microdict hash table

    main

    Use microdict.mdict.create(dtype, key_len=None, val_len=None) to initialize a new typed hash table.

    Available dtype strings:

    • "i32:i32": 32-bit signed integer keys and 32-bit signed integer values.
    • "i32:i64": 32-bit signed integer keys and 64-bit signed integer values.
    • "i64:i32": 64-bit signed integer keys and 32-bit signed integer values.
    • "i64:i64": 64-bit signed integer keys and 64-bit signed integer values.
    • "str:str": String keys and string values.

    String Type Constraints (str:str only):

    • key_len: Maximum number of bytes for UTF-8 string keys (max 65535). If a key exceeds this, it will not be accepted.
    • val_len: Maximum number of bytes for UTF-8 string values (max 65535). If a value exceeds this, it will not be accepted.
    from microdict import mdict

# Create a dictionary with 32-bit signed integer keys and values
dict_i32 = mdict.create("i32:i32")

# Create a string dictionary with specific byte limits
dict_str = mdict.create("str:str", key_len=10, val_len=20)

  7. Reference: Microdict API Methods

    main

    Summary of available methods on a Microdict instance.

    Methods:
- mdict.create(dtype, key_len=None, val_len=None)
- mdict.listDictionaryTypes()
- clear(key_list=None)
- copy()
- get_items()
- get_keys()
- get_values()
- items()
- pop(key)
- to_Pydict()
- update(dictionary)
- values()