DOCSEARCHbyAlgolia
Submit Docs

Antigranular Documentation

website·Indexed 34 days ago

https://docs.antigranular.com/

A workspace for privacy-first data science featuring a community platform, notebooks, and competitions. It provides a Private Python environment with implementations of differential privacy libraries including op pandas, OpenDP, Opacus, TensorFlow Privacy, SmartNoise SQL, SmartNoise Synth, SPlink, and Diffprivlib.

Tokens
26.1K
Snippets
159
Records
216
Agent score
56%

What's inside Antigranular

  1. Overview of op_tensorflow for Differential Privacy

    op_tensorflow is a custom library that integrates tensorflow-privacy with op_pandas private data objects. It allows developers to train machine learning models with differential privacy guarantees using DP-SGD (Differentially Private Stochastic Gradient Descent).

    Constraint: Currently, only Sequential models can be created directly within the Antigranular environment.

  2. Overview of op_diffprivlib

    The op_diffprivlib library provides differentially private versions of scikit-learn models and tools for data analysis. It includes mechanisms for adding noise to data, privacy-preserving machine learning algorithms, and statistical analysis tools. Unlike the standard diffprivlib where bounds may be optional, op_diffprivlib requires metadata and bounds to be explicitly provided for its methods to function.

  3. Overview of Antigranular Datasets

    Antigranular provides a curated collection of open-source, real-world datasets across domains such as finance, healthcare, social sciences, and natural language processing. These datasets are designed for experimentation, algorithm testing, and practicing data processing, cleaning, and feature engineering outside of traditional competition scenarios.

  4. Introduction to AG's differentially private Pandas (op_pandas)

    AG provides op_pandas, a differentially private version of the Pandas library. It allows users to handle private data frames and series and perform statistical analyses with differential privacy guarantees. The API is designed to be familiar to users already experienced with the standard open-source Pandas library.

  5. Overview of op_opendp

    op_opendp is a library for privacy-preserving data analysis that supports all functionality of OpenDP 0.8.0, maintaining the same module names and function signatures. It is used to construct and execute differentially private pipelines.
    import op_opendp

  6. Overview of Antigranular privacy-first data science platform

    Antigranular is a community platform designed for 'Eyes-Off Data Science'. It provides a workspace for data scientists to work with massive datasets using privacy-preserving technologies, specifically confidential computing (Trusted Execution Environments) and differential privacy. The platform allows users to execute code in a secure environment where sensitive data must pass through a differential privacy mechanism before results are returned to the local session.

  7. Calculate competition scores using the Accuracy-Epsilon formula

    While scoring mechanisms vary by competition (detailed in the competition's overview section), a common formula used is:

    Score = Λ - ε/κ

    • Accuracy (Λ): The accuracy of predictions against test data (maximum 1.0 for 100%).
    • Epsilon (ε): The privacy parameter quantifying the level of protection. Smaller ε provides stronger privacy but noisier results; larger ε provides weaker privacy but higher accuracy.
    • Constant (κ): A value that varies based on the dataset size, specified in the competition overview.

  8. Understand Antigranular scoring metric implementation

    Antigranular uses a combination of accuracy proxies and privacy budget loss to set goals or host competitions. To prevent information leakage via leaderboards, accuracy metrics are calculated using differentially private measurements.

  9. Understand Trusted Execution Environments (TEEs) for Confidential Computing

    Trusted Execution Environments (TEEs), also known as Secure Enclaves or Confidential Computing, provide a secure area of a main processor that guarantees the confidentiality and integrity of loaded code and data. TEEs act as a secure 'lockbox' within a processor, preventing the contents from being seen or altered by external entities, including other applications, users, or the cloud provider. A key feature of TEEs is attestation, which allows parties to verify that the software running inside an enclave aligns with pre-agreed specifications and standards, enabling secure multi-party collaboration on sensitive data and proprietary algorithms.

  10. Use op_diffprivlib for differentially private data analysis

    The op_diffprivlib library provides differential privacy (DP) equivalents of scikit-learn models and tools for data analysis. It maintains the same interfaces, function names, and signatures as the original diffprivlib library, but includes additional privacy implementations.

    Supported modules include:

    • op_diffprivlib.tools: General DP tools and mechanisms.
    • op_diffprivlib.models: DP-equivalent machine learning models.

    Data Type Handling: op_diffprivlib enforces specific return types based on the input:

    • If a PrivateDataFrame is provided as input, the function returns a PrivateDataFrame.
    • If a pandas.DataFrame is provided as input, the function returns a pandas.DataFrame.

  11. Understand the principles of Differential Privacy

    Differential Privacy (DP) is a framework used to safeguard individual data during statistical analysis by inserting controlled random noise into data queries. This ensures that the inclusion or exclusion of any single data point does not substantially alter the results, preventing the re-identification of individuals even when dealing with sensitive datasets. It is commonly applied in healthcare for patient outcome research, in banking for fraud detection and credit risk assessment, and in social sciences for population trend analysis.

  12. Use OpenDP functionality in AntiGranular

    AntiGranular provides full support for the OpenDP library for privacy-preserving data analysis via the op_opendp package. The op_opendp package maintains the same module names and function signatures as the official OpenDP library, allowing developers to use OpenDP's functions and methods for sensitive data analysis within the AG environment.