Data Transfer Project Documentation

repository·master·Indexed 25 days ago

https://github.com/dtinit/data-transfer-project

An open-source framework for direct, service-to-service data portability between online providers. Includes documentation for deploying via GCP, Azure, and Docker, as well as implementation details for extensions such as Apple, Deezer, Facebook, Flickr, and Generic Importers.

Tokens
19.7K
Snippets
27
Records
148
Agent score
86%

What's inside Data Transfer Project

  1. Overview of DTP shared libraries

    master

    The libraries/ directory contains shared logic and utility libraries used by the core Data Transfer Project (DTP) codebase. It is organized into three main functional areas:

    • config: Utilities for configuring DTP using command-line flags and configuration files.
    • security: Cryptographic helper libraries for secure operations.
    • transfer: Helper utilities designed to facilitate the actual transfer of data.
  2. Overview of the Mastodon extension

    master

    The Mastodon extension provides an implementation for the Mastodon API within the Data Transfer Project. It is currently in a Proof of Concept state.

    Supported Data:

    • Social Activity streams (limited to notes < 500 characters).

    Known Limitations:

    • Data import/export currently handles raw text only; images are not supported.
    • Imported data is automatically marked as private; the Mastodon UI does not currently appear to support marking this imported data as public.
  3. Overview of the Data Transfer Project

    master
    The Data Transfer Project (DTP) provides a common framework and open-source ecosystem designed to enable seamless, direct data transfers between online service providers. Instead of requiring users to download and then upload their data, DTP facilitates service-to-service portability, allowing data to move directly between providers.
  4. Understand API Key requirements for DTP

    master

    To run a Data Transfer Project (DTP) instance, you must acquire and manage unique API keys for every service you intend to facilitate transfers with. Each hosted instance requires its own set of keys to ensure isolation (preventing quota/abuse issues from affecting other instances), compliance with individual service Terms of Service (TOS), and improved security.

    API keys typically consist of an OAuth App Key and Secret, but the exact format depends on the authorization technology used by the specific service provider. You only need to provide keys for the specific providers you wish to support in your instance.