Data Transfer Project Documentation
repository·master·Indexed 25 days ago
https://github.com/dtinit/data-transfer-projectAn open-source framework for direct, service-to-service data portability between online providers. Includes documentation for deploying via GCP, Azure, and Docker, as well as implementation details for extensions such as Apple, Deezer, Facebook, Flickr, and Generic Importers.
What's inside Data Transfer Project
- This extension provides integration for Synology within the Data Transfer Project. It allows for the transfer of specific media types between Synology and supported source platforms.
Overview of the Imgur extension
masterThe Imgur extension provides implementation for transferring data to and from the Imgur service. It supports the import and export of Albums and Photos.Overview of the Deezer extension
masterThe Deezer extension provides an implementation for the Deezer API within the Data Transfer Project. It allows for the transfer of playlist data.Overview of the Spotify extension
masterThis extension provides implementation for the Spotify API within the Data Transfer Project. It enables the transfer of playlist data.Overview of the Flickr extension
masterThe Flickr extension provides implementation for the Flickr service within the Data Transfer Project. It is based on theFlickr4Javalibrary and enables data portability for Flickr users.Overview of the Koofr extension
masterThis extension provides integration for Koofr within the Data Transfer Project. It currently supports importing specific media types from Koofr.Overview of Microsoft extension for Data Transfer Project
masterThe Microsoft extension provides data portability capabilities for Microsoft services. It currently supports the import and export of Calendar, Contacts, and Photos data.Overview of DTP shared libraries
masterThe
libraries/directory contains shared logic and utility libraries used by the core Data Transfer Project (DTP) codebase. It is organized into three main functional areas:- config: Utilities for configuring DTP using command-line flags and configuration files.
- security: Cryptographic helper libraries for secure operations.
- transfer: Helper utilities designed to facilitate the actual transfer of data.
Overview of the Mastodon extension
masterThe Mastodon extension provides an implementation for the Mastodon API within the Data Transfer Project. It is currently in a Proof of Concept state.
Supported Data:
- Social Activity streams (limited to notes < 500 characters).
Known Limitations:
- Data import/export currently handles raw text only; images are not supported.
- Imported data is automatically marked as private; the Mastodon UI does not currently appear to support marking this imported data as public.
Overview of the Data Transfer Project
masterThe Data Transfer Project (DTP) provides a common framework and open-source ecosystem designed to enable seamless, direct data transfers between online service providers. Instead of requiring users to download and then upload their data, DTP facilitates service-to-service portability, allowing data to move directly between providers.Overview of Facebook Data Transfer Extension
masterThe Facebook extension for the Data Transfer Project enables the export of specific user data types from Facebook. Currently, the extension supports the export of Photos and Videos, which can be run and tested locally. Support for Social Posts and Notes is planned for future release.Understand API Key requirements for DTP
masterTo run a Data Transfer Project (DTP) instance, you must acquire and manage unique API keys for every service you intend to facilitate transfers with. Each hosted instance requires its own set of keys to ensure isolation (preventing quota/abuse issues from affecting other instances), compliance with individual service Terms of Service (TOS), and improved security.
API keys typically consist of an OAuth App Key and Secret, but the exact format depends on the authorization technology used by the specific service provider. You only need to provide keys for the specific providers you wish to support in your instance.