repository·main·Indexed 25 days ago
https://github.com/dragonflyoss/dragonflyDragonfly is a CNCF Graduated P2P-based data distribution system designed to accelerate the delivery of large-scale cloud-native assets, including container images, OCI artifacts, AI/ML models, and files. It provides a Manager raw API via Swagger, supports deployment via Docker Compose and Kubernetes, and includes components such as the Manager, Scheduler, and dfdaemon. The system features observability through OpenTelemetry tracing and pprof-based metrics.
Dragonfly is a CNCF Graduated project that provides efficient, stable, and secure data distribution and acceleration using P2P technology. It is designed for cloud-native architectures to improve the large-scale delivery of:
It includes an optional content-addressable filesystem to accelerate OCI container launches.
Developers can interact with Dragonfly by using the Manager's raw API to integrate Dragonfly management into their own services and DevOps workflows. The API documentation is provided in Swagger format (JSON and YAML) and is automatically generated.
To view the API documentation, you can:
manager directory./swagger/doc.json endpoint provided by the running Manager service.You can deploy Dragonfly components using docker-compose via the provided run.sh script. Note that this deployment method is currently tested for single-host use only and does not support High Availability (HA).
If you are not using network=host mode, you must provide the host IP address. To find the host IP address (the docker network gateway IP) on a bridge network, use:
docker network inspect bridge -f '{{range .IPAM.Config}}{{.Gateway}}{{end}}'Set the IP environment variable and execute the script to generate configurations and deploy all components.
docker compose down command.docker compose downSeveral cloud providers offer managed or integrated Dragonfly deployment options:
For users on Alibaba Cloud or those looking for the most efficient and stable deployment, it is recommended to use Dragonfly 2.0. You can follow the official setup guide for Kubernetes to deploy it.
https://d7y.io/docs/getting-started/quick-start/kubernetes/You can deploy a local Dragonfly environment using the provided Docker Compose configuration. This setup includes a client, a scheduler, and a seed-client service.
Service Details:
scheduler and seed-client. It exposes ports 4000, 4001, and 4002.8002.scheduler. It exposes ports 4010, 4011, and 4012.Volume Mapping Requirements: To run this configuration, you must have local directories and configuration files prepared as follows:
| Service | Local Log Path | Local Config Path | Local DynConfig Path |
|---|---|---|---|
client | ./log/client | ./config/client.yaml | ./config/client-dynconfig.yaml |
scheduler | ./log/scheduler | ./config/scheduler.yaml | ./config/scheduler-dynconfig.yaml |
seed-client | ./log/seed-client | ./config/seed-client.yaml | ./config/client-dynconfig.yaml |
services:
client:
image: dragonflyoss/client:latest
depends_on:
- scheduler
- seed-client
container_name: client
restart: always
healthcheck:
test: ["CMD-SHELL", "/bin/grpc_health_probe -addr=unix:///var/run/dragonfly/dfdaemon.sock || exit 1"]
interval: 1s
timeout: 2s
retries: 30
volumes:
- ./log/client:/var/log/dragonfly
- ./config/client.yaml:/etc/dragonfly/dfdaemon.yaml:ro
- ./config/client-dynconfig.yaml:/etc/dragonfly/dynconfig.yaml:ro
ports:
- 4000:4000
- 4001:4001
- 4002:4002
scheduler:
image: dragonflyoss/scheduler:latest
container_name: scheduler
restart: always
healthcheck:
test: ["CMD-SHELL", "/bin/grpc_health_probe -addr=:8002 || exit 1"]
interval: 1s
timeout: 2s
retries: 30
volumes:
- ./log/scheduler:/var/log/dragonfly
- ./config/scheduler.yaml:/etc/dragonfly/scheduler.yaml:ro
- ./config/scheduler-dynconfig.yaml:/etc/dragonfly/dynconfig.yaml:ro
ports:
- 8002:8002
seed-client:
image: dragonflyoss/client:latest
depends_on:
- scheduler
container_name: seed-client
restart: always
healthcheck:
test: ["CMD-SHELL", "/bin/grpc_health_probe -addr=unix:///var/run/dragonfly/dfdaemon.sock || exit 1"]
interval: 1s
timeout: 2s
retries: 30
volumes:
- ./log/seed-client:/var/log/dragonfly
- ./config/seed-client.yaml:/etc/dragonfly/dfdaemon.yaml:ro
- ./config/client-dynconfig.yaml:/etc/dragonfly/dynconfig.yaml:ro
ports:
- 4010:4010
- 4011:4011
- 4012:4012SECURITY-INSIGHTS.yml.The Options struct defines the configuration settings for the dependency command, including console output, profiling, and tracing. These settings can be provided via YAML configuration using the following keys:
console (bool): Enables or disables console output.pprofPort (int): Specifies the port for the pprof profiler.tracing (object): Configuration for OpenTelemetry tracing.The TracingConfig defines how OpenTelemetry tracing logs are transmitted to a collector. Use these keys to configure the tracing behavior:
protocol (string): The communication protocol. Supported values are "http", "https", and "grpc".endpoint (string): The address of the tracing server (e.g., "localhost:4317").path (string): The server path, required if using "http" or "https" (e.g., "/v1/traces").service-name (string): The identifier for the service in the tracing backend.headers (map[string]string): A collection of additional HTTP headers to include with tracing requests.The WatchConfig function allows your application to react to changes in the configuration file. It uses a polling mechanism (re-reading the file) rather than fsnotify, making it compatible with Kubernetes ConfigMaps which are often updated via symlink swaps.
Arguments:
interval: How often to check the file for changes.newConfig: A function that returns a new configuration instance.watcher: A callback function executed when a change is detected, receiving the new configuration.