repository·main·Indexed 18 days ago
https://github.com/mystenlabs/walrusA decentralized blob store utilizing the Sui blockchain for coordination and governance. This documentation covers the Walrus Upload Relay, the orchestrator CLI for geo-distributed deployments, performance testing with k6-tests, and setting up local testbeds using Docker Compose.
walrus/Move.lock file.Walrus is a decentralized storage and data availability protocol designed for blockchain applications and autonomous agents. It uses erasure coding to split unstructured data blobs into smaller "slivers" distributed across a network of storage nodes.
Key characteristics:
The Python examples directory contains scripts demonstrating different ways to interact with Walrus:
hello_walrus_jsonapi.py: Demonstrates storing and reading blobs using the Walrus client's JSON API.hello_walrus_webapi.py: Demonstrates storing and reading blobs using the Walrus client's HTTP API.track_walrus_events.py: A utility script used to track all Walrus-related events occurring on the Sui network.Walrus infrastructure is composed of several distinct service provider roles. Depending on your hardware and operational goals, you can choose to operate one or more of the following:
Refer to the specific guides for each role to understand their individual requirements and setup procedures.
Walrus v1.47.1 introduces several improvements to storage node and aggregator behavior:
HTTP 503 BLOB_UNAVAILABLE instead of a 500 error when a blob is only temporarily unretrievable. This allows clients to implement retry logic more effectively.Content-Type are now correctly included in range requests.Walrus v1.38.2 (Testnet build) introduces several performance improvements and new API capabilities:
aggregator, publisher, and CLI./v1alpha/blobs/concat endpoint.get_recovery_symbol endpoint has been removed.Walrus v1.26.3 introduced the following improvements:
Walrus v1.38.3 introduces several performance improvements and new API capabilities:
aggregator, publisher, and CLI./v1alpha/blobs/concat endpoint is now available.get_recovery_symbol endpoint has been removed.An upload relay is a server-side service that simplifies the blob upload process by handling the complexity of writing to multiple storage nodes on behalf of your application. Instead of your client making many parallel requests to different storage nodes (potentially 10+), you make a single request to the relay.
/v1/blob-upload-relay.Quilt is a batch storage feature in Walrus designed to optimize the storage cost and efficiency of large numbers of small blobs. It encodes multiple blobs (up to 666 for QuiltV1) into a single unit called a quilt, significantly reducing Walrus storage overhead and lowering costs for Walrus/Sui storage and Sui computation gas fees.
Use Quilt when:
Use a regular blob when:
delete, extend, or share an item on its own schedule, as these operations apply to the entire quilt).BlobId (items in a quilt are addressed by a QuiltPatchId which depends on the composition of the entire quilt).Set the site name using the site_name field. If the --site-name CLI flag is used during deployment, it takes priority and overwrites the value in ws-resources.json.
The object_id field stores the Sui object ID of your deployed site.
site-builder deploy command uses this field to identify an existing site. If a valid object_id is present, deploy updates that site.object_id is missing and no --object-id flag is provided, deploy publishes a new site and automatically populates the object_id in your ws-resources.json.Caution: Do not delete ws-resources.json after your first deployment. If it is missing during a subsequent deployment, the site-builder will create a brand new site object instead of updating your existing one.
The HTTP API returns different JSON structures depending on whether the blob is being created for the first time or already exists.
If the blob is new, the response contains a newlyCreated field. This includes the blobObject (the Sui object representing the blob) and resourceOperation details.
If the publisher finds a certified blob with the same ID and sufficient validity, it returns an alreadyCertified structure. This contains the blobId, the endEpoch, and an event object containing the txDigest and eventSeq used to track the original transaction on Sui.
Tip: If you are reading a blob immediately after upload via a CDN-fronted aggregator, you may encounter a brief 404 due to propagation delay. Implement a retry with backoff.
# Example response for a newly created blob
{
"newlyCreated": {
"blobObject": {
"id": "0xe91eee8c5b6f35b9a250cfc29e30f0d9e5463a21fd8d1ddb0fc22d44db4eac50",
"registeredEpoch": 34,
"blobId": "M4hsZGQ1oCktdzegB6HnI6Mi28S2nqOPHxK-W7_4BUk",
"size": 17,
"encodingType": "RS2",
"certifiedEpoch": 34,
"storage": {
"id": "0x4748cd83217b5ce7aa77e7f1ad6fc5f7f694e26a157381b9391ac65c47815faf",
"startEpoch": 34,
"endEpoch": 35,
"storageSize": 66034000
},
"deletable": false
},
"resourceOperation": {
"registerFromScratch": {
"encodedLength": 66034000,
"epochsAhead": 1
}
},
"cost": 132300
}
}
# Example response for an already certified blob
{
"alreadyCertified": {
"blobId": "M4hsZGQ1oCktdzegB6HnI6Mi28S2nqOPHxK-W7_4BUk",
"event": {
"txDigest": "4XQHFa9S324wTzYHF3vsBSwpUZuLpmwTHYMFv9nsttSs",
"eventSeq": "0"
},
"endEpoch": 35
}
}