Hasura Storage

Hasura Storage acts as a middleware between your users, Hasura, and an S3-compatible storage service. It leverages Hasura's GraphQL API to enforce permissions and manage file metadata.

Upload Workflow

1. Permission Check: The service asks Hasura if the user has permission to upload.
2. S3 Upload: If permitted, the file is uploaded to the S3-compatible service.
3. Metadata Storage: Upon successful S3 upload, the file's metadata is stored in Hasura via GraphQL.
4. Response: The user receives the file metadata.

Retrieval Workflow

1. Permission Check: The service asks Hasura if the user has permission to access the specific file.
2. S3 Fetch: If permitted, the service requests the file from S3.
3. Response: The file is streamed back to the user.

To verify your changes, follow these steps:

1. Start prerequisites: Run make dev-env-up-short to start the necessary containers.
2. Apply changes: Modify your code.
3. Build and run:
   • Use the Makefile: make dev-env-up-hasura (this builds and deploys in a Docker container).
   • Or, if your local environment is configured (via Nix or manual C dependencies), run go run main.go serve or nix run . -- serve.
4. Run tests: Execute make integration-tests to run the test suite.
5. Cleanup: Stop the environment with make dev-env-down.

To self-host Hasura Storage, you must satisfy the following requirements:

1. Hasura: A running Hasura instance connected to a supported database (e.g., PostgreSQL).
2. S3-Compatible Storage: An S3-compatible service such as AWS S3 or MinIO.

For a complete setup guide, refer to the docker-compose example in the /build/dev/docker/ directory of the repository. When using Docker, ensure you use a valid production image (e.g., nhost/hasura-storage:0.1.5) instead of the dev tag.

The project uses Nix to provide a consistent development environment. If you have Nix installed, you can initialize the environment by running nix develop. This ensures your local environment matches the one used in CI.

Note that hasura-storage relies on the C library libvips for performance. If you are not using Nix, you must manually install libvips-dev and the necessary compilation tools such as gcc and pkg-config.

Hasura Storage can integrate with ClamAV to scan uploaded files for viruses. This requires an external clamd service running and accessible via a TCP socket.

Workflow with Antivirus:

• When a file is uploaded, the service first checks the file with clamd.
• If a virus is found: The S3 upload is aborted, and details are inserted into the virus table in Hasura.
• If the file is clean: The upload to S3 proceeds and the metadata is updated in Hasura.

To enable this, use the --clamav-server flag with the TCP address of your clamd service.

--clamav-server <tcp-address>

You can automate the setup of your database and Hasura metadata by using the following flags during the serve command:

1. Postgres Migrations: Use --postgres-migrations and provide the connection string via --postgres-migrations-source.
2. Hasura Metadata: Use --hasura-metadata and provide the Hasura endpoint via --hasura-endpoint and the admin secret via --hasura-graphql-admin-secret.

# Example applying both Postgres and Hasura migrations
./hasura-storage serve \
  --postgres-migrations \
  --postgres-migrations-source postgres://user:pass@localhost:5432/mydb \
  --hasura-metadata \
  --hasura-endpoint http://localhost:8080 \
  --hasura-graphql-admin-secret mysecret

The service uses S3 for content storage. You must provide the --s3-bucket and --s3-endpoint. If you provide both --s3-access-key and --s3-secret-key, the service will use static credentials. Otherwise, it will attempt to use the default AWS credential provider chain. You can also specify a --s3-root-folder to nest all created buckets under a specific path in your S3 bucket.

# Example using static credentials and a custom endpoint
./hasura-storage serve \
  --s3-endpoint https://minio:9000 \
  --s3-access-key minioadmin \
  --s3-secret-key minioadmin \
  --s3-bucket my-storage-bucket \
  --s3-region us-east-1

To control Cross-Origin Resource Sharing (CORS), use the following flags:

• --cors-allow-origins: A list of allowed origins (defaults to *).
• --cors-allow-credentials: A boolean to allow credentials (defaults to false).

Note that the service automatically exposes several headers including Content-Length, Content-Type, Cache-Control, ETag, Last-Modified, and X-Error.

The following persistent flags are available to the hasura-storage command and all its subcommands:

--config, -c string   use this configuration file
--debug               enable debug messages