Hocuspocus
repository·main·Indexed 11 days ago
https://github.com/ueberdosis/hocuspocusA plug-and-play collaboration backend based on Y.js that provides a WebSocket server for real-time collaborative applications. It includes a CLI for quick deployment and a variety of extensions for persistence (SQLite, S3, and a generic Database extension), horizontal scaling via Redis, logging, and IP-based rate-limiting via a throttle extension.
What's inside Hocuspocus
- Hocuspocus is a plug-and-play collaboration backend built on top of Y.js. It provides a WebSocket server designed to handle real-time collaborative data synchronization.
Use @hocuspocus/transformer to convert Y.js documents
main@hocuspocus/transformer is a utility for converting Y.js documents to and from Tiptap / ProseMirror JSON and HTML formats. This is particularly useful for server-side operations where you need to process or render collaborative document state outside of a live editor environment, such as generating document previews, performing exports, or preparing webhook payloads.Configure Webhook events and debounce settings
mainThe
Webhookextension allows you to subscribe to specific lifecycle events using theEventsenum.Available events include:
Events.onChange: Triggered on document changes. These events are debounced by default to prevent excessive requests. You can tune this usingdebounceanddebounceMaxWaitoptions.Events.onConnect: Triggered when a client connects.Events.onDisconnect: Triggered when a client disconnects.Events.onCreate: Triggered when a document is created.
To verify the authenticity of incoming requests, use the
secretprovided during setup to check theX-Hocuspocus-Signature-256header, which contains an HMAC-SHA256 signature.Understand the role of @hocuspocus/common
main@hocuspocus/common is a utility package containing shared types, common enums, and helpers for the message/sync protocol used across the Hocuspocus ecosystem.
Note for developers: This package is primarily an internal building block. Most end-users should instead depend on
@hocuspocus/serveror@hocuspocus/providerto interact with the Hocuspocus ecosystem. You should only depend on@hocuspocus/commondirectly if you are building custom extensions or low-level protocol implementations that require specific shared types.Understand S3 document storage format
mainDocuments are stored as binary files in your S3 bucket. The naming convention used is:
{prefix}{documentName}.binBy default, the prefix is
hocuspocus-documents/.Scale Hocuspocus horizontally with S3 and Redis
mainTo scale Hocuspocus horizontally, combine the
S3extension with theRedisextension. In this architecture, Redis handles real-time synchronization between server instances, while S3 provides the persistent storage layer for the documents.import { Server } from '@hocuspocus/server' import { Logger } from '@hocuspocus/extension-logger' import { Redis } from '@hocuspocus/extension-redis' import { S3 } from '@hocuspocus/extension-s3' const server1 = new Server({ name: "server-1", port: 8001, extensions: [ new Logger(), new Redis({ host: "127.0.0.1", port: 6379, }), new S3({ bucket: 'hocuspocus-documents', endpoint: 'http://localhost:9000', forcePathStyle: true, credentials: { accessKeyId: 'minioadmin', secretAccessKey: 'minioadmin' } }), ], }) // Server 2 must have the same Redis and S3 configuration to sync correctly const server2 = new Server({ name: "server-2", port: 8002, extensions: [/* same extensions as server1 */], }) server1.listen() server2.listen()Quickstart: Start a Hocuspocus WebSocket server
mainTo start a Hocuspocus WebSocket server, import the
Serverclass from@hocuspocus/server. You can configure the server by passing an options object to the constructor, which allows you to define theport, lifecycle hooks likeonConnect, andextensions(such as@hocuspocus/extension-sqlitefor persistence). Call.listen()to start the server. By default, the server listens onhttp://127.0.0.1(orws://127.0.0.1for WebSocket connections).import { Server } from '@hocuspocus/server' import { SQLite } from '@hocuspocus/extension-sqlite' const server = new Server({ port: 1234, async onConnect() { console.log('🔮') }, extensions: [ new SQLite({ database: 'db.sqlite', }), ], }); server.listen();Install @hocuspocus/extension-logger
mainInstall the logger extension via npm to enable console logging of connection, document, and lifecycle events for your Hocuspocus server. This is useful for debugging client connections, document loading, and incoming changes.
npm install @hocuspocus/extension-loggerInstall @hocuspocus/extension-database
mainInstall the database extension via npm to use it as a generic persistence base class for Hocuspocus.
npm install @hocuspocus/extension-databaseInstall @hocuspocus/provider and yjs
mainTo use the Hocuspocus client-side provider, you must install both
@hocuspocus/providerandyjsvia npm.npm install @hocuspocus/provider yjsInstall @hocuspocus/extension-s3
mainInstall the S3 extension via npm to enable S3-compatible persistence for Hocuspocus. This allows you to store Y.js documents in Amazon S3 or compatible services like MinIO, DigitalOcean Spaces, Cloudflare R2, or Backblaze B2.
npm install @hocuspocus/extension-s3Scale Hocuspocus horizontally with Redis
mainTo scale Hocuspocus across multiple server instances, point every instance to the same Redis server. This allows document updates and awareness to be broadcasted via Redis pub/sub, ensuring clients connected to different instances stay in sync.
Note: Redis handles real-time synchronization between instances, but it does not handle long-term storage. You must still use a persistence extension (like
@hocuspocus/extension-sqlite,@hocuspocus/extension-s3, or@hocuspocus/extension-database) to save documents to a database.import { Server } from "@hocuspocus/server" import { Redis } from "@hocuspocus/extension-redis" const server = new Server({ port: 1234, extensions: [ new Redis({ host: "127.0.0.1", port: 6379, }), ], }) server.listen()