repository·main·Indexed 43 days ago
https://github.com/surrealdb/surrealdb.goThe official SurrealDB SDK for Go allows developers to interact with SurrealDB databases, performing queries and data operations directly from Go applications. It includes a type-safe SurrealQL query builder and utility tools such as surrealdump for CBOR format database dumps and surrealrestore for point-in-time and incremental restorations.
SurrealNote is a backend-only demonstration project designed to showcase zero-downtime database migration patterns. Specifically, it demonstrates migrating from PostgreSQL to SurrealDB using the Command Query Responsibility Segregation (CQRS) pattern within a hierarchical note-taking application context.
Note: This is a REST API demonstration with no UI and is intended for educational/demonstration purposes regarding migration patterns, not for production use.
Starting in v1.0 (introduced in v0.11.0), the default CBOR implementation changed from fxamacker/cbor to surrealcbor.
surrealcbor is optimized for the SurrealDB protocol and provides:
NONE tag (unmarshals to Go nil).Critical Behavioral Change: When selecting a non-existent record, the implementations differ:
fxamacker/cbor): Returns a struct with nil fields (e.g., user.ID == nil).surrealcbor): Returns nil for the entire struct (e.g., user == nil).You must update your error handling logic to check for the nullability of the entire struct rather than individual fields when a record is not found.
// Updated code for surrealcbor default:
user, err := surrealdb.Select[User](ctx, db, recordID)
if err != nil {
return err
}
if user == nil {
// Handle non-existent record
}Because SurrealDB does not expose transaction versionstamps directly to clients and backend isolation models vary, a standard full dump taken during active transactions may be internally inconsistent (containing some records from the start of the dump and others from the end).
surrealdump solves this by combining an inconsistent full dump with change feed data. It captures the complete change history from before the dump started (vs_0) through the end of the dump (vs_2). During restoration, it applies the inconsistent dump and then replays all changes from the change feed to bring the database to a consistent state at the EndVersionstamp (vs_2).
The surrealdb package uses the Concise Binary Object Representation (CBOR) format for communication with the backend. While most SurrealDB types map to idiomatic Go types (like string, bool, or time.Time), the library provides custom types for specific SurrealDB models that cannot be represented by standard Go types (such as Record IDs, Table names, or Geometry types).
When working with the SDK, use the provided surrealdb types to ensure correct serialization and deserialization of complex data structures.
To use incremental dumps, you must enable Change Feeds on your SurrealDB database before any data is created. This is done using the DEFINE DATABASE command with a CHANGEFEED retention period. Without this, incremental dumps will not capture changes and full dumps may be inconsistent.
DEFINE DATABASE OVERWRITE <database_name> CHANGEFEED <retention_period>
-- Example: 1 hour retention
DEFINE DATABASE OVERWRITE mydb CHANGEFEED 1hScopes are used to control access and visibility. A memory.ScopeSets is a [][]string representing a DNF (OR-of-ANDs) selector.
"team/eng" or "org/apple/product/ipad")."key/value".Examples:
memory.ScopeSets{{"team/acme"}}: A single scope.memory.ScopeSets{{"team/a"}, {"team/b"}}: team/a OR team/b.memory.ScopeSets{{"team/b", "clearance/secret"}}: team/b AND clearance/secret.Note: Labels (in Labels fields) use key=value format, whereas scope paths use the slash / format.
client.Remember(ctx, &memory.RememberRequest{
Text: "...",
Scopes: memory.ScopeSets{{"team/acme"}},
})
client.Documents().Upload(ctx, body,
memory.WithScopes(memory.ScopeSets{{"team/acme", "user/tobie"}}),
)The surrealdump system uses a chain-building mechanism to manage relationships between full and incremental dumps. This allows for reliable point-in-time recovery by scanning a directory and constructing a valid sequence of files.
.manifest.json files.To prevent data corruption, the system enforces:
You cannot continue an existing dump chain after a restoration because restored databases have different versionstamps than the original. To start a new backup chain from a restored database, follow these steps:
DEFINE DATABASE OVERWRITE <database_name> CHANGEFEED <retention_period>surrealdump.surrealdump -endpoint ws://localhost:8001 -namespace restored -database prod -output new-full.cborsurrealdump -incremental -endpoint ws://localhost:8001 -namespace restored -database prod -output new-inc.cborVersionstamps are unique to each database instance. When you restore a database, the new instance will have different versionstamps than the original. Consequently, you cannot continue an old incremental chain on a restored database.
To start a new incremental backup chain on a restored database:
# 1. Take a new full dump of the restored database
surrealdump -endpoint ws://localhost:8001 -namespace myapp -database prod -dir backups_new -output restored-full.cbor
# 2. Now you can take incremental dumps based on the restored database
surrealdump -incremental -endpoint ws://localhost:8001 -namespace myapp -database prod -dir backups_new -output restored-inc1.cborAs of v1.7.0, the spectron package has been renamed to memory. There is no compatibility shim, so you must update your import paths and package qualifiers. All exported names remain unchanged.
Migration steps:
github.com/surrealdb/surrealdb.go/spectron with github.com/surrealdb/surrealdb.go/memory in your imports.spectron.New(...) to memory.New(...).If you cannot migrate immediately, pin your dependency to v1.6.0 to continue using the spectron path.
-import "github.com/surrealdb/surrealdb.go/spectron"
+import "github.com/surrealdb/surrealdb.go/memory"
-client, err := spectron.New(ctx, endpoint, key)
+client, err := memory.New(ctx, endpoint, key)Apply incremental changes from an incremental dump using change feed data. This applies changes in order based on versionstamps.
You must use the -incremental flag.
surrealrestore -incremental -endpoint ws://localhost:8000 -username root -password root -input increment.cborInstall the surrealdump tool using go install. This tool is a Go-based utility for dumping SurrealDB databases to CBOR format, intended for demonstrating SDK capabilities.
go install github.com/surrealdb/surrealdb.go/contrib/surrealdump/cmd/surrealdump@latest