repository·master·Indexed 23 days ago
https://github.com/cysharp/mastermemoryA high-performance, embedded, read-only in-memory document database for .NET and Unity. It utilizes C# source generators to provide type-safe, zero-allocation querying based on user-defined schemas, offering a smaller memory footprint and faster performance than SQLite.
Tables are defined using classes or records marked with the [MemoryTable] attribute. The class must also be serializable by MessagePack (use [MessagePackObject(true)]).
[MemoryTable("name")]: Defines the table name used in the binary data.[PrimaryKey]: Marks the unique identifier for the table. Required for every table.[SecondaryKey(indexNo, keyOrder)]: Defines secondary indexes. Use [NonUnique] if the key does not guarantee a single result.[StringComparisonOption]: Configures how string keys are compared (default is Ordinal).[IgnoreMember]: Use this on computed properties to prevent MessagePack from attempting to serialize them.[MemoryTable("person"), MessagePackObject(true)]
public record Person
{
[PrimaryKey]
public required int PersonId { get; init; }
[SecondaryKey(0), NonUnique]
[SecondaryKey(1, keyOrder: 1), NonUnique]
public required int Age { get; init; }
[SecondaryKey(2), NonUnique]
[SecondaryKey(1, keyOrder: 0), NonUnique]
public required Gender Gender { get; init; }
public required string Name { get; init; }
}To use MasterMemory in a standard .NET project, install the library (which includes both the Runtime and the Source Generator/Analyzer) via NuGet.
dotnet add package MasterMemoryMasterMemory follows a two-phase workflow: Build (creating the binary data) and Query (loading the binary into a MemoryDatabase).
Use DatabaseBuilder to collect data and generate a byte array.
Load the byte array into a MemoryDatabase. The generated table classes provide type-safe methods for searching.
Query Methods:
FindBy***(key): Returns a single item. Throws KeyNotFoundException if not found (unless IsReturnNullIfKeyNotFound is enabled in options).TryFindBy***(key, out result): Returns bool indicating if the key exists.FindClosestBy***(key, selectLower): Returns the nearest value (default is lower).FindRangeBy***(min, max): Returns a RangeView<T> containing items within the inclusive range.MasterMemory requires Unity 2022.3.12f1 or later to support C# Incremental Source Generators.
Note on C# features:
Since Unity may not support the required keyword (C# 11), use the init keyword to ensure immutability. You may need to define IsExternalInit to enable init support:
namespace System.Runtime.CompilerServices
{
internal sealed class IsExternalInit { }
}Generated table classes are partial. You can extend them by creating another partial class in the same namespace. This allows you to add custom methods or perform post-construction logic using the OnAfterConstruct method.
Use OnAfterConstruct to initialize cached fields or derived data after the table is fully loaded.
public sealed partial class MonsterTable
{
int maxHp;
readonly int minHp;
partial void OnAfterConstruct()
{
maxHp = All.Select(x => x.MaxHp).Max();
// Use Unsafe.AsRef to set readonly fields
Unsafe.AsRef(minHp) = All.Select(x => x.MaxHp).Min();
}
public IEnumerable<Monster> GetRangedMonster(int arg1)
{
return All.Where(x => x.Hp > arg1);
}
}Configure the behavior of the Source Generator using the [assembly: MasterMemoryGeneratorOptions] attribute. This can be placed in any .cs file in your project.
Namespace: The namespace for generated files. Defaults to the project's RootNamespace or MasterMemory.IsReturnNullIfKeyNotFound: If true, FindBy*** methods return T? (null) instead of throwing KeyNotFoundException when a key is missing.PrefixClassName: Adds a prefix to generated classes (e.g., FooDatabaseBuilder instead of DatabaseBuilder). Useful for avoiding conflicts in multi-project solutions.[assembly: MasterMemoryGeneratorOptions(
Namespace = "MyConsoleApp",
IsReturnNullIfKeyNotFound = true,
PrefixClassName = "Foo"
)]You can implement custom validation logic by having your table classes implement IValidatable<T>. The MemoryDatabase.Validate() method will then execute these rules.
Key Validation Capabilities:
validator.GetReferenceSet<TRef>().Validate(predicate).validator.CallOnce() to perform checks that require looking at the entire table (e.g., uniqueness of a non-indexed field).[MemoryTable("quest_master"), MessagePackObject(true)]
public class Quest : IValidatable<Quest>
{
[PrimaryKey] public int Id { get; }
public int RewardId { get; }
public int Cost { get; }
void IValidatable<Quest>.Validate(IValidator<Quest> validator)
{
// Check if RewardId exists in Item table
var items = validator.GetReferenceSet<Item>();
if (this.RewardId > 0)
{
items.Exists(x => x.RewardId, x => x.ItemId);
}
// Range check
validator.Validate(x => x.Cost >= 10 && x.Cost <= 20);
// Global uniqueness check
if (validator.CallOnce())
{
var quests = validator.GetTableSet();
quests.Where(x => x.RewardId != 0).Unique(x => x.RewardId);
}
}
}Since MemoryDatabase is read-only, use ToImmutableBuilder() to create a builder that allows adding, removing, or replacing data. Once modifications are complete, call .Build() to create a new MemoryDatabase instance.
// Create builder from existing database
var builder = db.ToImmutableBuilder();
// Modify data
builder.Diff(addOrReplaceData);
builder.RemovePerson(new[] { 1, 10, 100 });
builder.ReplaceAll(newData);
// Create new database
MemoryDatabase newDatabase = builder.Build();
// Convert back to DatabaseBuilder to save to file
var newBuilder = newDatabase.ToDatabaseBuilder();
var newBinary = newBuilder.Build();For large databases, the construction time of MemoryDatabase can be significant. You can speed up the process by enabling parallel construction using the maxDegreeOfParallelism parameter in the constructor.
It is recommended to use Environment.ProcessorCount to utilize all available CPU cores.
var database = new MemoryDatabase(bin, maxDegreeOfParallelism: Environment.ProcessorCount);