WoWDBDefs

WoWDBDefs provides up-to-date column and field definitions for World of Warcraft database files (DBC and DB2). It serves as a human-readable and machine-readable source of truth for database structures across various WoW builds, ranging from version 7.3.5.26654 to the current version.

Key features include:

• Support for builds between 7.3.5.26654 and current.
• Machine-readable formats available via C# and Python implementations.
• A tool to convert DBD files to JSON or XML.

The DBD format begins with a COLUMNS section to define column names once, ensuring consistency throughout the file.

Column Types

• Regular: type ColName (e.g., int ColName)
• Foreign keys: type<ForeignDB::ForeignCol> ColName
• Localized strings: locstring ColName (equivalent to string type in 4.0+)
• Valid types: int, string, float, locstring.

Column Syntax Rules

• Unverified columns: Append a ? to the name (e.g., ColName?).
• Comments: Append // Comment goes here to the end of a line.
• Annotations: Use $annotation$ syntax. Supported annotations include:
  • $id$: Marks a primary key. Can be inline ($id$ColName) or non-inline ($noninline,id$).
  • $relation$: Marks a relationship. Can be inline ($relation$ColName) or non-inline ($noninline,relation$).

After the COLUMNS section, you define which builds and layouts the subsequent column structures apply to.

Version Keywords

• BUILD: (Required) Specifies the build version(s). Can be a single build, a comma-separated list of exact builds, or a range.
• LAYOUT: (Required if applicable) Specifies layout hashes. Starts with LAYOUT followed by a comma-separated list.
• COMMENT: (Optional) A human-readable comment for the version block.

Build Specification Formats

• Range: BUILD 7.2.0.23436-7.2.0.23514 (Note: For current expansions, use minor version ranges to avoid conflicts).
• Multiple exact builds: BUILD 0.7.0.3694, 0.7.1.3702, 0.7.6.3712
• Single exact build: BUILD 0.9.1.3810

The db-by-checksum.json file provides a grouping of databases based on their filename and MD5 checksum. The data is organized by <filename, MD5 checksum> and then sorted by <expansion, build>. This structure is useful for identifying identical database files across different builds using the ChecksumGroup class.

public class ChecksumGroup
{
  public string Name;
  public Dictionary<string, string[]> Builds; // <checksum, builds>
}

When new builds introduce changes, fields may appear as Field_x_x_x_xxxxx_yyy (where x is the build and y is the field index). These must be manually named to be useful.

Mapping Strategies:

• Minor changes: Look for fields that were added, resized, or removed.
• Reshuffled fields: If Blizzard shifts many fields at once, compare field types and sizes between builds. You may need to inspect field content using external tools to determine the correct mapping.
• New fields: Check Blizzard's Lua files or inspect other DB2s that changed in the same build to find clues about new features.

Best Practice: After manually naming fields, run DBDefsValidator.dll with the rewrite flag set to true to clean up orphaned definitions or whitespace.

The db-row-match feature (currently limited to Legion) performs a byte-level comparison of matching rows between two different builds. Databases are grouped by filename and sorted by <expansion, build>. The DBDMatchGroup class captures the comparison metrics between a previous build and the current build.

public class DBDMatchGroup
{
  public string BuildPrev;
  public string BuildCur;
  public int RecordCountPrev;
  public int RecordCountCur;
  public int MatchCount;
  public float MatchCountPercent;
}

Raw DBD dumps from DBDefsDumper lack field names and are specific to a single build. To integrate them into the main definition set, use DBDefsMerge to combine them with existing DBDs that contain named fields.

Workflow:

1. Compile the project in code/C#/DBDefsMerge.
2. Run the merger: dotnet DBDefsMerge.dll <dbddir> <rawdir> <outputdir>

Details:

• <dbddir>: The directory containing existing DBDs.
• <rawdir>: The directory containing the raw dumps.
• <outputdir>: The destination directory (can be the same as <dbddir>).
• Merging Logic: The tool attempts to match entries using layouthash first. If unavailable, it falls back to build identifiers, and finally build ranges.

When creating or editing .dbd files, adhere to these rules:

• Filename: Use DBName.dbd, where DBName is the exact name of the DBC/DB2.
• Multiple structures: If a file has multiple structures for different versions, separate them with an empty new line.
• Line endings: All files must use Unix line endings (\n).