nx-plugins
repository·main·Indexed 41 days ago
https://github.com/lucasvieirasilva/nx-pluginsA collection of Nx plugins and utilities to extend monorepo capabilities, featuring @nxlv/python for Python development with Poetry and uv support, and @nxlv/data-migration for TypeScript-based data migrations specifically designed for DynamoDB.
What's inside nx-plugins
- The Nx Data Migration Plugin provides a set of generators and executors designed to facilitate data migrations using TypeScript. It is specifically built to support DynamoDB features, making it suitable for moving data or schemas between states or performing data fixes (such as adding new fields to items or correcting typos in existing fields).
Overview of @nxlv/nx-plugins
mainThe@nxlv/nx-pluginsrepository provides a collection of extensible development tools designed for Nx monorepos. It includes specialized plugins for Python integration and data migration, along with utility libraries to support these workflows.Overview of @nxlv/data-migration
main@nxlv/data-migration is a separate package that provides tools for running data migrations. It includes:
- Executors and Generators for managing migration workflows.
- A
migrator/runtime that utilizes AWS SDK clients. - Support for decorators to define migration logic.
Understand the workspace structure and package layout
mainThis monorepo is managed with pnpm workspaces and Nx (version 22.3.3). The workspace layout is configured via
nx.jsonwithappsDirset toe2eandlibsDirset topackages.Directory Layout
packages/: Contains all publishable projects and libraries.e2e/: Contains end-to-end testing projects.tools/: Contains workspace-wide tooling and scripts.types/: Contains ambient type declarations.
Available Packages
Package Path npm Name Type Description packages/nx-python@nxlv/pythonNx plugin Main plugin containing executors, generators, migrations, and inference plugin. packages/data-migration@nxlv/data-migrationNx plugin DynamoDB-style data migration plugin with executors, generators, and a migratorruntime.packages/util@nxlv/utillibrary Shared pure string utility helpers (e.g., kebab-case,chunks).packages/data-migration-example(private) example A non-published example application demonstrating @nxlv/data-migration.How @nxlv/python works with Poetry and uv
mainThe
@nxlv/pythonplugin provides a unified abstraction for managing Python projects within an Nx workspace, supporting both Poetry and uv package managers.Instead of calling package managers directly, the plugin uses a Provider abstraction (
src/provider/). This allows executors and generators to perform operations likeinstall,build,lock, orsyncwithout needing to know which specific package manager is being used.Package Manager Resolution Logic: When you run a command, the plugin determines the provider using
getProviderin this order:- Explicit Configuration: If the
packageManageroption is set to'poetry'or'uv', that manager is used. - Auto-detection: The plugin inspects the project's
pyproject.tomland looks for lockfiles (uv.lockvspoetry.lock). - Fallback: If no lockfiles are found and no configuration is provided, it defaults to Poetry.
Note: If both
uv.lockandpoetry.lockare present, the plugin will throw an error due to ambiguity.- Explicit Configuration: If the
Test Executors and Providers (Pattern A)
mainWhen testing executors or providers, the goal is to mock the filesystem and child processes to avoid hitting the real OS.
Key Strategies
- Mock Executable Checks: Use
vi.spyOnon utility modules (e.g.,poetryUtils.checkPoetryExecutable) to returnundefinedso tests don't require real binaries. - Mock Shell Commands: Use
vi.mocked(spawn.sync).mockReturnValue(...)to simulate command execution. - Stub Process Changes: Stub
process.chdirsince executors typically change directory into the workspace root. - Verify via Memfs: Assert that the correct files (like
pyproject.toml) were written by reading them from thememfsvolume. - Cleanup: Always call
vol.reset()andvi.resetAllMocks()in anafterEachblock to ensure test isolation.
describe('Add Executor', () => { afterEach(() => { vol.reset(); // wipe the in-memory FS vi.resetAllMocks(); }); describe('poetry', () => { beforeEach(() => { vi.spyOn(poetryUtils, 'checkPoetryExecutable').mockResolvedValue(undefined); vi.spyOn(poetryUtils, 'getPoetryVersion').mockResolvedValue('1.8.2'); vi.spyOn(PoetryProvider.prototype, 'activateVenv').mockResolvedValue(undefined); vi.mocked(spawn.sync).mockReturnValue({ status: 0, output: [''], pid: 0, signal: null, stderr: null, stdout: null }); vi.spyOn(process, 'chdir').mockReturnValue(undefined); }); // ... assertions }); });- Mock Executable Checks: Use
Available @nxlv/nx-plugins packages
mainThe ecosystem is divided into specific plugins and libraries:
Plugins
- @nxlv/nx-python: Provides Nx support for Python projects.
- @nxlv/data-migration: An Nx plugin specifically for managing data migrations.
Libraries
- @nxlv/util: Shared utility libraries used within the ecosystem.
Test Generators using Nx Tree and Snapshots (Pattern B)
mainWhen testing Nx generators, use an in-memory Nx
Treeinstead of the real filesystem. This ensures tests are fast and do not leave side effects on your workspace.Implementation Steps
- Initialize Tree: Use
createTreeWithEmptyWorkspace({ layout: 'apps-libs' })from@nx/devkit/testingto create a fresh in-memory workspace. - Run Generator: Pass the
appTreeto your generator function. - Verify with Snapshots: Use
expect(...).toMatchSnapshot()to verify the state of the project configuration and generated files. - Read Results: Use
@nx/devkithelpers likereadProjectConfiguration,readNxJson, ortree.read(path, 'utf-8')to inspect the tree.
Updating Snapshots
If you intentionally change the generator output, update the snapshots using:
pnpm nx test <package-name> -- -uimport { createTreeWithEmptyWorkspace } from '@nx/devkit/testing'; import { Tree, readProjectConfiguration, readNxJson } from '@nx/devkit'; import generator from './generator'; let appTree: Tree; beforeEach(() => { vi.resetAllMocks(); appTree = createTreeWithEmptyWorkspace({ layout: 'apps-libs' }); }); it('scaffolds the project', async () => { await generator(appTree, options); expect(readProjectConfiguration(appTree, 'test')).toMatchSnapshot(); expect(appTree.read('test/pyproject.toml', 'utf-8')).toMatchSnapshot(); });- Initialize Tree: Use
Handle Vitest hook execution order changes
mainVitest hooks (
beforeAll,beforeEach,afterAll,afterEach) now execute serially instead of in parallel. Additionally,afterAllandafterEachhooks now run in reverse declaration order.If your tests rely on parallel hook side effects or specific teardown ordering, you may need to audit your shared state mutations. You can revert to parallel behavior by configuring the
sequence.hooksoption, though this is recommended only as a last resort.// To revert to parallel behavior: export default defineConfig({ test: { sequence: { hooks: 'parallel', }, }, });How automatic dependency syncing works
mainWhen
inferDependencies: trueis enabled, the@nxlv/pythondependency graph plugin scans source code for local package imports. This process runs in the background via the Nx Daemon.Once the dependency graph is complete, the sync generator (
@nxlv/python:pkg-sync) compares the project's current dependencies against the graph. If a local dependency is missing, the generator automatically adds it topyproject.tomland updates the dependency tree and lock files.How the Nx Data Migration engine works
mainThe data migration engine executes TypeScript scripts, manages migration state, and handles rollbacks.
Migration State
State is stored in a DynamoDB table, which is automatically created upon the first execution.
- Default Table Name:
migration-states-{region}-{env}(e.g.,migration-states-us-east-1-development). - Customization: Override the table name by setting the
migrationTableNameproperty in yournx.jsonfile.
Migration Lifecycle
Migrations follow two primary lifecycle phases defined in the script:
before:deploy: For migrations that must run before the application is deployed.after:deploy: For migrations that must run after the application is deployed.
Migration Baseline
- Default Table Name:
Customize project generation with templateDir
mainIf you need to customize the files generated by
@nxlv/pythonwithout writing a custom generator, you can use the--templateDiroption. This allows you to provide a custom directory containing EJS templates.Template Variables
The following variables are available within your EJS templates:
packageName: The value provided via--packageName.offsetFromRoot: Relative path from the project to the root (e.g.,../../forapps/my-project).projectRoot: Relative path of the project.individualPackage: Boolean indicating if the workspace uses a shared virtual environment.dot: A string containing a dot (.), used for files like.gitignore(e.g.,__dot__gitignore.template).template: An empty string used to handle files without extensions in conjunction withdot.