nx-plugins

repository·main·Indexed 41 days ago

https://github.com/lucasvieirasilva/nx-plugins

A collection of Nx plugins and utilities to extend monorepo capabilities, featuring @nxlv/python for Python development with Poetry and uv support, and @nxlv/data-migration for TypeScript-based data migrations specifically designed for DynamoDB.

Tokens
32.7K
Snippets
93
Records
132
Agent score
100%

What's inside nx-plugins

  1. Overview of the Nx Data Migration Plugin

    main
    The Nx Data Migration Plugin provides a set of generators and executors designed to facilitate data migrations using TypeScript. It is specifically built to support DynamoDB features, making it suitable for moving data or schemas between states or performing data fixes (such as adding new fields to items or correcting typos in existing fields).
  2. Overview of @nxlv/nx-plugins

    main
    The @nxlv/nx-plugins repository provides a collection of extensible development tools designed for Nx monorepos. It includes specialized plugins for Python integration and data migration, along with utility libraries to support these workflows.
  3. Understand the workspace structure and package layout

    main

    This monorepo is managed with pnpm workspaces and Nx (version 22.3.3). The workspace layout is configured via nx.json with appsDir set to e2e and libsDir set to packages.

    Directory Layout

    • packages/: Contains all publishable projects and libraries.
    • e2e/: Contains end-to-end testing projects.
    • tools/: Contains workspace-wide tooling and scripts.
    • types/: Contains ambient type declarations.

    Available Packages

    Package Pathnpm NameTypeDescription
    packages/nx-python@nxlv/pythonNx pluginMain plugin containing executors, generators, migrations, and inference plugin.
    packages/data-migration@nxlv/data-migrationNx pluginDynamoDB-style data migration plugin with executors, generators, and a migrator runtime.
    packages/util@nxlv/utillibraryShared pure string utility helpers (e.g., kebab-case, chunks).
    packages/data-migration-example(private)exampleA non-published example application demonstrating @nxlv/data-migration.
  4. How @nxlv/python works with Poetry and uv

    main

    The @nxlv/python plugin provides a unified abstraction for managing Python projects within an Nx workspace, supporting both Poetry and uv package managers.

    Instead of calling package managers directly, the plugin uses a Provider abstraction (src/provider/). This allows executors and generators to perform operations like install, build, lock, or sync without needing to know which specific package manager is being used.

    Package Manager Resolution Logic: When you run a command, the plugin determines the provider using getProvider in this order:

    1. Explicit Configuration: If the packageManager option is set to 'poetry' or 'uv', that manager is used.
    2. Auto-detection: The plugin inspects the project's pyproject.toml and looks for lockfiles (uv.lock vs poetry.lock).
    3. Fallback: If no lockfiles are found and no configuration is provided, it defaults to Poetry.

    Note: If both uv.lock and poetry.lock are present, the plugin will throw an error due to ambiguity.

  5. Test Executors and Providers (Pattern A)

    main

    When testing executors or providers, the goal is to mock the filesystem and child processes to avoid hitting the real OS.

    Key Strategies

    1. Mock Executable Checks: Use vi.spyOn on utility modules (e.g., poetryUtils.checkPoetryExecutable) to return undefined so tests don't require real binaries.
    2. Mock Shell Commands: Use vi.mocked(spawn.sync).mockReturnValue(...) to simulate command execution.
    3. Stub Process Changes: Stub process.chdir since executors typically change directory into the workspace root.
    4. Verify via Memfs: Assert that the correct files (like pyproject.toml) were written by reading them from the memfs volume.
    5. Cleanup: Always call vol.reset() and vi.resetAllMocks() in an afterEach block to ensure test isolation.
    describe('Add Executor', () => {
      afterEach(() => {
        vol.reset(); // wipe the in-memory FS
        vi.resetAllMocks();
      });
    
      describe('poetry', () => {
        beforeEach(() => {
          vi.spyOn(poetryUtils, 'checkPoetryExecutable').mockResolvedValue(undefined);
          vi.spyOn(poetryUtils, 'getPoetryVersion').mockResolvedValue('1.8.2');
          vi.spyOn(PoetryProvider.prototype, 'activateVenv').mockResolvedValue(undefined);
          vi.mocked(spawn.sync).mockReturnValue({ status: 0, output: [''], pid: 0, signal: null, stderr: null, stdout: null });
          vi.spyOn(process, 'chdir').mockReturnValue(undefined);
        });
        // ... assertions
      });
    });
  6. Available @nxlv/nx-plugins packages

    main

    The ecosystem is divided into specific plugins and libraries:

    Plugins

    • @nxlv/nx-python: Provides Nx support for Python projects.
    • @nxlv/data-migration: An Nx plugin specifically for managing data migrations.

    Libraries

    • @nxlv/util: Shared utility libraries used within the ecosystem.
  7. Test Generators using Nx Tree and Snapshots (Pattern B)

    main

    When testing Nx generators, use an in-memory Nx Tree instead of the real filesystem. This ensures tests are fast and do not leave side effects on your workspace.

    Implementation Steps

    1. Initialize Tree: Use createTreeWithEmptyWorkspace({ layout: 'apps-libs' }) from @nx/devkit/testing to create a fresh in-memory workspace.
    2. Run Generator: Pass the appTree to your generator function.
    3. Verify with Snapshots: Use expect(...).toMatchSnapshot() to verify the state of the project configuration and generated files.
    4. Read Results: Use @nx/devkit helpers like readProjectConfiguration, readNxJson, or tree.read(path, 'utf-8') to inspect the tree.

    Updating Snapshots

    If you intentionally change the generator output, update the snapshots using:

    pnpm nx test <package-name> -- -u
    import { createTreeWithEmptyWorkspace } from '@nx/devkit/testing';
    import { Tree, readProjectConfiguration, readNxJson } from '@nx/devkit';
    import generator from './generator';
    
    let appTree: Tree;
    
    beforeEach(() => {
      vi.resetAllMocks();
      appTree = createTreeWithEmptyWorkspace({ layout: 'apps-libs' });
    });
    
    it('scaffolds the project', async () => {
      await generator(appTree, options);
      expect(readProjectConfiguration(appTree, 'test')).toMatchSnapshot();
      expect(appTree.read('test/pyproject.toml', 'utf-8')).toMatchSnapshot();
    });
  8. Handle Vitest hook execution order changes

    main

    Vitest hooks (beforeAll, beforeEach, afterAll, afterEach) now execute serially instead of in parallel. Additionally, afterAll and afterEach hooks now run in reverse declaration order.

    If your tests rely on parallel hook side effects or specific teardown ordering, you may need to audit your shared state mutations. You can revert to parallel behavior by configuring the sequence.hooks option, though this is recommended only as a last resort.

    // To revert to parallel behavior:
    export default defineConfig({
      test: {
        sequence: {
          hooks: 'parallel',
        },
      },
    });
  9. How automatic dependency syncing works

    main

    When inferDependencies: true is enabled, the @nxlv/python dependency graph plugin scans source code for local package imports. This process runs in the background via the Nx Daemon.

    Once the dependency graph is complete, the sync generator (@nxlv/python:pkg-sync) compares the project's current dependencies against the graph. If a local dependency is missing, the generator automatically adds it to pyproject.toml and updates the dependency tree and lock files.

  10. How the Nx Data Migration engine works

    main

    The data migration engine executes TypeScript scripts, manages migration state, and handles rollbacks.

    Migration State

    State is stored in a DynamoDB table, which is automatically created upon the first execution.

    • Default Table Name: migration-states-{region}-{env} (e.g., migration-states-us-east-1-development).
    • Customization: Override the table name by setting the migrationTableName property in your nx.json file.

    Migration Lifecycle

    Migrations follow two primary lifecycle phases defined in the script:

    1. before:deploy: For migrations that must run before the application is deployed.
    2. after:deploy: For migrations that must run after the application is deployed.

    Migration Baseline

  11. Customize project generation with templateDir

    main

    If you need to customize the files generated by @nxlv/python without writing a custom generator, you can use the --templateDir option. This allows you to provide a custom directory containing EJS templates.

    Template Variables

    The following variables are available within your EJS templates:

    • packageName: The value provided via --packageName.
    • offsetFromRoot: Relative path from the project to the root (e.g., ../../ for apps/my-project).
    • projectRoot: Relative path of the project.
    • individualPackage: Boolean indicating if the workspace uses a shared virtual environment.
    • dot: A string containing a dot (.), used for files like .gitignore (e.g., __dot__gitignore.template).
    • template: An empty string used to handle files without extensions in conjunction with dot.