cnpmcore

repository·master·Indexed 20 days ago

https://github.com/cnpm/cnpmcore

A TypeScript-based reimplementation of cnpmjs.org used to power private NPM registries for enterprises and the npmmirror.com China NPM mirror. It includes a standard Registry HTTP API, an Internal API for advanced operations like package synchronization and administration, and integration guides for development pipelines.

Tokens
32.6K
Snippets
102
Records
118
Agent score
72%

What's inside cnpmcore

  1. NPM Registry API Overview

    master

    The NPM Registry API provides programmatic access to the registry's data and services. All communication is performed over HTTPS or HTTP using the registry.npmjs.org domain, and all data is exchanged in JSON format.

    The API supports several core functional areas:

    • Package: Operations related to package metadata and publishing.
    • User: User-related information and management.
    • Search: Querying the registry for packages.
    • Token: Managing authentication tokens.
    • Trend: Accessing trending package data.
    $ curl -i https://registry.npmjs.org
    
    HTTP/1.1 200 OK
    
    {
      "db_name": "registry",
      "doc_count": 123772,
      "doc_del_count": 377,
      "update_seq": 685591,
      "purge_seq": 0,
      "compact_running": false,
      "disk_size": 634187899,
      "data_size": 445454185,
      "instance_start_time": "1420670152481614",
      "disk_format_version": 6,
      "committed_update_seq": 685591
    }
  2. Understand the CPU Profile Analysis Report structure

    master

    A CPU Profile Analysis Report (generated via xprofiler-cpu-profile) provides a breakdown of resource consumption during a profiling session. It is used to identify performance bottlenecks in application code, third-party NPM packages, or Node.js core modules.

    Key sections include:

    • Profile Information: Metadata about the session (duration, sample count, total nodes).
    • CPU Time Distribution: A breakdown of time spent in Idle, Program, GC (Garbage Collection), and Active/User modes.
    • Top 30 Functions by Self Time: Identifies specific functions consuming the most CPU time.
    • Top 30 Files/Modules by CPU Time: Identifies which files or modules are responsible for the most CPU load.
    • CPU Time by Category: Aggregates time into high-level buckets: NPM Packages, Native/V8, Node.js Core, and Application Code.
    • Application Code Hotspots: Specifically highlights functions within the user's own application code that are consuming significant resources.
  3. Understand the cnpmcore architecture and layers

    master

    The project follows a layered architecture inspired by Domain-Driven Design (DDD). The layers are organized as follows:

    • app/port/controller (Port Layer): HTTP controllers that handle incoming requests. They inherit from AbstractController and MiddlewareController.
    • app/core (Core Layer): Contains the business logic.
      • entity: Core models implementing business behavior.
      • service: Core business logic (e.g., PackageManagerService).
      • event: Asynchronous event definitions and consumers.
    • app/repository (Repository Layer): Handles data persistence.
      • model: ORM models (data definitions).
      • XXXRepository: Interfaces for storage and querying.
    • app/common (Common Layer): Shared utilities and external service adapters.
    • app/infra (Infrastructure Layer): Real implementations of adapters (e.g., NFSClientAdapter, AuthAdapter) tailored for specific PaaS environments.
  4. Understand the cnpmcore Org and Team permission model

    master

    cnpmcore uses a hierarchical permission model: Organization (Org) $\rightarrow$ Team $\rightarrow$ Package.

    • Org: Corresponds to an npm scope (e.g., mycompany $\rightarrow$ @mycompany).
    • OrgMember: Users in an Org with roles owner (can manage the Org) or member.
    • Team: A permission unit within an Org. Every Org automatically includes a default developers team.
    • TeamMember: Users within a Team with roles owner (can manage the Team) or member (can only read authorized packages).
    • TeamPackage: Represents the read authorization granted to a Team for a specific package.

    This model extends the standard npm model by adding roles to Team members, allowing Team Owners to manage their own teams without needing full Organization Owner permissions.

    | Concept | Description |
    | :--- | :--- |
    | **Org** | Organization, corresponds to a scope (e.g. org `mycompany` $\rightarrow$ `@mycompany`) |
    | **OrgMember** | Organization member, role is `owner` (manageable) or `member` |
    | **Team** | Permission unit. Each Org automatically creates a `developers` default team |
    | **TeamMember** | Team member, role is `owner` (can manage team) or `member` |
    | **TeamPackage** | Team's read authorization for a package |
  5. Configure Private Package Read Authentication

    master

    cnpmcore allows you to restrict access to packages within specific scopes using Team-Package binding. By default, all packages in a self-scope are publicly readable. Read authentication is only enforced for packages that have an explicit Team binding.

    Authentication Logic for allowScopes:

    1. Scope not in allowScopes: Public package, no authentication required.
    2. Scope in allowScopes (self scope) AND no Team binding: Publicly readable.
    3. Scope in allowScopes (self scope) AND has Team binding:
      • Not logged in: 401 Unauthorized
      • Admin user: Allowed
      • User is in a Team with access to the package: Allowed
      • Otherwise: 403 Forbidden

    CDN Behavior:

    • Packages in a self-scope use Cache-Control: private, no-store and are not cached by CDNs.
    • Non-self scope packages use their original caching strategy.
  6. Understand cnpmcore's Org and Team permission model

    master

    cnpmcore uses a hierarchical permission model: Organization $\rightarrow$ Team $\rightarrow$ Package.

    • Org (Organization): Corresponds to a scope (e.g., mycompany maps to @mycompany).
    • OrgMember: Users within an Org with roles owner (can manage the Org) or member.
    • Team: A permission unit within an Org. Every Org automatically includes a default developers team.
    • TeamMember: Users within a Team with roles owner (can manage the Team) or member (read-only access to authorized packages).
    • TeamPackage: Represents the specific read access granted to a Team for a package.

    This model extends the standard npm model by adding roles to Team members, allowing for granular control where a Team Owner can manage their specific team without needing full Organization Owner permissions.

  7. Configure private package read authentication via allowScopes

    master

    cnpmcore uses allowScopes (self-scopes) to manage private package access. By default, all packages within a self-scope are publicly readable. Read authentication is only enforced when a specific Team-Package binding is established.

    Authentication Logic for allowScopes packages:

    1. No Team Binding: The package is publicly readable by anyone (no login required).
    2. With Team Binding:
      • Unauthenticated users: Receive 401.
      • Admin users: Access granted.
      • Team members: Access granted if the user belongs to a Team that has been authorized to access that specific package.
      • Others: Receive 403.

    CDN Behavior

    • Packages in allowScopes are served with Cache-Control: private, no-store to prevent CDN caching.
    • Packages outside of allowScopes follow standard CDN caching policies.
  8. Understand the npmmirror.com Usage Policy

    master

    npmmirror.com is the China NPM mirror powered by cnpmcore. To ensure service stability, users must adhere to the following policies:

    • No Abuse: Do not abuse the registry with excessive crawling or scraping.
    • Rate Limiting: Automated access that generates unusually high traffic may be rate-limited without notice.
    • Large-scale Usage: If you have large-scale requirements or are being affected by rate limiting, contact the maintainers by opening a GitHub Issue.
  9. Manually install MySQL and Redis on macOS

    master

    If you prefer not to use Docker, you can install the required dependencies manually using Homebrew on macOS.

    MySQL 9

    Install and start the MySQL service:

    brew install mysql
    brew services start mysql

    Redis

    Install and start the Redis service:

    brew install redis
    brew services start redis
    # MySQL 9
    brew install mysql
    brew services start mysql
    
    # Redis
    brew install redis
    brew services start redis
  10. Bundle cnpmcore into a self-contained artifact

    master

    You can bundle cnpmcore into a self-contained, deployable CommonJS artifact using @eggjs/egg-bundler. This is driven by the egg-bin bundle command and runs in Egg's single-process mode (mode: 'single'), where the agent and worker share the same process. This mode is ideal for accelerating cold starts and reducing deployment image sizes in Serverless environments.

    Requirements: Ensure your version of egg, @eggjs/core, or @eggjs/tegg-plugin includes the bundle startup path fix. Use a release version that contains this fix.

    Output: By default, the bundle is output to ./dist-bundle and contains:

    • worker.js (the entry point)
    • Various @utoo/pack chunks
    • Copied runtime resources (e.g., app/port/*.html)
    • bundle-manifest.json
    npx egg-bin bundle
  11. Manage Organization Members

    master

    Admin or Org Owners can manage members within an organization. When a new member is added, they are automatically added to the Org's default developers team.

    Add a Member

    • npm CLI: npm org set <orgName> <username> [role]
    • HTTP: PUT /-/org/:orgName/member with a JSON body {"user": "<username>", "role": "<role>"}.

    Remove a Member

    • npm CLI: npm org rm <orgName> <username>
    • HTTP: DELETE /-/org/:orgName/member/<username>.
    • Note: Removing a member from an Org automatically removes them from all teams within that Org.

    View Members

    • npm CLI: npm org ls <orgName> (returns a list of usernames).
    • HTTP: GET /-/org/:orgName/member (returns an object mapping usernames to roles, e.g., {"alice": "owner"}).

    View User's Teams

    Use GET /-/org/:orgName/member/:username/team to see which teams a specific user belongs to.

    # Add member via npm CLI
    npm org set mycompany alice --registry=http://localhost:7001
    
    # Add member via HTTP
    curl -X PUT http://localhost:7001/-/org/mycompany/member \
      -H "Authorization: Bearer <admin-token>" \
      -H "Content-Type: application/json" \
      -d '{"user": "alice", "role": "member"}'
    
    # Remove member via npm CLI
    npm org rm mycompany alice --registry=http://localhost:7001
    
    # View user's teams
    curl http://localhost:7001/-/org/mycompany/member/alice/team \
      -H "Authorization: Bearer <token>"
    # Returns: [{"name": "developers", "description": "...", "role": "owner"}, ...]
  12. Manage Organizations via npm CLI and HTTP API

    master

    Administrators or Org Owners can manage organization membership and lifecycle.

    Create an Organization

    Use a PUT request to the /-/org endpoint.

    Delete an Organization

    Use a DELETE request to /-/org/:orgName. Warning: This performs a cascade delete of all teams, members, and package grants within that Org.

    Manage Members

    • Add Member: Use npm org set <org> <user> or a PUT request to /-/org/:orgName/member. New members are automatically added to the developers team.
    • Remove Member: Use npm org rm <org> <user> or a DELETE request to /-/org/:orgName/member/:username. This also removes them from all teams in the Org.
    • List Members: Use npm org ls <org> or a GET request to /-/org/:orgName/member.
    # Create Org
    curl -X PUT http://localhost:7001/-/org \
      -H "Authorization: Bearer <admin-token>" \
      -H "Content-Type: application/json" \
      -d '{"name": "mycompany", "description": "My Company"}'
    
    # Add Member (npm CLI)
    npm org set mycompany alice --registry=http://localhost:7001