Apache DevLake Documentation

The Domain Layer provides a set of Platform Independent Entities that allow DevLake to calculate metrics across different platforms (e.g., Jira vs. TAPD, or GitHub vs. GitLab).

DevLake categorizes all platforms into three main categories:

1. Project Management
2. Code Hosting
3. Devops

By abstracting common properties into Domain Entities, DevLake can use a single metric implementation to support multiple underlying platforms.

DevLake supports two primary types of plugins that hook into the core at runtime to extend capabilities like integrating new data sources, transforming data, or exporting data to other systems:

1. Go plugins: The primary plugin type. Developers write Go code to fetch data from sources, convert it into normalized DevLake data models, and store it. These are shared libraries built using Go's plugin package.
2. Python plugins: Written entirely in Python. They perform the same functions as Go plugins but communicate with the DevLake framework via RPC calls (currently based on shell calls).

When using the CircleCI plugin, DevLake maps CircleCI's hierarchical execution model to its internal Domain Layer. This mapping ensures that CircleCI data can be unified with other CI/CD tools in DevLake's analytics.

CircleCI Hierarchy

• Pipeline: A historical execution for a specific commit. Each new commit push triggers a new pipeline.
• Workflow: Orchestrates a set of jobs; represents a historical execution record.
• Job: A component of a workflow that can contain multiple steps.

DevLake Domain Layer Mapping

To maintain consistency across different CI/CD providers, DevLake converts CircleCI entities as follows:

• CircleCI Pipeline $\rightarrow$ cicd_pipeline_commit (mapped this way because CircleCI pipelines are tied to a specific commit SHA).
• CircleCI Workflow $\rightarrow$ cicd_pipeline.
• CircleCI Job $\rightarrow$ cicd_task.

Changes to GITHUB_PR_EXCLUDELIST require a DevLake restart to take effect.

Steps to update:

1. Update the environment variable configuration.
2. Restart DevLake.
3. Trigger re-collection for the affected repositories.

Important: Previously collected bot data remains in the database. Only new collections will respect the updated filter.

To use these dashboards, ensure the following prerequisites are met:

Prerequisites:

1. GitHub Copilot Plugin (gh-copilot) configured and collecting data.
2. GitHub Plugin for PR metrics (project_pr_metrics table).
3. DORA Metrics (deployments and incidents data from CI/CD tools).

Template Variables: Both dashboards utilize these variables for filtering:

• connection_id: DevLake GitHub Copilot connection.
• scope_id: Enterprise/Organization scope.
• project: DevLake project filter.

Optional: Rollout Milestone Annotation: To annotate specific rollout dates for teams:

1. Navigate to Connections > GitHub Copilot > Edit Scope.
2. Set Implementation Date.
3. Set Baseline Period (number of days to use for 'before' comparison).

To generate static and minified production assets in the dist/ directory, run the build command. For actual production deployments, it is recommended to use the official Docker Image for Config-UI as described in the main project documentation.

$ yarn build

The Linear plugin uses a personal API key for authentication. This key must be passed verbatim in the Authorization header without a Bearer prefix.

To generate a key, navigate to Settings → Security & access → Personal API keys within your Linear account.

To manually build and push a new version of the lake-builder image to Docker Hub, set the VERSION environment variable and use the standard Docker build and push commands. This image is a Golang builder image for DevLake used primarily by GitHub workflows.

export VERSION=0.0.11
docker build -t mericodev/lake-builder:$VERSION .
docker push mericodev/lake-builder:$VERSION

Blueprints allow you to set pipeline plans and schedule jobs. When creating or updating a blueprint, you can provide a cronConfig in standard crontab format to define the execution schedule.

The cronConfig format is "M H D M WD":

• M: minute
• H: hour
• D: day (month)
• M: month
• WD: day (week)

You can verify your cron expression at https://crontab.guru/.

To build the gitextractor plugin using a locally compiled version of libgit2 (v1.3.2), you must provide the appropriate CGO flags to the go build command so the compiler can locate the headers and library files.

CGO_LDFLAGS="-L/usr/local/lib -lgit2" CGO_CFLAGS="-I/usr/local/include" go build ./plugins/gitextractor/... 2>&1 | tail -5