Emerge (emerge-viz)

Emerge (also known as emerge-viz) is an interactive code analysis tool designed to gather insights into software project structure, metrics, dependencies, and complexity. It scans source code, calculates metrics (like SLOC, Fan-In/Fan-Out, and Louvain modularity), and generates interactive web applications featuring force-directed graphs (using D3.js) to visualize filesystem hierarchies, dependency graphs, and inheritance structures.

Supported Languages:

• File Scan: C, C++, Groovy, Java, JavaScript, TypeScript, Kotlin, ObjC, Ruby, Swift, Python, Go.
• Entity Scan (Classes, etc.): Groovy, Java, Kotlin, Swift.

Key Features:

• Metrics: SLOC, Whitespace Complexity, Number of Methods, Fan-In/Fan-Out, Louvain Modularity, and experimental git-based metrics (Change Coupling, code churn).
• Visualizations: Interactive HTML/web apps with dark mode, semantic search (TF-IDF), heatmap support, and cluster visualization.
• Exports: GraphML, D3-compatible JSON, tabular console/file output, and interactive HTML.

Emerge generates force-directed graphs to visualize software architecture. You can interpret these graphs using specific visual indicators to assess code quality:

• Modularity: Look for coherent, colored clusters with low coupling (minimal distance between clusters). Activating cluster hulls helps identify well-defined interfaces; minimal hull overlap indicates good abstraction and modularity.
• Code Smells (God Classes/High Coupling): High overlap between clusters or large entities (visible via SLOC metrics) can indicate 'god classes' or increased maintenance effort.
• Big Ball of Mud: If enabling hull rendering for all clusters results in a single large cluster, the architecture is likely a 'spaghetti code jungle' lacking clear structure.
• Irrelevant Dependencies: Large-scale graphs may be cluttered by standard libraries (e.g., java.lang, java.util) or third-party dependencies. Removing these can reveal the true underlying architecture.

To install emerge for development, clone the repository and set up a virtual environment. Follow the specific steps for your operating system.

macOS Setup

1. Install graphviz via Homebrew:

   brew install graphviz

   Note for Apple Silicon: If you encounter graphviz/cgraph.h not found errors, run:

   pip install --global-option=build_ext --global-option="-I$(brew --prefix graphviz)/include/" --global-option="-L$(brew --prefix graphviz)/lib/" pygraphviz

2. Create a virtual environment:

   cd emerge
   pip3 install virtualenv
   virtualenv -p python3 venv

3. Activate the environment:

   source venv/bin/activate

4. Install dependencies:

   pip install -r requirements.txt

Ubuntu Setup

1. Install prerequisites:

   apt-get install python3-venv python3-dev graphviz graphviz-dev

2. Create and activate a virtual environment:

   cd emerge
   python3 -m venv venv
   source venv/bin/activate

3. Install dependencies:

   pip install wheel
   pip install -r requirements.txt

git clone https://github.com/glato/emerge.git

Follow these steps to perform a full analysis of your project:

1. Generate a configuration template: Use the -a flag with a language keyword to create a template file (e.g., java-template.yaml).

   emerge -a java

2. **Configure the template:**
   Open the generated `.yaml` file and adjust the `analyses.source_directory` and `export.directory` to point to your actual project paths.

3. **Start the scan:**
   Run the analysis using the `-c` flag pointing to your config file.
   ```bash
emerge -c java-template.yaml

4. View results: Once finished, the console will provide a file:// path to an emerge.html file. Copy this path into a modern web browser to interact with the visualization.

# 1. Create template
emerge -a java

# 2. Run analysis
emerge -c java-template.yaml

# 3. Open the output (example path)
# file:///Users/user1/tmp/java/html/emerge.html

The config.yml file defines the scope and behavior of an Emerge analysis. The top-level analysis key controls how files are discovered and filtered.

Key configuration options include:

• source_directory: The starting point for the recursive file scan.
• git_directory: The path to the git repo (required if using git_metrics).
• git_commit_limit: Number of recent commits to mine (default: 150).
• git_exclude_merge_commits: Whether to exclude merge commits (default: true).
• only_permit_languages: Restrict scanning to specific languages (e.g., java, kotlin, objc, swift, ruby, groovy, javascript, c).
• only_permit_file_extensions: Explicitly allow specific extensions (e.g., .java).
• ignore_files_containing / ignore_directories_containing: Substring filters to exclude files or directories.
• import_aliases: Map aliases to paths (e.g., "@foo": src/foo).
• ignore_dependencies_containing / ignore_dependencies_matching: Filter dependencies by substring or regex.
• ignore_entities_containing / ignore_entities_matching: Filter entities (like classes) by substring or regex.

Emerge uses YAML files to define project-level settings and specific analyses.

Project Level Configuration

Analysis Configuration

Each item in the analyses array defines how a specific part of the project is scanned. Key fields include:

• source_directory: The path to the code being analyzed.
• only_permit_languages: List of languages to include (e.g., [c]).
• only_permit_file_extensions: List of extensions to include (e.g., [.c, .h]).
• ignore_dependencies_containing: Exclude dependencies matching a string.
• ignore_entities_containing: Exclude entities (like SwiftUI previews) matching a string.
• file_scan: List of metrics to collect for files (e.g., number_of_methods, source_lines_of_code, dependency_graph).
• entity_scan: List of metrics to collect for entities (e.g., dependency_graph, tfidf).
• export: Configuration for output, including directory and formats like json, d3, graphml, or tabular_file.

Example: Full Configuration

project_name: java_project_example
loglevel: info
analyses:
- analysis_name: check_java_files_and_classes
  source_directory: /Users/user1/emerge/project/source
  only_permit_languages:
  - java
  only_permit_file_extensions:
  - .java
  ignore_dependencies_containing:
  - java.util
  file_scan:
  - number_of_methods
  - source_lines_of_code
  - dependency_graph
  - fan_in_out
  - louvain_modularity
  - tfidf
  entity_scan:
  - dependency_graph
  - source_lines_of_code
  - number_of_methods
  - fan_in_out
  - louvain_modularity
  - tfidf
  export:
  - directory: /Users/user1/emerge/project/export
  - graphml
  - json
  - tabular_file
  - tabular_console_overall
  - d3

python emerge.py -c configs/c-template.yaml