AllenNLP

AllenNLP supports dynamic loading of plugins. A plugin is a Python package that provides custom registered classes or additional allennlp subcommands.

Loading Plugins

• Official AI2 Plugins: AllenNLP automatically finds any official AI2-maintained plugins that are installed.
• Third-party/Personal Plugins: To load non-official plugins, you must create a plugins file listing the plugin modules (one per line):
  • Local: Create a file named .allennlp_plugins in the directory where you run the allennlp command.
  • Global: Create a file at ~/.allennlp/plugins.

Verifying Plugins

To verify that your plugins are correctly discovered and can be imported, run the following command:

allennlp test-install

Each discovered plugin will be logged to the terminal.

The simplest way to install AllenNLP is using conda. You can specify a Python version during installation.

To install the base package:

conda install -c conda-forge python=3.8 allennlp

To install optional packages like checklist:

conda install -c conda-forge allennlp-checklist

Alternatively, you can install allennlp-all to get everything. You can also install specific plugins individually:

• allennlp-models
• allennlp-semparse
• allennlp-server
• allennlp-optuna

AllenNLP uses Markdown-formatted docstrings for all reasonably complex public and private methods. The goal is to ensure readability for modeling decisions.

Docstring Structure

1. Description: A brief explanation of what the method does (including how or why).
2. Parameters: A section titled # Parameters describing arguments.
3. Returns: A section describing the return value (if any).

Key Rules

• Constructors: Treat the class docstring as the documentation for the __init__ method. Provide parameters in the class docstring and omit the docstring on the constructor itself.
• Mandatory Sections: For model/module constructors and methods like forward, you must always include the # Parameters and return value sections.
• Tensor Operations: When manipulating tensors, include a comment on most lines describing the tensor's shape.

It is recommended to install the PyTorch ecosystem before installing AllenNLP by following the instructions at pytorch.org.

To install the base package:

pip install allennlp

To install optional dependencies:

• For checklist: pip install allennlp[checklist]
• For all optional dependencies: pip install allennlp[all]

To install the official models package (contains NLP constructs for training/running supported models):

pip install allennlp-models

Note for Python 3.7+ users: Ensure you do not have the PyPI version of dataclasses installed, as it can cause issues. Check with pip freeze | grep dataclasses. If dataclasses=0.6 (or similar) appears, run pip uninstall -y dataclasses.

If you need a specific version of PyTorch or Python, you can build your own image using make from the root of the AllenNLP repository.

• Default tag: allennlp/allennlp
• Custom tag: Set DOCKER_IMAGE_NAME (e.g., make docker-image DOCKER_IMAGE_NAME=my-allennlp)
• Custom Python/PyTorch versions: Set DOCKER_PYTHON_VERSION and DOCKER_TORCH_VERSION.

Example for Python 3.9 and PyTorch 1.9.0 with CUDA 10.2:

make docker-image DOCKER_PYTHON_VERSION=3.9 DOCKER_TORCH_VERSION=1.9.0-cuda10.2

AllenNLP enforces consistency using the following tools:

• flake8: Linting
• black: Code formatting
• mypy: Type checking

Formatting Rules

• Line Length: Uses a 100-character limit (instead of the standard 80) to accommodate type annotations and descriptive variable names.
• Naming: Follows Google's general naming rules and camel case definitions.
• Imports: Follows PEP 8 recommendations. Imports must be organized into three distinct, sorted sections separated by blank lines:
  1. Standard library imports
  2. Third-party library imports
  3. Internal AllenNLP imports

To install AllenNLP in editable mode using the local source code:

1. Clone the repository:

   git clone https://github.com/allenai/allennlp.git

2. Create a Python 3.7 or 3.8 virtual environment.
3. Install dependencies and the package in editable mode:

   pip install -U pip setuptools wheel
   pip install --editable .[dev,all]

This makes allennlp available on your system while using the local source files.

AllenNLP provides official Docker images that include the library and all dependencies. This is useful for GPU/CPU isolation and consistency.

Prerequisites:

• Docker
• NVIDIA Container Toolkit (if using GPUs)

Run with GPU support:

mkdir -p $HOME/.allennlp/
docker run --rm --gpus all -v $HOME/.allennlp:/root/.allennlp allennlp/allennlp:latest

Run without GPU support: Omit the --gpus all flag.

Test the Docker installation:

docker run --rm --gpus all -v $HOME/.allennlp:/root/.allennlp allennlp/allennlp:latest test-install

The --auto-names flag allows the evaluate command to automatically generate filenames for metrics and predictions based on the input filenames.

• NONE: No automatic generation. You must provide both --output-file and --predictions-output-file.
• METRICS: Automatically creates a filename for the metrics file (appends .outputs to the input stem).
• PREDS: Automatically creates a filename for the predictions file (appends .preds to the input stem).
• ALL: Automatically creates filenames for both metrics and predictions.

Note: If you use METRICS or ALL, providing an explicit --output-file will be ignored.

When evaluating on a dataset that contains words or tokens not seen during training, you can extend the model's vocabulary using the --extend-vocab flag.

If you used pretrained embeddings (like GloVe) during training, you should also provide --embedding-sources-mapping to ensure the extension process knows which embedding files to use. This mapping should be a JSON dictionary defining the mapping from the embedding module path to the embedding pretrained-file used during training.

If --embedding-sources-mapping is not provided, the system will attempt to use the original file paths from the training configuration. If those are unavailable, it will use random vectors for the extension.

allennlp evaluate model.tar.gz data.jsonl --extend-vocab --embedding-sources-mapping '{"model.embedders.word_embedders": "path/to/glove.txt"}'