Megaparsec Documentation

Megaparsec has a ecosystem of specialized packages designed to extend its functionality:

• Testing: hspec-megaparsec provides utilities for testing parsers with Hspec.
• Stream Editing: replace-megaparsec enables stream editing and find-and-replace operations.
• CSV Parsing: cassava-megaparsec allows parsing CSV files while maintaining compatibility with the Cassava library.
• HTML/TagSoup: tagsoup-megaparsec lets you use TagSoup as a token type within Megaparsec.
• Combinators: parser-combinators provides permutation and expression parsers (previously bundled with Megaparsec).
• Performance: faster-megaparsec optimizes parsing speed by attempting a simple MonadParsec instance before falling back to ParsecT for error reporting.

Megaparsec is built around MonadParsec, an MTL-style type class. Most features work with any instance of MonadParsec.

To achieve various effects, you can build a monadic stack using monad transformers. Because common transformers like WriterT, StateT, and ReaderT are instances of MonadParsec, you can wrap ParsecT inside these monads to achieve features like backtracking state.

ParsecT itself implements several useful type classes:

• Monad
• Applicative
• Alternative
• MonadParsec

Megaparsec provides sophisticated error reporting features:

• Typed Errors: Supports typed error messages and custom parse errors tailored to specific domains.
• Independent Error Locations: Since version 8, the location of a parse error can be independent of the current offset, allowing you to point to specific positions after performing checks.
• ParseErrorBundle: Instead of a single error, Megaparsec produces a ParseErrorBundle which manages and pretty-prints multiple parse errors simultaneously.

Megaparsec uses nix for development. To enter the development shell, run nix develop.

Once inside the shell, you can use the following commands:

• Build packages: Use cabal build all to build both megaparsec and megaparsec-tests.
• Run tests: Use cabal test all to run tests from the megaparsec-tests package.
• Interactive feedback: Use ghcid for real-time feedback while editing.
  • For megaparsec: ghcid --command="cabal repl megaparsec"
  • For megaparsec-tests: ghcid --command="cabal repl megaparsec-tests --enable-tests"

$ nix develop
$ cabal build all
$ cabal test all
$ ghcid --command="cabal repl megaparsec"

Benchmarks are located in the bench sub-directory. Note: You must cd into the bench directory before running them, as they rely on relative paths for data files.

• Build all benchmarks: nix build .#all_benches (creates symlinks in result).
• Build specific package benchmarks: To build only megaparsec microbenchmarks, run nix build .#benches/megaparsec.

$ nix build .#all_benches
$ nix build .#benches/megaparsec

Follow these steps to release a new version:

1. Bump versions: Update the version in megaparsec.cabal. Also update megaparsec-tests.cabal to match, including its dependency on megaparsec.
2. Tagging: Create a git tag and push it to the repository.
3. Generate tarballs: Run nix build .#all_dist. This creates a result directory containing megaparsec-source-* and megaparsec-tests-source-* tarballs.
4. Upload to Hackage: Use cabal upload --publish for both the source and test tarballs:

$ cabal upload --publish result/megaparsec-source-*/megaparsec-*.tar.gz
$ cabal upload --publish result/megaparsec-tests-source-*/megaparsec-tests-*.tar.gz

$ nix build .#all_dist
$ cabal upload --publish result/megaparsec-source-*/megaparsec-*.tar.gz
$ cabal upload --publish result/megaparsec-tests-source-*/megaparsec-tests-*.tar.gz

To gain higher confidence in non-trivial changes, you should run tests beyond the standard megaparsec-tests suite.

Build the base test group

Use the base group to build and test megaparsec, hspec-megaparsec, megaparsec-tests, and parser-combinators-tests:

$ nix build .#all_base --no-link

To build a specific derivation from the base group (e.g., parser-combinators-tests):

$ nix build .#base/parser-combinators-tests --no-link

Check impact on dependent packages

To see how changes affect a selected set of high-quality dependent packages, run:

$ nix build .#all_deps --no-link

To test a specific package (e.g., mmark):

$ nix build .#deps/mmark --no-link

You can run the built-in benchmarks using nix-build.

$ nix-build -A benches.parsers-bench
$ cd result/bench
$ ./bench-memory
$ ./bench-speed

If a breaking change in Megaparsec causes a dependent package to fail compilation, you can create a patch to maintain compatibility:

1. Clone the failing package's repository.
2. Checkout the commit corresponding to the version used in the current nixpkgs.
3. Attempt to compile the package using the current development version of Megaparsec (e.g., by adding the Megaparsec path to extra-deps in Stack).
4. Apply necessary changes to fix the build.
5. Generate a patch file:
   • For unstaged changes: git diff > my-package.patch
   • For staged changes: git diff --cached > my-package.patch
6. Apply the patch in default.nix by editing the deps attribute set:

# Example of applying a patch in default.nix
deps = {
  # ...
  idris = patch haskellPackages.idris ./nix/patches/idris.patch;
};

Megaparsec provides specialized high-performance combinators that are significantly faster than standard approaches:

• tokens: Parses several tokens in a row. It is approximately 100x faster than matching a string token by token. It returns a "chunk" of the original input (e.g., if parsing Text, it returns Text without repacking).
• takeWhileP and takeWhile1P: Approximately 150x faster than using many, manyTill, or similar combinators.
• takeP: Grabs n tokens from the stream and returns them as a "chunk" of the stream.