repository·master·Indexed 19 days ago
https://github.com/ufal/udpipeUDPipe is a language-agnostic, trainable NLP pipeline for tokenization, tagging, lemmatization, and dependency parsing, optimized for CoNLL-U formatted data. It provides a C++ API with native bindings for Python, Java, C#, and Perl, as well as a REST API web service and an R package. The toolkit includes core classes such as Pipeline for orchestrating NLP sequences, Model for managing linguistic models, Trainer for creating new models, and Evaluator for model assessment.
UDPipe is a language-agnostic, trainable pipeline designed for processing CoNLL-U files. It performs the following linguistic tasks:
UDPipe can be trained using annotated data in the CoNLL-U format. Pre-trained models are available for nearly all UD treebanks.
UDPipe is available in several formats depending on your integration needs:
udpipe) is available for R users.Parsito is a fast, open-source dependency parser written in C++. It uses greedy transition-based parsing and an artificial neural network classifier, which allows it to be trained on any input data without manual feature engineering.
Key characteristics:
libmicrohttpd and provides utilities for producing JSON and XML, as well as XML parsing and manipulation via the pugixml library. It is released under the MPL 2.0 license.UniLib is an embeddable C++11 Unicode library designed for integration into other C++ projects. It provides essential Unicode processing capabilities including encoding support, character categorization, and text transformations.
Key Features:
Supported Platforms & Compilers:
gcc and clangVisual C++ 2019clangMorphoDiTa (Morphological Dictionary and Tagger) is an open-source tool designed for the morphological analysis of natural language texts. It provides the following core functionalities:
MorphoDiTa can be used as a standalone command-line tool or integrated into applications as a library. It is distributed alongside trained linguistic models. For the Czech language, it is highly performant, achieving throughputs between 10,000 and 200,000 words per second.
The C# bindings are located in the Ufal.UDPipe namespace. They serve as a direct conversion of the C++ API.
Requirement: The bindings require the native C++ library libudpipe_csharp (on Windows, this is named udpipe_csharp).
ufal.udpipe module. The API is a direct conversion of the C++ bindings API, with the primary difference being that Python's native bytes type is used instead of the C++ Bytes type.T2T DocSys supports standard txt2tags markup with the following specific behaviors and additions:
[ link #anchor ] format are supported within verbatim code, but they are only preserved in HTML and PDF outputs.MicroRestD supports the following environments:
Platforms:
Compilers:
gccclangtdm-gcc (Windows)Visual C++ 2013 or later (Windows)You can obtain C# bindings in two ways:
bindings/csharp directory and run make. You can optionally provide build options as described in the main UDPipe installation guide.# To compile manually
cd bindings/csharp
makeThe Perl bindings are provided by the Ufal::UDPipe package. You can import all classes into your current namespace using the :all export tag.
Note that the bindings are a direct conversion of the C++ API. Because Perl does not have a native interface for C++ vectors, you should refer to Ufal::UDPipe::Words for handling word sequences. Additionally, static methods and enumerations must be accessed through the module itself rather than through an object instance.
use Ufal::UDPipe :all;