zhparser

To use zhparser for Chinese text search, you must create a text search configuration that uses the zhparser parser and map token types to a dictionary (usually simple).

Basic Workflow

1. Create Extension: CREATE EXTENSION zhparser;
2. Create Configuration: CREATE TEXT SEARCH CONFIGURATION <name> (PARSER = zhparser);
3. Add Mappings: ALTER TEXT SEARCH CONFIGURATION <name> ADD MAPPING FOR n,v,a,i,e,l WITH simple;

Available Functions

• ts_parse('zhparser', 'text'): Returns a table of tokid and token for the given text.
• to_tsvector('<config>', 'text'): Converts text into a tsvector using the specified configuration.
• to_tsquery('<config>', 'query'): Converts a string into a tsquery using the specified configuration.

-- Setup
CREATE EXTENSION zhparser;
CREATE TEXT SEARCH CONFIGURATION testzhcfg (PARSER = zhparser);
ALTER TEXT SEARCH CONFIGURATION testzhcfg ADD MAPPING FOR n,v,a,i,e,l WITH simple;

-- Parsing
SELECT * FROM ts_parse('zhparser', 'hello world! 2010年保障房建设');

-- Search
SELECT to_tsvector('testzhcfg', '今年保障房新开工数量虽然有所下调');
SELECT to_tsquery('testzhcfg', '保障房资金压力');

Use the regress/regress.sh wrapper to run regression tests against different PostgreSQL versions. This provides a fast signal for whether C code changes have broken existing functionality.

Available Commands

• check: Runs tests against the default PostgreSQL version (PG 16).
• check <version>: Runs tests against a specific PostgreSQL version (e.g., 17).
• matrix: Runs tests against multiple PostgreSQL versions (16, 17, and 18) sequentially.
• shell <version>: Drops you into a shell inside a container running a test cluster for that specific PostgreSQL version. This allows you to run manual queries via psql against the running instance.

# From the project root.
regress/regress.sh check          # PG 16 by default
regress/regress.sh check 17       # PG 17
regress/regress.sh matrix         # 16 + 17 + 18

# Drop into a shell with the test cluster running:
regress/regress.sh shell 16

# (inside the container)
psql -h /tmp -p 55432 -U postgres

To install zhparser on a system with PostgreSQL 9.2 or higher, follow these steps:

1. Install SCWS

zhparser is based on SCWS (Simple Chinese Word Segmentation). Download and install it first:

wget -q -O - http://www.xunsearch.com/scws/down/scws-1.2.3.tar.bz2 | tar xf -
cd scws-1.2.3
./configure
make install

Note: On FreeBSD 10+, use --with-pic during ./configure.

2. Build and Install zhparser

Clone the repository and compile it. You can specify SCWS_HOME if SCWS is not in /usr/local and PG_CONFIG if you have multiple PostgreSQL versions installed.

git clone https://github.com/amutu/zhparser.git
cd zhparser
# Optional: set SCWS_HOME or PG_CONFIG
SCWS_HOME=/usr/local PG_CONFIG=/usr/lib/postgresql/9.5/bin/pg_config make && make install

3. Create the Extension

Run the following command in your PostgreSQL instance:

CREATE EXTENSION zhparser;

wget -q -O - http://www.xunsearch.com/scws/down/scws-1.2.3.tar.bz2 | tar xf -
cd scws-1.2.3 ; ./configure ; make install

git clone https://github.com/amutu/zhparser.git
cd zhparser
make && make install

# Then in psql
psql dbname superuser -c 'CREATE EXTENSION zhparser'

zhparser provides several configuration options to control dictionary loading and segmentation behavior. Most options default to false.

Note: zhparser.extra_dicts and zhparser.dict_in_memory must be set before the backend starts (e.g., in the configuration file and then reloaded). Other options can be set in the current session.

For database-level custom word management (requires superuser permissions), zhparser provides a table-based approach. This is more flexible for dynamic updates than file-based dictionaries.

Workflow

1. Insert a word: INSERT INTO zhparser.zhprs_custom_word VALUES('word_text');
2. Delete a word: INSERT INTO zhprs_custom_word(word, attr) VALUES('word_text', '!');
3. Sync changes: After modifying the table, you must run SELECT sync_zhprs_custom_word(); and then reconnect to the database for changes to take effect in the parser.

To inspect the table structure (which supports TD and IDF fields), use \d zhprs_custom_word in psql.

-- Add a custom word
INSERT INTO zhparser.zhprs_custom_word VALUES('资金压力');

-- Sync the changes
SELECT sync_zhprs_custom_word();

-- Note: You must reconnect to the database after syncing

-- Delete a word
INSERT INTO zhprs_custom_word(word, attr) VALUES('资金压力', '!');