spider_xhs

Spider_XHS is an all-in-one manager for Xiaohongshu (XHS) designed to provide stable API support for AI agents. It handles the complex reverse-engineered signature algorithms (such as a1, web_id, x-s, x-t, etc.) required to interact with XHS. The project covers three main scenarios:

1. Data Collection (PC side): Scraping notes, user info, search results, and comments.
2. Content Publishing (Creator platform): Uploading images and videos to the creator platform.
3. KOL Data (Pugongying platform): Accessing influencer data and managing collaborations.

Warning: This project is for learning and exchange purposes only. Commercial use is strictly prohibited.

You can build and run the project using Docker. If using cookie-based authentication, pass the COOKIES environment variable during runtime.

docker build -t spider_xhs .
docker run -e COOKIES='your_cookie_here' spider_xhs

To set up the environment for Spider_XHS, ensure you have Python 3.10+ and Node.js 20+ installed. Then, install the Python and JavaScript dependencies using the following commands:

pip install -r requirements.txt
npm install

The project operates without a browser. You can configure the login method by setting the login_type variable in spider/spider.py. Supported types are:

• qrcode: Local request for a QR code; scan with the Xiaohongshu App.
• phone: Direct call to the phone number verification code interface.
• cookie: Use a full Cookie string copied from a logged-in session.

If using cookie mode, copy .env.example to .env and set the COOKIES variable:

COOKIES='your_cookie_here'

To run the crawler, modify the entry point in spider/spider.py (e.g., note links, user links, search keywords, and filters) and execute the module using Python:

python -m spider.spider

Use XHSCreatorAuth to authenticate for the Creator platform (used for uploading and posting). You can choose between QR code, phone, or cookie-based authentication. Once authenticated, initialize XHS_Creator_Apis to access creator-specific endpoints.

from apis.xhs_creator_apis import XHS_Creator_Apis
from xhs_utils.xhs_creator import XHSCreatorAuth

# Choose one of the three:
creator_auth = XHSCreatorAuth.from_qrcode_login()
# creator_auth = XHSCreatorAuth.from_phone_login()
# creator_auth = XHSCreatorAuth.from_cookie(完整_creator_cookie)

creator_api = XHS_Creator_Apis(creator_auth).bootstrap()
success, message, notes = creator_api.get_all_posted_notes()

Use XHSPcAuth to manage login states for the PC interface (used for data collection). You can authenticate via QR code or by reusing existing cookies. Once authenticated, initialize XHS_Apis to access data endpoints.

from apis.xhs_pc_apis import XHS_Apis
from xhs_utils.xhs_pc import XHSPcAuth

# No-browser QR code login
auth = XHSPcAuth.from_qrcode_login()

# Or reuse saved login Cookie
# auth = XHSPcAuth.from_cookie(cookies_str)

api = XHS_Apis(auth).bootstrap()
success, message, data = api.get_unread_message()

The repository is organized as follows:

• spider/: Main entry point and crawler logic.
• apis/: Complete API implementations for PC, Creator, Pugongying (KOL), and Qianfan (Distributor) platforms.
• xhs_utils/xhs_pc/: Authentication, state management, and signature algorithms for the PC interface.
• xhs_utils/xhs_creator/: Authentication, state management, and request assembly for the Creator platform.
• xhs_utils/xhs_core/: Shared core algorithms (b1, MNS, X-s, etc.) used by both PC and Creator modules.
• .env.example: Template for local configuration.

Use the Creator platform functions to log in via QR code or phone verification to retrieve a list of all your posted works.

Available methods:

• creator_qrcode_login(): Performs a QR code login (displays QR in terminal) and fetches all posted notes.
• creator_phone_login(): Performs a phone verification login and fetches all posted notes.