Community Notes Documentation

The Template API Note Writer is an open-source client designed to call the AI Note Writer API and write rudimentary notes. It serves as a "hello, world" level starting point for developers looking to build or improve upon note-writing automation. It utilizes GitHub Actions and Grok to facilitate the process.

This project is intended to be forked and used as an optional foundation to quickly get up and running with the Note Writer API.

To prevent notes intended for a specific post from appearing on irrelevant posts with the same media, media notes require an additional layer of agreement. When rating a media note, contributors are asked if the note would be helpful if shown on all posts with matching media.

Sufficient agreement is currently defined by two criteria using the rater factor from the model that determined the note's status (e.g., CoreModel, GroupModel01):

1. Minimum Raters: 5+ raters with positive rater factors AND 5+ raters with negative rater factors must agree the note will be helpful on all matching posts.
2. Agreement Percentage: 80%+ of raters from each factor must agree the note will be helpful on all matching posts.

The Community Notes scoring algorithm is split into two distinct binaries: prescoring and final scoring.

In the production environment at X, these binaries are run separately and independently, each reading the most recent available input data.

Note for exact reproduction: To exactly match production results, the prescorer should be run on input data that is approximately 1 hour older than the data used by the final scorer. While this difference has minimal impact in practice, it is a requirement for perfect parity with the production pipeline.

Note Diligence scoring identifies notes that people from different viewpoints agree may have issues with accuracy or sourcing.

It specifically monitors the following rating tags:

• Sources not included or unreliable
• Sources do not support note
• Incorrect information

The algorithm uses a rater weight component determined via gradient descent. This component identifies and upweights raters who have a demonstrated pattern of identifying notes that are agreed upon by diverse viewpoints as having diligence issues. Notes scoring above a certain threshold are blocked from being shown on X.

Safety measures to prevent harassment of contributors include:

• Anonymity via Aliases: Contributors use auto-generated display names (aliases) that are not linked to their X accounts, allowing for private writing and rating.
• Direct Feedback Channel: Contributors can report issues or concerns directly to the Community Notes team via DM at @CommunityNotes.
• Content Reporting: Notes that violate X's Rules, Terms of Service, or Privacy Policy can be reported using the ••• menu on a note (selecting "Report") or via the Report a Community Note form.

To prevent notes from rapidly losing and re-acquiring "Helpful" status due to small fluctuations in the note intercept $i_n$, the algorithm employs CRH Inertia.

A note must drop below its threshold by more than 0.01 to lose its status.

Examples:

• If a note became helpful with $i_n > 0.40$, it must drop to $i_n < 0.39$ to lose status.
• If a note was impacted by the tag outlier filter and required $i_n > 0.50$, it must drop to $i_n < 0.49$ to lose status.

To mitigate the impact of spam or low-quality notes/ratings, the system uses:

• Onboarding System: New contributors must first rate existing notes to understand what constitutes a "Helpful" vs "Not Helpful" note before they are granted the ability to write new notes.
• Writing Ability Locks: If a contributor writes too many notes that are rated as "Not Helpful" by others, their ability to write notes may be temporarily locked. They receive warnings and must repeat the rating process to regain writing privileges.
• Rating Reputation: New accounts have lower influence in the rating system until they demonstrate a track record of helpful contributions, making it harder for spam accounts to impact the system.

Because Matrix Factorization does not inherently estimate parameter uncertainty, the system uses Pseudo-rating Sensitivity Analysis to quantify it.

This involves adding "extreme" ratings from "pseudo-raters" (using max/min possible rater intercepts and factors, including 0-factor raters) to measure the maximum and minimum possible values for a note's intercept and factor parameters.

Note Status Rule: A note is assigned a Not Helpful status if the upper confidence bound (the maximum intercept value calculated after adding pseudo-ratings) is less than -0.04, in addition to standard raw intercept rules.

Community Notes aims to provide context that is helpful to people from diverse perspectives. The system uses two primary methods to evaluate this without letting political bias influence the note ranking algorithm:

1. Representative Surveys: Using participants who optionally declare political leanings to understand how notes affect different groups.
2. Viewpoint Estimation: Using anonymized, aggregate data from follow, like, and repost graphs to estimate political leanings based on network proximity to political accounts. This is used for comparative analysis of note quality and does not influence individual note decisions.