houbb/sensitive

The integration design for chars-scan provides several benefits for developers implementing sensitive data masking:

• Unified Configuration Management: Both log4j2 and logback share the same configuration set, preventing duplication.
• High Extensibility: Supports custom scanning rules (identification) and custom replacement rules (masking logic), including the ability to override built-in strategies.
• Zero Intrusion: Integrates with the existing chars-scan library without requiring modifications to its source code.
• Performance Optimization: Uses a Singleton pattern to manage CharsScanBs and supports multiple core implementations such as defaults, concurrency, and threadLocal.
• Ease of Use: Driven by configuration files and uses reflection to load custom strategies, requiring no code changes to update logic.

The EnhancedSensitiveScanner uses the chars.scan.custom.priority setting to decide the order of operations:

1. higher (Default): Custom strategies are applied to the original text first. After custom masking is complete, the native chars-scan logic is applied to the result, skipping segments already processed by custom rules.
2. native: The native chars-scan logic is applied first. Custom strategies are then applied only to parts of the text that were not modified by the native scanner (i.e., segments that do not contain the masking character *).

The project provides annotations to mark sensitive data for masking during deep copying or JSON serialization.

• @Sensitive: The core annotation used to mark a field as sensitive.
• @SensitiveEntry: Used to mark an entry point for sensitive data processing, allowing for custom configuration of how the entry is handled.

These annotations allow you to define which fields should be masked when an object is processed by the library's utility methods.

The CharsScanBs framework is organized into several specialized components that manage the lifecycle of sensitive data scanning and masking. The architecture consists of:

• CharsScanBs: The main entry point/guiding class.
• charsScanFactory: Manages scanning strategy rules (what to identify).
• charsReplaceFactory: Manages replacement/masking logic (how to mask).
• charsReplaceHash: Controls whether a hash value is displayed.
• prefixCharSet: Defines the set of prefix characters that trigger scanning.
• whiteListTrie: A prefix tree used to manage whitelist rules to skip specific patterns.
• charsCore: Handles core implementation details, such as concurrency safety.

When implementing and using custom sensitive data strategies, consider the following:

• Performance: Custom strategies use regular expressions; adding many complex patterns will increase the overhead of regex matching during scanning.
• Conflict Management: Ensure your custom regexPattern does not conflict with existing built-in strategies to avoid unexpected masking behavior.
• Testing: Always perform thorough unit testing on your desensitize logic and regex accuracy before deployment.
• Strategy IDs: Use unique IDs to avoid collisions. For reference, common IDs include 1 for phone numbers, 2 for ID cards, 3 for bank cards, 4 for emails, and 5 for names.

To use the sensitive data masking features, add the sensitive-core dependency to your pom.xml file.

<dependency>
    <groupId>com.github.houbb</groupId>
    <artifactId>sensitive-core</artifactId>
    <version>1.1.0</version>
</dependency>

To enable automatic log masking using Log4j2, add the sensitive-log4j2 dependency to your project. Ensure you also have the standard log4j-api and log4j-core dependencies present.

<dependency>
    <groupId>com.github.houbb</groupId>
    <artifactId>sensitive-log4j2</artifactId>
    <version>1.2.1</version>
</dependency>

<!-- Ensure log4j2 core/api are also present -->
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-api</artifactId>
    <version>${log4j2.version}</version>
</dependency>
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-core</artifactId>
    <version>${log4j2.version}</version>
</dependency>

To automatically desensitize log messages, override the toSerializable(LogEvent event) method in your custom PatternLayout. Use EnhancedSensitiveScanner.scanAndReplace(text) on the formatted log string to ensure all sensitive data is masked before it is written to the logs.

@Override
public String toSerializable(LogEvent event) {
    StringBuilder stringBuilder = new StringBuilder();
    for(PatternFormatter formatter : patternFormatterList) {
        formatter.format(event, stringBuilder);
    }
    String text = stringBuilder.toString();
    
    try {
        // 使用增强的扫描处理器
        return EnhancedSensitiveScanner.scanAndReplace(text);
    } catch (Exception e) {
        return text;
    }
}