picklescan

To use picklescan, install it via pip. You can scan models directly from Hugging Face by providing the repository name using the --huggingface flag.

To scan Numpy's .npy files, ensure the numpy package is installed in your environment before running picklescan.

pip install picklescan
picklescan --huggingface ykilcher/totally-harmless-model

To develop on picklescan, use conda to create the environment from the provided conda.yaml and install the package in editable mode.

Development Workflow:

1. Create and activate the environment: conda env create -f conda.yaml && conda activate picklescan
2. Install in editable mode: python3 -m pip install -e .
3. Run unit tests: pytest tests
4. Lint code: Use black and flake8.
5. Manual testing: Use picklescan -l DEBUG with local or remote files to verify behavior.

# Environment setup
conda env create -f conda.yaml
conda activate picklescan

# Editable install
python3 -m pip install -e .

# Run tests
pytest tests

# Linting
black src tests --line-length 140
flake8 src tests --count --show-source

The scanner supports multiple input types:

• Local files: Use --path with the file path.
• Directories: Use --path with the directory path to scan all eligible files within.
• URLs: Use --url to scan a remote file.
• Archives: Supports zip archives (similar to PyTorch loading behavior).

Note: When scanning directories, you can use filtering options to include or exclude specific files or subdirectories.

# Scan a local file
picklescan --path downloads/pytorch_model.bin

# Scan a directory
picklescan --path downloads

# Scan a remote URL
picklescan --url https://huggingface.co/sshleifer/tiny-distilbert-base-cased-distilled-squad/resolve/main/pytorch_model.bin

The ScanResult object contains the aggregated findings of a scan. Each identified global import is assigned a SafetyLevel.

SafetyLevel

• Innocuous: The import is known to be safe (e.g., torch.LongStorage).
• Suspicious: The import is not in the safe list and not explicitly in the dangerous list.
• Dangerous: The import is explicitly flagged as dangerous (e.g., os.system, eval, subprocess).

ScanResult Fields

• globals: A list of Global objects containing module, name, and safety level.
• scanned_files: Total number of files processed.
• issues_count: Number of dangerous imports found.
• infected_files: Number of files containing at least one dangerous import.
• scan_err: Boolean indicating if a parsing error occurred during the scan.

When calling any scanning function (e.g., scan_file_path, scan_bytes, scan_directory_path), you can set the strict parameter to True.

In strict mode, any global import that is not explicitly found in the _safe_globals list is automatically promoted to SafetyLevel.Dangerous. This is useful for high-security environments where you want to flag anything that isn't explicitly whitelisted.

The scanner uses exit status codes to indicate the result of the scan:

• 0: scan did not find malware
• 1: scan found malware
• 2: scan failed

By default, picklescan provides a summary of scanned and infected files. To see a detailed list of every global found during the scan, use the -g or --globals flag. This will output the module path, the name of the global, and its safety status.

picklescan --path ./my_model --globals

The picklescan CLI is a security scanner designed to detect Python Pickle files performing suspicious actions. You must provide exactly one target using one of the following mutually exclusive flags:

• -p, --path: A local path to a file or a directory.
• -u, --url: A URL pointing to a file or a directory.
• -hf, --huggingface: The name of a Hugging Face model to scan.

Exit Codes

• 0: No issues found.
• 1: Issues (dangerous globals) were detected.
• 2: An error occurred during the scan (e.g., path not found, unhandled exception, or scan error).

# Scan a local directory
picklescan --path /path/to/directory

# Scan a specific URL
picklescan --url https://example.com/model.pkl

# Scan a Hugging Face model
picklescan --huggingface username/model-name