Nginx Ultimate Bad Bot Blocker

The Nginx Ultimate Bad Bot Blocker is a comprehensive security solution for Nginx web servers. It is designed to block various forms of malicious internet traffic, including:

• Bad Bots and User-Agents
• Spam Referrers and Bad Referrers
• Vulnerability Scanners, Malware, and Adware
• Fake Googlebots
• Clickjacking and Click Re-Directing attacks
• SEO Company bots
• Bad IPs
• DDoS attacks (via Nginx Rate Limiting)
• Wordpress Theme Detectors

It is specifically tested on Nginx mainstream versions (e.g., 1.10.x).

When a rule matches a bad bot, spam referrer, or malicious IP, the blocker issues an Nginx 444 response. This is a non-standard Nginx status code that instructs the server to immediately close the connection without sending any response headers or body to the client.

To the requester, it appears as if the server does not exist or is offline. You can verify this behavior using curl; a successful block will result in an Empty reply from server error.

curl: (52) Empty reply from server

You can use a bash script to automate the update process for the blocker. The latest universal auto-updater script is available via the project's raw GitHub content URL.

https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/update-ngxblocker

The provided robots.txt is optimized for WordPress. By default, it allows all user agents to access the site but restricts access to the /wp-admin/ directory while explicitly allowing access to /wp-admin/admin-ajax.php. You can modify these initial lines to suit your requirements before the specific bot blocklist begins.

User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php

User-agent: 360Spider
Disallow:/ 
User-agent: 404checker
Disallow:/ 

To avoid service disruptions due to false positives, follow these deployment best practices:

1. Staged Rollout: Load the blocker into a single website first and monitor its logs closely before deploying it to all production websites.
2. Log Monitoring: Regularly monitor your web server logs for any legitimate traffic being blocked (false positives).
3. Continuous Updates: Use your own server logs as a primary source for identifying new bad referrers or user-agent strings to block. You can contribute these findings back to the project via Pull Requests.

To ensure you always have the latest protection, schedule the update-ngxblocker script using crontab.

With Email Notifications: To receive an update report via email, use the -e flag. This example runs the update every night at 10 PM.

00 22 * * * sudo /usr/local/sbin/update-ngxblocker -e yourname@youremail.com

Without Email Notifications: To run the update without sending an email, use the -n flag.

00 */8 * * * sudo /usr/local/sbin/update-ngxblocker -n

Run the install-ngxblocker script to download all necessary configuration and script files.

1. Dry Run: Run sudo ./install-ngxblocker without flags to see what files will be downloaded and where they will be placed.
2. Actual Installation: Run sudo ./install-ngxblocker -x to perform the download.
3. Post-Install: Ensure the downloaded setup and update scripts are executable:

   sudo chmod +x /usr/local/sbin/setup-ngxblocker
   sudo chmod +x /usr/local/sbin/update-ngxblocker

# Dry run
cd /usr/local/sbin
sudo ./install-ngxblocker

# Actual installation
sudo ./install-ngxblocker -x

# Make scripts executable
sudo chmod +x /usr/local/sbin/setup-ngxblocker
sudo chmod +x /usr/local/sbin/update-ngxblocker

To install the blocker on Linux, download the install-ngxblocker script to /usr/local/sbin/ and make it executable. You can use wget or curl depending on your distribution's availability.

# Using wget
sudo wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/install-ngxblocker -O /usr/local/sbin/install-ngxblocker
sudo chmod +x /usr/local/sbin/install-ngxblocker

# Using curl
curl -sL https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/install-ngxblocker -o /usr/local/sbin/install-ngxblocker