Block Bad Bots, Spams, Vulnerability Scanners, on Nginx with NGINX BAD BOT BLOCKER
Nginx is a free and open-source web server, released in 2004 by a Russian developer. It is widely used to run and manage websites. Later, it was made available as an open-source project. Nginx is user-friendly, with simple and easy-to-configure methods that allow developers, DevOps, and webmasters to manage multiple websites on the same server.
Time to Block Bad Seeds/ Bots
However, poor or improper configuration can expose your websites and Nginx-installed web applications to serious threats such as bad bots, spam, vulnerability scanners, malware, and more. To address this, Nginx offers a variety of tools and features to protect against these threats, including:
- Bad bots and spam referrer blockers: Protects your site from malicious bots and spammy traffic sources.
- Vulnerability scanner blockers: Prevents automated vulnerability scanning tools from exploiting weaknesses.
- User-agent blocking: Blocks malicious or suspicious user agents from accessing your site.
- Malware, adware, and ransomware protection: Guards against harmful software trying to infiltrate your system.
- Anti-DDOS protection: Defends your server from Distributed Denial of Service (DDoS) attacks.
- WordPress theme detector blocking: Prevents automated tools from detecting themes and exploiting WordPress vulnerabilities.
- Fail2Ban jail for repeat offenders: Automatically bans IPs that engage in repeated malicious activity.
Type of Bad Bots that Targets Web Server
- Bad Referrers
- Bad User-Agent Strings
- Spam Referrers
- Spam Bots and Bad Bots
- Nuisance or Unwanted Bots
- Sites Linked to Lucrative Malware, Adware and Ransomware Clickjacking Campaigns
- Vulnerability scanners
- Gambling and Porn Web Sites
- E-mail harvesters
- Content scrapers
- Link Ranking Bots
- Aggressive bots that scrape content
- Image Hotlinking Sites and Image Thieves
- Bots or Servers linked to viruses or malware
- Government surveillance bots
- Botnet Attack Networks (Mirai)
- Known Wordpress Theme Detectors (Updated Regularly)
- SEO companies that your competitors use to try improve their SEO
- Link Research and Backlink Testing Tools
- Stopping Google Analytics Ghost Spam
- Browser Adware and Malware (Yontoo etc)
WHat is NGINX BAD BOT BLOCKER?
NGINX BAD BOT BLOCKER is an open-source project that enables you to block these unwanted requests efficiently.
The project repo provides detailed instructions on how to install, configure, and use these protective features, allowing users to safeguard their Nginx servers effectively.
Features of NGINX BAD BOT BLOCKER!
- Extensive Lists of Bad and Known Bad Bots and Scrapers (updated almost daily)
- Blocking of Spam Referrer Domains and Web Sites
- Blocking of SEO data collection companies like Semalt.com, Builtwith.com, WooRank.com and many others (updated regularly)
- Blocking of clickjacking Sites linked to Adware, Malware and Ransomware
- Blocking of Porn and Gambling Web Sites who use Lucrative Ways to Earn Money through Serving Ads by hopping off your domain names and web sites.
- Blocking of Bad Domains and IP's that you cannot even see in your Nginx Logs. Thanks to the Content Security Policy (CSP) on all my SSL sites I can see things trying to pull resources off my sites before they even get to Nginx and get blocked by the CSP.
- Anti DDOS Filter and Rate Limiting of Agressive Bots
- Alphabetically ordered for easier maintenance (Pull Requests Welcomed)
- Commented sections of certain important bots to be sure of before blocking
- Includes the IP range of Cyveillance who are known to ignore robots.txt rules and snoop around all over the Internet.
- Whitelisting of Google, Bing and Cloudflare IP Ranges
- Whitelisting of your own IP Ranges that you want to avoid blocking by mistake.
- Ability to add other IP ranges and IP blocks that you want to block out.
- If it's out there and it's bad it's already in here and BLOCKED !!
The Developer
The project is created by Mitchell Krog, who is a software developer and contributor to several projects focused on internet security, particularly in the realm of phishing threat detection.
Mitchell Krog is a Linux Specialist focus on network security, intrusion detection, SSL and generally a safer internet. Proxmox, Wordpress, Woocommerce, Server Setup Expert 4 Hire
One of his notable contributions is the Phishing.Database repository on GitHub, which tracks phishing domains and URLs.
This project regularly updates lists of active, inactive, and invalid phishing domains using tools like PyFunceble for automated domain status testing.
The aim is to keep users and services informed about malicious domains and provide resources for blocking these threats. Krog's work is highly regarded in the cybersecurity community for helping mitigate phishing attacks through this open-source initiative.
Project Acknowledgments & Contributors
The project owes much of its success to the contributions of several developers and communities, who provided code snippets, scripts, and ongoing support.
A few key individuals made notable contributions:
- Konstantin Goretzki improved the Fail2Ban filter's regex, enhancing its ability to detect and block repeat offenders.
- Stuart Cardall contributed to the installation, update, and setup scripts while also maintaining the Alpine Linux packages for the project.
- Mike van Eckendonk and Nissar Chababy (creator of PyFunceble) contributed crucial domain-checking scripts, which help keep the list of blocked domains up-to-date by identifying active, inactive, and expired domains.
Other valuable contributors include Marius Voila, Cătălin Mariș, deformhead, bluedragonz, Alexander, Steven Black, and Stevie-Ray Hartog, each of whom provided enhancements across various parts of the project.
Moreover, Sir Athos and the broader StackOverflow and SuperUser communities were instrumental in solving challenges related to bash scripts and automation. These individuals and communities played essential roles in continuously improving the bot blocker, bringing new features, security enhancements, and ease of use to the project.
License
MIT License
Copyright (c) 2017 Mitchell Krog
Resources & Downloads
mitchellkrogza
