DOCSEARCHbyAlgolia
Submit Docs

CrawlerDetect Documentation

repository·master·Indexed 25 days ago

https://github.com/jaybizzle/crawler-detect

A PHP library for detecting bots, crawlers, and spiders by analyzing User-Agent and HTTP_FROM headers. It provides methods to identify automated traffic via isCrawler(), retrieve matching bot names with getMatches(), and manually configure headers and user agents.

Tokens
891
Snippets
2
Records
8
Agent score
32%

What's inside CrawlerDetect

  1. Detect crawlers and bots with CrawlerDetect

    master

    CrawlerDetect is a PHP library that identifies bots, crawlers, and spiders by analyzing the User-Agent and HTTP_FROM headers. You can use it to check the current visitor's user agent or pass a specific string for inspection.

    use Jaybizzle\\\CrawlerDetect\\\CrawlerDetect;

$CrawlerDetect = new CrawlerDetect;

// Check the user agent of the current visitor
if ($CrawlerDetect->isCrawler()) {
    // true if a crawler user agent was detected
}

// Pass a user agent as a string
if ($CrawlerDetect->isCrawler('Mozilla/5.0 (compatible; Sosospider/2.0; +http://help.soso.com/webspider.htm')) {
    // true if a crawler user agent was detected
}

// Output the name of the bot that matched (if any)
echo $CrawlerDetect->getMatches();

  3. Use the CrawlerDetect class methods

    master

    The Jaybizzle\CrawlerDetect\CrawlerDetect class provides the following primary methods for detection:

    • isCrawler(): Returns true if the current visitor's user agent is recognized as a crawler.
    • isCrawler(string $userAgent): Returns true if the provided user agent string is recognized as a crawler.
    • getMatches(): Returns the name of the bot that matched the current user agent (or the provided string).

  4. Configure the User-Agent with setUserAgent()

    master
    The setUserAgent() method allows you to explicitly set the User-Agent string used for detection. If you pass null, the class will attempt to build a User-Agent string by concatenating values from the headers currently stored in the instance, using the keys defined in the Headers fixture.

  6. Initialize CrawlerDetect

    master
    To use the library, instantiate the CrawlerDetect class. You can optionally pass an array of HTTP headers and a specific User-Agent string to the constructor. If no headers are provided, the class defaults to using the PHP $_SERVER superglobal. If no User-Agent is provided, the class will attempt to construct one by looking for common User-Agent headers within the provided headers array.

  7. Configure HTTP headers with setHttpHeaders()

    master
    You can manually update the headers used for detection using setHttpHeaders(). The method filters the input array and only retains keys that start with HTTP_ (the standard format for HTTP headers in PHP's $_SERVER array). If an empty array or null is passed, the class reverts to using the global $_SERVER superglobal.

  8. Detect if a User-Agent is a crawler using isCrawler()

    master

    The isCrawler() method is the primary way to check if a request is coming from a bot, crawler, or spider. It works by checking the User-Agent string against a compiled list of known crawler patterns while simultaneously applying a list of exclusions.

    If you call isCrawler() without arguments, it uses the User-Agent stored in the instance. If you pass a string, it uses that string instead. It returns true if a match is found, and false otherwise.