search

Home >Backend Development >PHP Tutorial >PHP determines whether the visitor is a search engine crawler

PHP determines whether the visitor is a search engine crawler

WBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWB
WBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOriginal
2016-07-25 08:44:241127browse

We can use HTTP_USER_AGENT to determine whether it is a spider. Search engine spiders have their own unique signs, some of which are listed below.

  1. function is_crawler() {
  2. $userAgent = strtolower($_SERVER['HTTP_USER_AGENT']);
  3. $spiders = array(
  4. 'Googlebot', // Google crawler
  5. 'Baiduspider', // Baidu crawler
  6. 'Yahoo! Slurp', // Yahoo crawler
  7. 'YodaoBot', // Youdao crawler
  8. 'msnbot' // Bing crawler
  9. // More crawler keywords
  10. );
  11. foreach ($spiders as $spider) {
  12. $spider = strtolower($spider);
  13. if (strpos($userAgent, $spider) !== false) {
  14. return true;
  15. }
  16. }
  17. return false;
  18. }
Copy code

The php code below comes with more spider logos
  1. function isCrawler() {
  2. echo $agent= strtolower($_SERVER['HTTP_USER_AGENT']);
  3. if (!empty($agent)) {
  4. $ spiderSite= array(
  5. "TencentTraveler",
  6. "Baiduspider+",
  7. "BaiduGame",
  8. "Googlebot",
  9. "msnbot",
  10. "Sosospider+",
  11. "Sogou web spider",
  12. "ia_archiver",
  13. "Yahoo! Slurp",
  14. "YoudaoBot",
  15. "Yahoo Slurp",
  16. "MSNBot",
  17. "Java (Often spam bot)",
  18. "BaiDuSpider",
  19. "Voila",
  20. "Yandex bot",
  21. "BSpider",
  22. "twiceler",
  23. "Sogou Spider",
  24. "Speedy Spider",
  25. "Google AdSense",
  26. "Heritrix",
  27. "Python-urllib",
  28. "Alexa (IA Archiver)",
  29. "Ask",
  30. "Exabot",
  31. "Custo",
  32. "OutfoxBot/YodaoBot",
  33. "yacy",
  34. "SurveyBot",
  35. "legs",
  36. "lwp-trivial",
  37. "Nutch",
  38. "StackRambler",
  39. " The web archive (IA Archiver)",
  40. "Perl tool",
  41. "MJ12bot",
  42. "Netcraft",
  43. "MSIECrawler",
  44. "WGet tools",
  45. "larbin",
  46. "Fish search",
  47. );
  48. foreach($spiderSite as $val) {
  49. $str = strtolower($val);
  50. if (strpos($agent, $str) !== false) {
  51. return true;
  52. }
  53. }
  54. } else {
  55. return false;
  56. }
  57. }
  58. if (isCrawler()){
  59. echo "Hello Spider! ";
  60. }
  61. else{
  62. echo "You are not a spider spirit! ";
  63. }
Copy code

php


Statement：
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Previous article：PHP realizes saving website visits as shortcut desktop modeNext article：PHP realizes saving website visits as shortcut desktop mode

Related articles

See more