./www/py-robot-detection, Detect web crawlers using HTTP User Agent

[ CVSweb ] [ Homepage ] [ RSS ] [ Required by ] [ Add to tracker ]

Branch: CURRENT, Version: 0.4, Package name: py310-robot-detection-0.4, Maintainer: pkgsrc-users

robot_detection is a python module to detect if a given HTTP User Agent
is a web crawler. It uses the list of registered robots from
http://www.robotstxt.org: (Robots Database)

Master sites:

Filesize: 6.237 KB

Version history: (Expand)

CVS history: (Expand)


   2023-05-10 23:59:57 by Mark Davies | Files touched by this commit (4) 
Log message:
py-robot-detection: add version 0.4

robot_detection is a python module to detect if a given HTTP User Agent
is a web crawler. It uses the list of registered robots from
http://www.robotstxt.org: (Robots Database)