./www/ruby-webrobots, Ruby library to help write robots.txt compliant web robots

[ CVSweb ] [ Homepage ] [ RSS ] [ Required by ] [ Add to tracker ]


Branch: CURRENT, Version: 0.1.2, Package name: ruby23-webrobots-0.1.2, Maintainer: pkgsrc-users

This library helps write robots.txt compliant web robots in Ruby.


Required to run:
[textproc/ruby-nokogiri]

Required to build:
[pkgtools/cwrappers]

Master sites:

SHA1: c0db0eb3e383be42319a0614b878a4044420ed0f
RMD160: 2da85b8ba194677cfba8671a49293f81bb7c756b
Filesize: 23.5 KB

Version history: (Expand)


CVS history: (Expand)


   2017-09-04 20:08:31 by Thomas Klausner | Files touched by this commit (163)
Log message:
Follow some redirects.
   2016-02-01 15:08:23 by Takahiro Kambe | Files touched by this commit (3) | Package updated
Log message:
Update ruby-webrobots to 0.1.2.

* Make the default `http_get` proc act more like Googlebot.

  - It only allows up to 5 redirect hops rather than 10.
  - It treats too many redirections as full allow.
  - It treats any 4xx error as full allow.

* Fix context/should wording

* Follow the deprecation warning from SimpleCov::Formatter::MultiFormatter

* Update real world tests, introducing Webmock+VCR
   2015-11-04 03:47:43 by Alistair G. Crooks | Files touched by this commit (758)
Log message:
Add SHA512 digests for distfiles for www category

Problems found locating distfiles:
	Package haskell-cgi: missing distfile haskell-cgi-20001206.tar.gz
	Package nginx: missing distfile array-var-nginx-module-0.04.tar.gz
	Package nginx: missing distfile encrypted-session-nginx-module-0.04.tar.gz
	Package nginx: missing distfile headers-more-nginx-module-0.261.tar.gz
	Package nginx: missing distfile nginx_http_push_module-0.692.tar.gz
	Package nginx: missing distfile set-misc-nginx-module-0.29.tar.gz
	Package nginx-devel: missing distfile echo-nginx-module-0.58.tar.gz
	Package nginx-devel: missing distfile form-input-nginx-module-0.11.tar.gz
	Package nginx-devel: missing distfile lua-nginx-module-0.9.16.tar.gz
	Package nginx-devel: missing distfile nginx_http_push_module-0.692.tar.gz
	Package nginx-devel: missing distfile set-misc-nginx-module-0.29.tar.gz
	Package php-owncloud: missing distfile owncloud-8.2.0.tar.bz2

Otherwise, existing SHA1 digests verified and found to be the same on
the machine holding the existing distfiles (morden).  All existing
SHA1 digests retained for now as an audit trail.
   2015-06-13 00:57:04 by Takahiro Kambe | Files touched by this commit (1)
Log message:
ruby-webrobots builds on Ruby 2.2.  Fix one of pbulkscan problems.
   2014-04-28 23:57:46 by Joerg Sonnenberger | Files touched by this commit (2)
Log message:
Dependency closure of textproc/ruby-nokogiri
   2013-04-06 16:30:19 by Blue Rats | Files touched by this commit (2)
Log message:
Reverted HOMEPAGE as recommended by taca@
   2013-04-06 16:09:35 by Blue Rats | Files touched by this commit (22)
Log message:
Various MASTER_SITES-related fixes.
   2013-03-17 16:51:54 by Takahiro Kambe | Files touched by this commit (2) | Package updated
Log message:
Update ruby-webrobots to 0.1.1.

A few fixes.