Path to this page:
Subject: CVS commit: pkgsrc/www/py-scrapy
From: Adam Ciarcinski
Date: 2017-05-20 08:25:36
Message id: 20170520062536.8D2D6FBE4@cvs.NetBSD.org
Log Message:
Scrapy 1.4 does not bring that many breathtaking new features
but quite a few handy improvements nonetheless.
Scrapy now supports anonymous FTP sessions with customizable user and
password via the new :setting:`FTP_USER` and :setting:`FTP_PASSWORD` settings.
And if you're using Twisted version 17.1.0 or above, FTP is now available
with Python 3.
There's a new :meth:`response.follow <scrapy.http.TextResponse.follow>` method
for creating requests; **it is now a recommended way to create Requests
in Scrapy spiders**. This method makes it easier to write correct
spiders; ``response.follow`` has several advantages over creating
``scrapy.Request`` objects directly:
* it handles relative URLs;
* it works properly with non-ascii URLs on non-UTF8 pages;
* in addition to absolute and relative URLs it supports Selectors;
for ``<a>`` elements it can also extract their href values.
Files: