scrapy

Description

Scrapy is a web crawler and screen/web scaper.

User instructions

scrapy is available via the modules system. To add it to your environment run:

$ module load scrapy/1.4.0

Extensive documentation is available on the scrapy website.

License

BSD.

Admin notes

The package along with it dependencies were installed to the module using pip.

PYTHONUSERBASE=/common/debian/9.1/PythonPackages/python/2.7/scrapy/scrapy-1.4.0 pip install --upgrade scrapy
PYTHONUSERBASE=/common/debian/9.1/PythonPackages/python/3.5/scrapy/scrapy-1.4.0 pip3 install --upgrade scrapy