.. index::  scrapy
.. _scrapy:

scrapy
======

Description
-----------

Scrapy is a web crawler and screen/web scaper.

User instructions
-----------------

scrapy is available via the modules system.  To add it to your environment run:

.. code-block:: bash

    $ module load scrapy/1.4.0

Extensive documentation is available on the `scrapy website <http://www.scrapy.org>`_.

Source
------

http://www.scrapy.org

License
-------

BSD.

Admin notes
-----------

The package along with it dependencies were installed to the module using pip.

.. code-block:: bash

    PYTHONUSERBASE=/common/debian/9.1/PythonPackages/python/2.7/scrapy/scrapy-1.4.0 pip install --upgrade scrapy
    PYTHONUSERBASE=/common/debian/9.1/PythonPackages/python/3.5/scrapy/scrapy-1.4.0 pip3 install --upgrade scrapy