Ndtyjky

I am attempting to deploy a spider to Scrapy cloud, but I am repeatedly running into requirements problems, even importing commonly used modules such as io, re, shlex and PyPDF2. I am using Python 3.5. My scrapinghub.yml file contains these lines:

projects:

    default: 358310

stacks:

    default: scrapy:1.3-py3

requirements:

    file: requirements.txt

My requirements.txt file contains these lines:

io

re

shlex

PyPDF2==1.26.0

This is the error I got:

Login succeeded

Building an image:

Step 1/12 : FROM scrapinghub/scrapinghub-stack-scrapy:1.3-py3

[91m# Executing 2 build triggers...

[0m

Step 1/1 : ENV PIP_TRUSTED_HOST $PIP_TRUSTED_HOST PIP_INDEX_URL $PIP_INDEX_URL

 ---> Using cache

Step 1/1 : RUN test -n $APT_PROXY && echo 'Acquire::http::Proxy "$APT_PROXY";'     >/etc/apt/apt.conf.d/proxy

 ---> Using cache

 ---> 8d2af0ecc1ce

Step 2/12 : ENV PYTHONUSERBASE /app/python

 ---> Using cache

 ---> c5bc537289c7

Step 3/12 : ADD eggbased-entrypoint /usr/local/sbin/

 ---> Using cache

 ---> 210ce92ef42e

Step 4/12 : ADD run-pipcheck /usr/local/bin/

 ---> Using cache

 ---> 2d0a46143fa4

Step 5/12 : RUN chmod +x /usr/local/bin/run-pipcheck

 ---> Using cache

 ---> a2eefa41c642

Step 6/12 : RUN chmod +x /usr/local/sbin/eggbased-entrypoint &&     ln -sf /usr/local/sbin/eggbased-entrypoint /usr/local/sbin/start-crawl &&     ln -sf /usr/local/sbin/eggbased-entrypoint /usr/local/sbin/scrapy-list &&     ln -sf /usr/local/sbin/eggbased-entrypoint /usr/local/sbin/shub-image-info &&     ln -sf /usr/local/sbin/eggbased-entrypoint /usr/local/sbin/run-pipcheck

 ---> Using cache

 ---> f3f5b5c713e3

Step 7/12 : ADD requirements.txt /app/requirements.txt

 ---> Using cache

 ---> e6417a1c3fea

Step 8/12 : RUN mkdir /app/python && chown nobody:nogroup /app/python

 ---> Using cache

 ---> 6720be2ef393

Step 9/12 : RUN sudo -u nobody -E PYTHONUSERBASE=$PYTHONUSERBASE     pip install --user --no-cache-dir -r /app/requirements.txt

 ---> Running in 8d8694511a74

Collecting io (from -r /app/requirements.txt (line 1))

[91m  Could not find a version that satisfies the requirement io (from -r /app/requirements.txt (line 1)) (from versions: )

[0m

[91mNo matching distribution found for io (from -r /app/requirements.txt (line 1))

[0m

[91mYou are using pip version 9.0.3, however version 18.1 is available.

You should consider upgrading via the 'pip install --upgrade pip' command.

[0m

{"message": "The command '/bin/sh -c sudo -u nobody -E PYTHONUSERBASE=$PYTHONUSERBASE     pip install --user --no-cache-dir -r /app/requirements.txt' returned a non-zero code: 1", "details": {"message": "The command '/bin/sh -c sudo -u nobody -E PYTHONUSERBASE=$PYTHONUSERBASE     pip install --user --no-cache-dir -r /app/requirements.txt' returned a non-zero code: 1", "code": 1}, "error": "requirements_error"}

Where am I going wrong? BTW, I have the latest version of pip installed (contrary to what the error message states).

io, re, shlex are internal libraries, not packages, as I know. They should not be in requirements file.