This question already has answers here:
python spyder conda install failure
(2 answers)
Why does "pip install" inside Python raise a SyntaxError?
(7 answers)
Closed 5 years ago.
I am installing Scrapy in Python 3.5. I am referring to the installation guide here .
I have installed pip and conda. I tried both ways mentioned in the guide, but am getting this error:
I have already installed pip and minoconda:
conda install -c scrapinghub scrapy
and
pip install Scrapy
What is wrong with the installation?
You're trying to install scrapy in the python interpreter, you should install the package in your terminal. What's more, one requirement of installing scrapy is python 2.7, you're using python 3.5
Update:
Good news: Python 3 is Coming to Scrapy, you can try pip install scrapy==1.1.0rc1 now. Note that there are still some limitations, but won't be long that scrapy will have full of 3.x support. Pay close attention to the scrapy development ;)
Scrapy doesn't work with Python 3. At the moment, scrapy works only with Python 2.7.
You can pip install scrapy on python 3.5+, you just have to pip install all the deppendencies first - from here https://docs.scrapy.org/en/latest/intro/install.html
pip install lxml
pip install parsel
pip install w3lib
pip install twisted
pip install cryptography
pip install cryptography
pip install pyOpenSSL
and finally
pip install scrapy - worked for me
Before installing Scrapy on Python 3.5 on Windows 10 in virtualenv
Install lxml,PyDispatcher then simpy install Scrapy code is attached.
(pymote_env) C:\Users\DDSHARMA\Documents\pymote_env>pip install PyDispatcher
Collecting PyDispatcher
Installing collected packages: PyDispatcher
Successfully installed PyDispatcher-2.0.5
(pymote_env) C:\Users\DDSHARMA\Documents\pymote_env>pip install Scrapy
Collecting Scrapy
Using cached Scrapy-1.2.1-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): six>=1.5.2 in c:\users\ddsharma\documents\pymote_env\lib\site-packages (from Scrapy)
Collecting service-identity (from Scrapy)
Using cached service_identity-16.0.0-py2.py3-none-any.whl
Collecting Twisted>=10.0.0 (from Scrapy)
Using cached Twisted-16.4.1.tar.bz2
Collecting parsel>=0.9.3 (from Scrapy)
Using cached parsel-1.0.3-py2.py3-none-any.whl
Collecting pyOpenSSL (from Scrapy)
Using cached pyOpenSSL-16.2.0-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): PyDispatcher>=2.0.5 in c:\users\ddsharma\documents\pymote_env\lib\site-packages (from Scrapy)
Requirement already satisfied (use --upgrade to upgrade): lxml in c:\users\ddsharma\documents\pymote_env\lib\site-packages (from Scrapy)
Collecting queuelib (from Scrapy)
Using cached queuelib-1.4.2-py2.py3-none-any.whl
Collecting w3lib>=1.15.0 (from Scrapy)
Using cached w3lib-1.15.0-py2.py3-none-any.whl
Collecting cssselect>=0.9 (from Scrapy)
Using cached cssselect-1.0.0-py2.py3-none-any.whl
Collecting pyasn1-modules (from service-identity->Scrapy)
Using cached pyasn1_modules-0.0.8-py2.py3-none-any.whl
Collecting pyasn1 (from service-identity->Scrapy)
Using cached pyasn1-0.1.9-py2.py3-none-any.whl
Collecting attrs (from service-identity->Scrapy)
Using cached attrs-16.2.0-py2.py3-none-any.whl
Collecting zope.interface>=4.0.2 (from Twisted>=10.0.0->Scrapy)
Using cached zope.interface-4.3.2-cp35-cp35m-win32.whl
Collecting cryptography>=1.3.4 (from pyOpenSSL->Scrapy)
Using cached cryptography-1.5.2-cp35-cp35m-win32.whl
Requirement already satisfied (use --upgrade to upgrade): setuptools in c:\users\ddsharma\documents\pymote_env\lib\site-packages (from zope.interface>=4.0.2->Twisted>=10.0.0->Scrapy)
Collecting idna>=2.0 (from cryptography>=1.3.4->pyOpenSSL->Scrapy)
Using cached idna-2.1-py2.py3-none-any.whl
Collecting cffi>=1.4.1 (from cryptography>=1.3.4->pyOpenSSL->Scrapy)
Using cached cffi-1.8.3-cp35-cp35m-win32.whl
Collecting pycparser (from cffi>=1.4.1->cryptography>=1.3.4->pyOpenSSL->Scrapy)
Downloading pycparser-2.16.tar.gz (230kB)
100% |################################| 235kB 286kB/s
Building wheels for collected packages: Twisted, pycparser
Running setup.py bdist_wheel for Twisted ... done
Stored in directory: C:\Users\DDSHARMA\AppData\Local\pip\Cache\wheels\0e\53\62\e7b4cea7df9113fb2818b224eb5d143be981568d9c43057a0a
Running setup.py bdist_wheel for pycparser ... done
Stored in directory: C:\Users\DDSHARMA\AppData\Local\pip\Cache\wheels\34\cc\b5\34e805b130a9227f0681d5517cf554d950bb0361c3a373496f
Successfully built Twisted pycparser
Installing collected packages: pyasn1, pyasn1-modules, idna, pycparser, cffi, cryptography, pyOpenSSL, attrs, service-identity, zope.interface, Twisted, w3lib, cssselect, parsel, queuelib, Scrapy
Successfully installed Scrapy-1.2.1 Twisted-16.4.1 attrs-16.2.0 cffi-1.8.3 cryptography-1.5.2 cssselect-1.0.0 idna-2.1 parsel-1.0.3 pyOpenSSL-16.2.0 pyasn1-0.1.9 pyasn1-modules-0.0.8 pycparser-2.16 queuelib-1.4.2 service-identity-16.0.0 w3lib-1.15.0 zope.interface-4.3.2
(pymote_env) C:\Users\DDSHARMA\Documents\pymote_env>
I was able to install scrapy on Python 3.5.
Check to see if you are able to install these Twisted, lxml and pyOpenSSL.
In my case I got error in installing lxml. I fixed it by downloading lxml‑3.6.4‑cp35‑cp35m‑win_amd64.whl from http://www.lfd.uci.edu/~gohlke/pythonlibs/#lxml based on your Python and windows version. (Credit to Christoph Gohlke's generosity for sharing this library.). Installed it using the following command:
pip install lxml‑3.6.4‑cp35‑cp35m‑win_amd64.whl
Related
I'm having trouble installing pyautogui, it gives me the following error. By the way, I'm new to python so if I'm not understanding something please explain it to me.
Thanks in advance!
pip install pyautogui
Collecting pyautogui
Using cached https://files.pythonhosted.org/packages/f0/76/7a0ec1013bc3559b7438f6773cba05ffaec600b8989be2d621a144e39b50/PyAutoGUI-0.9.53.tar.gz
Collecting pymsgbox (from pyautogui)
Collecting PyTweening>=1.0.1 (from pyautogui)
Using cached https://files.pythonhosted.org/packages/e9/4f/acb6a2d95260a4377885e40c167fd5df587630696a6a7934675f86aebb06/pytweening-1.0.4.tar.gz
Collecting pyscreeze>=0.1.21 (from pyautogui)
Collecting pygetwindow>=0.0.5 (from pyautogui)
Using cached https://files.pythonhosted.org/packages/e1/70/c7a4f46dbf06048c6d57d9489b8e0f9c4c3d36b7479f03c5ca97eaa2541d/PyGetWindow-0.0.9.tar.gz
Collecting mouseinfo (from pyautogui)
Using cached https://files.pythonhosted.org/packages/28/fa/b2ba8229b9381e8f6381c1dcae6f4159a7f72349e414ed19cfbbd1817173/MouseInfo-0.1.3.tar.gz
Collecting Pillow>=6.2.1; python_version == "3.8" (from pyscreeze>=0.1.21->pyautogui)
Using cached https://files.pythonhosted.org/packages/8f/59/97618ad67fc0639ed588c60cfe9d91417f7bae8c87bbe7c7784b0ffdb9f1/Pillow-9.2.0-cp38-cp38-win_amd64.whl
Collecting pyrect (from pygetwindow>=0.0.5->pyautogui)
Using cached https://files.pythonhosted.org/packages/cb/04/2ba023d5f771b645f7be0c281cdacdcd939fe13d1deb331fc5ed1a6b3a98/PyRect-0.2.0.tar.gz
Collecting pyperclip (from mouseinfo->pyautogui)
Using cached https://files.pythonhosted.org/packages/a7/2c/4c64579f847bd5d539803c8b909e54ba087a79d01bb3aba433a95879a6c5/pyperclip-1.8.2.tar.gz
Installing collected packages: pymsgbox, PyTweening, Pillow, pyscreeze, pyrect, pygetwindow, pyperclip, mouseinfo, pyautogui
ERROR: Could not install packages due to an EnvironmentError: [WinError 5] Access is denied: 'c:\program files\python38\Lib\site-packages\pymsgbox'
Consider using the --user option or check the permissions.
WARNING: You are using pip version 19.2.3, however version 22.1.2 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
The error actually gives you your answer by their recommendation to use the --user option. I run into this as well, if you want to install something without admin privileges. Just add the --user flag to the end of your pip command. This will limit the installation to the current user and not require administrator privileges.
Try this:
pip install pyautogui --user
The second error also gives you the answer of what to do.
pip install --upgrade pip
or, if needed...
pip install --upgrade pip --user
I did the following command just now,
pip install --upgrade ipykernel
However, i got
Requirement already satisfied: ipykernel in ./anaconda3/lib/python3.8/site-packages (5.3.4)
Collecting ipykernel
Downloading ipykernel-6.5.0-py3-none-any.whl (125 kB)
|████████████████████████████████| 125 kB 4.3 MB/s
Collecting ipython<8.0,>=7.23.1
Downloading ipython-7.29.0-py3-none-any.whl (790 kB)
|████████████████████████████████| 790 kB 9.2 MB/s
Collecting debugpy<2.0,>=1.0.0
Downloading debugpy-1.5.1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.9 MB)
|████████████████████████████████| 1.9 MB 126.8 MB/s
Collecting matplotlib-inline<0.2.0,>=0.1.0
Downloading matplotlib_inline-0.1.3-py3-none-any.whl (8.2 kB)
Collecting traitlets<6.0,>=5.1.0
Downloading traitlets-5.1.1-py3-none-any.whl (102 kB)
|████████████████████████████████| 102 kB 20.5 MB/s
...
Requirement already satisfied: python-dateutil>=2.1 in ./anaconda3/lib/python3.8/site-packages (from jupyter-client<8.0->ipykernel) (2.8.1)
Requirement already satisfied: ptyprocess>=0.5 in ./anaconda3/lib/python3.8/site-packages (from pexpect>4.3->ipython<8.0,>=7.23.1->ipykernel) (0.7.0)
Requirement already satisfied: wcwidth in ./anaconda3/lib/python3.8/site-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->ipython<8.0,>=7.23.1->ipykernel) (0.2.5)
Requirement already satisfied: six>=1.5 in ./anaconda3/lib/python3.8/site-packages (from python-dateutil>=2.1->jupyter-client<8.0->ipykernel) (1.15.0)
Installing collected packages: traitlets, matplotlib-inline, ipython, debugpy, ipykernel
Attempting uninstall: traitlets
Found existing installation: traitlets 5.0.5
Uninstalling traitlets-5.0.5:
Successfully uninstalled traitlets-5.0.5
Attempting uninstall: ipython
Found existing installation: ipython 7.22.0
Uninstalling ipython-7.22.0:
Successfully uninstalled ipython-7.22.0
Attempting uninstall: ipykernel
Found existing installation: ipykernel 5.3.4
Uninstalling ipykernel-5.3.4:
Successfully uninstalled ipykernel-5.3.4
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
spyder 4.2.5 requires pyqt5<5.13, which is not installed.
spyder 4.2.5 requires pyqtwebengine<5.13, which is not installed.
conda-repo-cli 1.0.4 requires pathlib, which is not installed.
Successfully installed debugpy-1.5.1 ipykernel-6.5.0 ipython-7.29.0 matplotlib-inline-0.1.3 traitlets-5.1.1
I would like to revert my command, since some of my codes suddenly does not work
. Is it possible? Thanks!
It seems a lot fo packages are installed.
Update
So I actually had my environment.yaml from my previous conda env export > environment.yaml
If I do
conda env update --file environment.yaml --prune
It does not help me revert to my previous versions.... Can I force my base environment back to exactly environment.yaml?
I do not believe pip keeps a history of installed packages. If you have the text output from the terminal when you did the upgrade, pip does output a list of packages that will be installed, which version you had and which version you're replacing it with. You can manually revert each package by doing
pip uninstall <package_name> && pip install <package_name>==<version_number>
Edit: Based on your edits, I suggest you repost this question in terms of your conda environment. That's a totally different system, even though it does utilize pip.
As a side note, you might be able to get away with just running
pip install traitlets==5.0.5
pip install ipykernel==5.3.4
pip install ipython==7.22.0
pip install attrs==19.2.0 --user Collecting attrs==19.2.0 Using cached attrs-19.2.0-py2.py3-none-any.whl (40 kB) Installing collected
packages: attrs Attempting uninstall: attrs
Found existing installation: attrs 19.1.0
Uninstalling attrs-19.1.0:
Successfully uninstalled attrs-19.1.0 Successfully installed attrs-19.2.0 Note: you may need to restart the kernel to use updated
packages. ERROR: pdflatex 0.1.3 has requirement attrs<19.0,>=18.2, but
you'll have attrs 19.2.0 which is incompatible.
pip install pdflatex --user Requirement already satisfied: pdflatex in
c:\users\haris\appdata\roaming\python\python37\site-packages (0.1.3)
Collecting attrs<19.0,>=18.2 Using cached
attrs-18.2.0-py2.py3-none-any.whl (34 kB) Installing collected
packages: attrs Attempting uninstall: attrs
Found existing installation: attrs 19.2.0
Uninstalling attrs-19.2.0:
Successfully uninstalled attrs-19.2.0 Successfully installed attrs-18.2.0 Note: you may need to restart the kernel to use updated
packages. ERROR: hypothesis 5.5.4 has requirement attrs>=19.2.0, but
you'll have attrs 18.2.0 which is incompatible.
need help to install pdflatex
I run my python scrapy project shows the error no module named 'requests'
So I type pip install requests
and then terminal information:
Requirement already satisfied: requests in ./Library/Python/2.7/lib/python/site-packages (2.22.0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in ./Library/Python/2.7/lib/python/site-packages (from requests) (3.0.4)
Requirement already satisfied: idna<2.9,>=2.5 in ./Library/Python/2.7/lib/python/site-packages (from requests) (2.8)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in ./Library/Python/2.7/lib/python/site-packages (from requests) (1.25.7)
Requirement already satisfied: certifi>=2017.4.17 in ./Library/Python/2.7/lib/python/site-packages (from requests) (2019.11.28)
type command pip list can see request 2.22.0
I type command python --version to check the python version:
python 2.7.16
Finally I run my scrapy project again still see the same error no module named 'requests'
I have no idea how to fix the error now, any help would be appreciated. Thanks.
Install python3 and pip3 and then pip3 install requests
if you are on ubuntu python3 is installed by default
you should first install pip3 by apt install python3-pip and then pip3 install requests
If you are using two different versions of Python, it should explain why you can't use your module.
To install the module on Python 3, try:
pip3 install requests
And make sure, you are using the correct version.
Check if you are using the same interpreter to which you have installed the package using pip install. As a best practice to avoid this type of issues when you have multiple versions of python, use pip as a module instead of directly calling pip.
eg:
python -m pip install requests
python3 -m pip install requests
Trying to install pyad package for Python 2.7.13 (windows x64) using pip.
There is pywin32 package among requirements, so we get an error:
C:\Windows\system32>pip install pyad
Collecting pyad
Using cached pyad-0.5.16.tar.gz
Requirement already satisfied: setuptools in c:\python27\lib\site-packages (from pyad)
Collecting pywin32 (from pyad)
Could not find a version that satisfies the requirement pywin32 (from pyad) (from versions: )
No matching distribution found for pywin32 (from pyad)
Then I install pypiwin32 which used to work in such cases:
C:\Windows\system32>pip install pypiwin32
Collecting pypiwin32
Using cached pypiwin32-219-cp27-none-win32.whl
Installing collected packages: pypiwin32
Successfully installed pypiwin32-219
But after this I still can't install pyad with the same error as above, like pypiwin32 has never been installed:
C:\Windows\system32>pip install pyad
Collecting pyad
Using cached pyad-0.5.16.tar.gz
Requirement already satisfied: setuptools in c:\python27\lib\site-packages (from pyad)
Collecting pywin32 (from pyad)
Could not find a version that satisfies the requirement pywin32 (from pyad) (from versions: )
No matching distribution found for pywin32 (from pyad)
Folder C:\Python27\Lib\site-packages\pywin32_system32 exists, the pypiwin32 package seems to be installed.
Any ideas to fix this?
You are facing pywin32 installation problem, which is a binary wheel.
You can download binary wheels of pywin32 here. You install it with
pip install pywin32-221-cp27-cp27m-win32.whl
if it is the file you have downloaded
Select the correct version with respect to your installation (Python version and 32/64 bit Python version)
You can have detailed instructions here
pyad is declaring it needs "the pywin32 distribution package" in order to be installed. Distribution packages, i.e. something from PyPI that you can pip install, differ from Python packages, folders with an __init__.py that can be imported. The naming is confusing.
So, because it's not installed, it tries to install pywin32, but it can't find a distribution that matches your Python environment: Python 2.7 on Windows x64. On PyPI it appears like the only distributions are for Python 3.6: https://pypi.python.org/pypi/pypiwin32/220