python install scrapy - failed with error code 1 - python

I tried to install scrapy by "pip install scrapy", and I got:
c:\python36\python.exe -u -c "import setuptools,
tokenize;__file__='C:\\Users\\tzvia\\AppData\\Local\\Temp\\pip-install-
07h63108\\Twisted\\setup.py';f=getattr(tokenize, 'open', open)
(__file__);code=f.read().replace('\r\n',
'\n');f.close();exec(compile(code, __file__, 'exec'))" install --record
C:\Users\tzvia\AppData\Local\Temp\pip-record-hy8qkma1\install-record.txt
--single-version-externally-managed --compile" failed with error code 1
in C:\Users\tzvia\AppData\Local\Temp\pip-install-07h63108\Twisted\
I tried "conda install -c conda-forge scrapy" and it seems like the scrapy already installed:
C:\Users\tzvia>conda install -c conda-forge scrapy
Solving environment: done
# All requested packages already installed.
so I tried using scrapy and go an error:
C:\Users\tzvia>scrapy
Traceback (most recent call last):
File "C:\Users\tzvia\Anaconda3\Scripts\scrapy-script.py", line 6, in
<module>
from scrapy.cmdline import execute
File "C:\Users\tzvia\Anaconda3\lib\site-packages\scrapy\__init__.py", line
34, in <module>
from scrapy.spiders import Spider
File "C:\Users\tzvia\Anaconda3\lib\site-packages\scrapy\spiders\__init__.py", line 10, in <module>
from scrapy.http import Request
File "C:\Users\tzvia\Anaconda3\lib\site-packages\scrapy\http\__init__.py", line 11, in <module>
from scrapy.http.request.form import FormRequest
File "C:\Users\tzvia\Anaconda3\lib\site-
packages\scrapy\http\request\form.py", line 11, in <module>
import lxml.html
File "C:\Users\tzvia\Anaconda3\lib\site-packages\lxml\html\__init__.py",
line 54, in <module>
from .. import etree
ImportError: DLL load failed: The specified module could not be found.
I tried to install lxml, and got again "failed with error code 1".
can someone help me?
I use python 3.6.1 and windows 10

I think it is because your Twisted install failed.
Download the Twisted .whl file from https://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted .
Then run
pip install <Twisted-your-version.whl>

Installing lxml on Windows can be a bit of pain.
You might want to check out the info they have on their site here. They suggest that if building from source fails you might want to use the unofficial pre-built binaries by downloading from here and then running
python -m pip install .\lxml-4.2.5-cp37-cp37m-win32.whl
Make sure to download the version that you need. If you're using python 3.6 you should get the cp36 one.

Forget about pip install! The issue with 'scrapy' is all over the Web. Download 'Scrapy-2.5.0.tar.gz' (or whatever version is current) from https://pypi.org/project/Scrapy/, exract it to a temp folder and run < python setup.py install >

Related

conda env: pip install . fails whereas python setup.py install succeeds

I have what seems a simple question though I could not find a solution. I want to install a library (TIGRE), which comes with a setup.pyfile, to my conda-environment. In order to do that, I'm running pip install ., of course with the environment activated.
Nevertheless, python gives me the following error
Traceback (most recent call last):
File "C:\Users\***\Miniconda3\envs\tigre3.9\lib\site-packages\pip\_vendor\pep517\in_process\_in_process.py", line 363, in <module>
main()
...
...
File "C:\Users\***\AppData\Local\Temp\pip-build-env-4t_kjciu\overlay\Lib\site-packages\setuptools\build_meta.py", line 174, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 11, in <module>
from Cython.Distutils import build_ext
ModuleNotFoundError: No module named 'Cython'
even though Cython is infact installed in that exact same environment. Running python setup.py install runs fine however. What could be the cause of this problem? Why doesn't pip find the package when installing, but finds it e.g. with pip freeze?
Since there is something else wrong with the library I wanted to follow the installation guide step-by-step and use pip install ., because obviously both commands are not the exact same.
Thank you!

No module named '_ctypes'

I'm trying to install pyautogui, but pip keeps throwing errors. How to fix it? I've tried installing libffi library. Here is some code:
python3 -m pip install pyautogui
Defaulting to user installation because normal site-packages is not writeable
Collecting pyautogui
Using cached PyAutoGUI-0.9.50.tar.gz (57 kB)
ERROR: Command errored out with exit status 1:
command: /usr/local/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-sxm4ewnq/pyautogui/setup.py'"'"'; __file__='"'"'/tmp/pip-install-sxm4ewnq/pyautogui/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-85ugzov6
cwd: /tmp/pip-install-sxm4ewnq/pyautogui/
Complete output (11 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/local/lib/python3.8/site-packages/setuptools/__init__.py", line 19, in <module>
from setuptools.dist import Distribution
File "/usr/local/lib/python3.8/site-packages/setuptools/dist.py", line 34, in <module>
from setuptools import windows_support
File "/usr/local/lib/python3.8/site-packages/setuptools/windows_support.py", line 2, in <module>
import ctypes
File "/usr/local/lib/python3.8/ctypes/__init__.py", line 7, in <module>
from _ctypes import Union, Structure, Array
ModuleNotFoundError: No module named '_ctypes'
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
That's from python REPL
>>> sys.path
['', '/home/walenty/apps/Python-3.8.5/Modules/_ctypes', '/usr/local/lib/python38.zip', '/usr/local/lib/python3.8', '/usr/local/lib/python3.8/lib-dynload', '/home/walenty/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/site-packages']
>>> import _ctypes
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named '_ctypes'
Required
Install foreign function interface headers
sudo apt install libffi-dev
Reinstall Python
Substitute desired python version
Ubuntu
sudo add-apt-repository ppa:deadsnakes/ppa -y && sudo apt reinstall python3.9-distutils
MacOS
Use brew install python3.9 or port install python3.9 (I recommend port)
Windows
Use Microsoft Store
Specify project python version
Poetry
poetry env use 3.9
Virtual envs
virtualenv -p python3.9 myproject
etc...
okay, I've got it. This is the answer Python3: ImportError: No module named '_ctypes' when using Value from module multiprocessing
I cloned python3.10 from git and installed it from scratch.
As other answers say, you need to install libffi-dev. If you're using pyenv/virtualenv, also reinstall the base python version:
Install libffi
sudo apt-get install libffi-dev
Load the new libffi.so, as suggested here
sudo ldconfig
Reinstall the python version available to pyenv
pyenv install 3.9.12
Finally, create the fresh virtualenv
pyenv virtualenv 3.9.12 new_environment

ModuleNotFoundError during pip install when module is actually installed

I tried installing a python package (BentoML) using pip install bentoml, and I received the following error, which gives a ModuleNotFoundError: No module named 'wrapt'. However, the wrapt module is actually included in my site-packages folder, so a pip install wrapt results in the message that I already have wrapt installed.
I can install other packages using pip, but the alembic package causes this issue. I was able to install using pip using my Anaconda Python, but I want it installed with this Python version. What is the best way forward?
I'm using Python 3.8.1 on a Windows 10 device.
ERROR: Command errored out with exit status 1:
command: 'c:\users\username\appdata\local\programs\python\python38\python.exe' 'c:\users\username\appdata\local\programs\python\python38\lib\site-packages\pip\_vendor\pep517\_in_process.py' get_requires_for_build_wheel 'C:\Users\user~1.lastname\AppData\Local\Temp\tmpd30adik_'
cwd: C:\Users\username\AppData\Local\Temp\pip-install-qers2adl\alembic
Complete output (14 lines):
Fatal Python error: init_import_size: Failed to import the site module
Python runtime state: initialized
Traceback (most recent call last):
File "c:\users\username\appdata\local\programs\python\python38\lib\site.py", line 580, in <module>
main()
File "c:\users\username\appdata\local\programs\python\python38\lib\site.py", line 573, in main
execsitecustomize()
File "c:\users\username\appdata\local\programs\python\python38\lib\site-packages\certifi_win32\bootstrap.py", line 37, in _execsitecustomize
_register_bootstrap_functions()
File "c:\users\username\appdata\local\programs\python\python38\lib\site-packages\certifi_win32\bootstrap.py", line 25, in _register_bootstrap_functions
from . import wrapt_certifi
File "c:\users\username\appdata\local\programs\python\python38\lib\site-packages\certifi_win32\wrapt_certifi.py", line 3, in <module>
import wrapt
ModuleNotFoundError: No module named 'wrapt'
----------------------------------------
ERROR: Command errored out with exit status 1: 'c:\users\username\appdata\local\programs\python\python38\python.exe' 'c:\users\username\appdata\local\programs\python\python38\lib\site-packages\pip\_vendor\pep517\_in_process.py' get_requires_for_build_wheel 'C:\Users\user~1.lastname\AppData\Local\Temp\tmpd30adik_' Check the logs for full command output.
Install using another version of Python. Open up an Anaconda prompt (or another version of Python on the same device), and run:
pip install alembic --target="c:\users\username\appdata\local\programs\python\python38\lib\site-packages"
This was the same approach I used when I broke Python after trying to uninstall/reinstall wrapt. The other Python version installs alembic to the correct location and the original install works.

Trouble shooting when tring to install and import `stats_exporter` from `opencensus.ext.stackdriver`

Im trying to install and use stats_exporter from opencensus.ext.stackdriver using the following guide: opencensus-ext-stackdriver
after installing it through pip:
pip install opencensus-ext-stackdriver
Im trying to import it and:
from opencensus.ext.stackdriver import stats_exporter as stackdriver
ImportError: cannot import name 'stats_exporter' from 'opencensus.ext.stackdriver'
When comparing the Git repo, and my local venv/lib/python3.7/site-packages/... it seems like the pip version isn't compatible with Github , so i tried to install it though cloning, and using setup.py
pip install ../opencensus-python/contrib/opencensus-ext-stackdriver/dist/opencensus-ext-stackdriver-0.2.dev0.tar.gz
which gives me the following error:
(venv) Yehoshaphats-MacBook-Pro:present-value yehoshaphatschellekens$ pip install ../opencensus-python/contrib/opencensus-ext-stackdriver/dist/opencensus-ext-stackdriver-0.2.dev0.tar.gz
Processing /Users/yehoshaphatschellekens/opencensus-python/contrib/opencensus-ext-stackdriver/dist/opencensus-ext-stackdriver-0.2.dev0.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/s2/y6vcdc1105s8xlpb12slr9z00000gn/T/pip-req-build-7m1ibdpd/setup.py", line 17, in <module>
from version import __version__
ModuleNotFoundError: No module named 'version'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/s2/y6vcdc1105s8xlpb12slr9z00000gn/T/pip-req-build-7m1ibdpd/
Similar errors of this type indicated that i need to upgrade setuptools, tried that also :(
This post suggests that it might related to the fact that i'm using python3, which isn't completable with version though i really need to install this package on my python3 venv.
Any Help on this issue would be great!
Try this:
#!/usr/bin/env python
import os
from opencensus.common.transports.async_ import AsyncTransport
from opencensus.ext.stackdriver import trace_exporter as stackdriver_exporter
from opencensus.trace import tracer as tracer_module
from opencensus.stats import stats as stats_module
def main():
sde = stackdriver_exporter.StackdriverExporter(
project_id=os.environ.get("PROJECT_ID"),
transport=AsyncTransport)
tracer = tracer_module.Tracer(exporter=sde)
with tracer.span(name='doingWork') as span:
for i in range(10):
continue
if __name__ == "__main__":
main()
and
grpcio==1.19.0
opencensus==0.3.1
opencensus-ext-stackdriver==0.1.1
NB The OpenCensus libraries need gRPC too.
You will need:
a GCP Project and it's Project ID (${PROJECT_ID})
a service account with roles/cloudtrace.agent and its key.
Then:
virtualenv venv
source venv/bin/activate
export PROJECT_ID=[[YOUR-PROJECT-ID]]
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/your/key.json
pip3 install --requirement requirements.txt
python3 stackdriver.py

Can't access Python modules

I can't get any python modules to install, please help me if you can.
I'm running Windows and I have python 3.5.1 (v3.5.1:37a07cee5969) [MSC v.1900 64 bit (AMD64)] on win32. Python35 is installed to C:\Python35. I also have Anaconda3 installed. I've set up PATH to include: (C:\Python35\Scripts\;C:\Anaconda3;C:\Anaconda3\Lib\site-packages\;C:\Anaconda3\Scripts), which I think encompasses everything I need for that.
Specifically, I'm trying to install Numpy, Pandas, and the Quandl module using pip install. From the command prompt, I've already done "conda install numpy", "conda install pandas", and "conda install Quandl", all of which installed and I can find them in C:\Anaconda3\Lib\site-packages\, but when I go to the command prompt and access python, and then enter "import numpy" (or pandas), it returns a Traceback error - ImportError: No module named 'numpy' ('pandas'). When I try to enter "import Quandl", it errors returning:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python35\lib\site-packages\Quandl\__init__.py", line 11, in <module>
from .Quandl import (
File "C:\Python35\lib\site-packages\Quandl\Quandl.py", line 11, in <module>
import pandas as pd
ImportError: No module named 'pandas'
For more information, when I attempt to do 'pip install numpy' from command prompt, it errors out on a lot of things stating "NOT AVAILABLE", and at the bottom is states:
Command "c:\python35\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\BradyA\\AppData\\Local\\Temp\\pip-build-9t5yqzmk\\numpy\\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record C:\Users\BradyA\AppData\Local\Temp\pip-i6hj44h_-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in C:\Users\BradyA\AppData\Local\Temp\pip-build-9t5yqzmk\numpy\
Any thoughts? Thank you.
Here is a solution that worked for many:
Open cmd as admin.
Go to required pythons scripts folder: C:\Python34\Scripts
Now type pip pip install --upgrade pip(just a check)
For NumPy (you cannot do it directly)(others also):
4.1. Download all the packages you need from here :
4.2. Now type : pip install [file location of package]
4.3. This should do the trick.
Note: For more detailed explanation please look at this tutorial
Keep me posted on the progress.
Try run your command prompt as an administrator, then issuing the pip install commands.
It seems that you have conflict between the two different installations of Python.
I had similar problems in the past and deleted all Pythons, cleaned the path and reinstalled Anaconda and used Anaconda environments to work with multiple versions of Python.
Installing with conda should work with you have just Anaconda.

Categories