Unable to install scrapy - python

I am trying to install scrapy and for that I have installed python 2.7 and conda(as pip was giving me errors that i couldn't solve.) after that I created a separate environment for scrapy named py2 with conda.
With this output I suppose that scrapy is installed successfully on my machine.
[py2] C:\Users\ADMIN>conda install -c https://conda.anaconda.org/scrapinghub scrapy
Fetching package metadata: ......
Solving package specifications: ...........................
# All requested packages already installed.
# packages in environment at C:\dev\Anaconda3\envs\py2:
#
scrapy 1.0.3 py27_2 scrapinghub
Now I make it sure that it is python 2.7 I executed this
[py2] C:\Users\ADMIN>py
Python 2.7.11 (v2.7.11:6d1b6a68f775, Dec 5 2015, 20:40:30) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
Which is also correct I suppose.
Now still I am unable to do this
>>> import scrapy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named scrapy
which indicates that scrapy is not installed.
Environment : Windows 8 (64 bit)
I am a beginner both in python and scrapy.

There are interestingly a few hacks that are needed to run Scrapy on Windows.
You would need to install unofficial support for Windows in Python, look for PyWin32 section:
http://www.lfd.uci.edu/~gohlke/pythonlibs/ Afterwards remember to run postinstall.py -install as advised on this page.
Depending how you are setup, you might need to install C++ compiler for Python, required to run lxml:
https://www.microsoft.com/en-us/download/details.aspx?id=44266
Please look for similar questions referring to overall Python not only Scrapy e.g. error: Unable to find vcvarsall.bat Depending on how you are setup you will find hints around.

As it was all in virtual environment of conda, while installing scrapy I needed to pass the name of conda virtual environment, except that scrapy would have been installed in the root(I mean outside the environment or for the machine.)
So for py2 virtual environment, after activating this, following command will install scrapy:
conda install -n py2 -c https://conda.anaconda.org/scrapinghub scrapy
Here py2 is environment's name.
This page helped me to figure out what was happening here.
I must say python is not that easy to learn.

Related

Conda vtk is already installed but python shows - No module named vtk

I am trying to install vtk for python 2.7 for Windows 64 bit PC
C:\Users\Abhi>conda install vtk
Collecting package metadata (current_repodata.json): done
Solving environment: done
# All requested packages already installed.
Where as
C:\Users\Abhi>python
Python 2.7.15 (v2.7.15:ca079a3ea3, Apr 30 2018, 16:30:26) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import vtk
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named vtk
>>>
for pip also I've tried
C:\Users\Abhi>pip install vtk
Collecting vtk
ERROR: Could not find a version that satisfies the requirement vtk
(from versions: none) ERROR: No matching distribution found for vtk
I've tried installing from executable file https://vtk.org/download/
Any help would be appreciated
Thanks
Abhishek
It sounds like you may be installing VTK for a python environment, but then using another one when you try importing it. It's important to keep track of which python you are using when performing an install.
The way I keep things organized is to create conda environments. If I were you, I would do the following:
Create a conda environment (you can specify any version of python you want)
conda create -n myenv python=3.4
Now, you've created an environment in which you can install packages for your own python project. You can create other environments with different packages and versions for other python projects using this command. Next, make sure to activate your conda environment (you must do this every time you start a new session)
conda activate myenv
Now, after activating your environment, make sure that your environment is indeed active by running
conda info --envs
The active environment will have a * next to it. Make sure that the * is next to "myenv", not base.
After making sure you are indeed in your active environment, install all packages you need (including VTK). Here's how to install vtk:
conda install -c anaconda vtk
After you're done installing packages, you can make sure they are installed in your environment by running
conda list
Scroll down until you see your package.
Now, you should be able to run python scripts with the packages you installed. Test things out by running your import vtk line and let us know if things worked.

ModuleNotFoundError: No module named 'gensim'

My goal is to import gensim in Python 3 on Windows.
I am using Python 3.7.2 (checked by running python -V in Windows command prompt). I installed gensim by running pip install gensim. I checked the installation by running pip freeze, and saw the line gensim==3.7.3.
Then, I ran the command py to enter the interactive python mode (still in Windows command prompt). I ran the line import gensim and got the following output:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'gensim'
I also tried from gensim import test and got the following output:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'gensim'
Any suggestions? How do I install gensim on Windows with Python 3? How do I test gensim?
In Mac, open anaconda navigator, Click on 'Open Terminal option'. If you are using Windows, Run anaconda prompt as administrator and run the following command:
conda install -c conda-forge gensim
I think you have installed it using normal cmd, so it may have installed it on python2.x. Install it with anaconda prompt.
Let me know if it worked for you.
I got the same error after installing gensim in Anaconda. It worked only after I re-started the Anaconda: by exiting it, and re-opening it via the command prompt. I wanted to share this experience since someone else may meet the same issue.
To understand why this happens, you must know how Windows finds executables to run, and how the Python software is installed.
When running a command, Windows searches for an executable in the environment variable PATH. It executes the first one found.
python.exe is installed in <PYTHON_INSTALL_DIR> (e.g. C:\Python\3.7).
pip.exe and other Python tools (e.g. pylint, virtualenv, pycrust, etc.) are installed in <PYTHON_INSTALL_DIR>\Scripts.
py.exe is installed in your Windows system directory (e.g. C:\Windows).
python and pip commands use the modules found in the directory their installed in, they do not look at PATH.
So, let's say you have the following Python versions:
C:\Python\2.7
C:\Python\3.6
C:\Python\3.7
and your PATH environment contains the following directories:
C:\Python\2.7
C:\Python\3.6\Scripts
then, see the following output:
C:\>python -V
Python 2.7.16
C:\>pip -V
pip 19.1.1 from c:\python\3.6\lib\site-packages\pip (python 3.6)
C:\>py -V
Python 3.7.3
So, when running pip, it is possible that the packages are installed in another Python version then the version you'll get when running python.
To see which versions are (correctly) installed on your system, run py -0p. Example output:
C:\>py -0p
Installed Pythons found by py Launcher for Windows
-3.7-64 C:\Python\3.7-64\python.exe *
-3.7-32 C:\Python\3.7-32\python.exe
-3.6-64 C:\Python\3.6-64\python.exe
-2.7-64 C:\Python\2.7-64\python.exe
-2.7-32 C:\Python\2.7-32\python.exe
General solution (for Windows)
The best thing is not to rely on your system PATH. Use the py launcher to select the version you want. To run the pip module corresponding to the Python version you want to use, start pip as a module instead of executable.
So instead of:
pip install <package>
run:
py -3.6 -m pip install <package>
Here's My hypothesis towards your situation, since your OS is able to recognize both python and py commands in commandline, this may mean that you have two separate versions of python installed.
Since, as you mentioned that python -V shows gensim as an installed module. Try opening python interactive interpreter via command python instead of py, and import gensim module in it.
C:\Users> Python
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 21:26:53) [MSC v.1916 32 bit (Intel)]
on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import gensim
see if that works.
P.S.
I would not recommend having two different compiler versions on a single OS as it creates a lot of commotion, and create incompatibility issues with program's made on one compiler with the other. And makes problems (like you mentioned) a lot more prevalent.
Most probably you have > 1 python installed in your machine. To install gensim (or any package) inside python command line, you can run below:
type "python" then enter
type "import subprocess" then enter
type
"subprocess.check_call(["python", '-m', 'pip', 'install', 'gensim'])"
then enter
Sample below:
>>> import subprocess
>>> subprocess.check_call(["python", '-m', 'pip', 'install', 'gensim'])
Collecting gensim
..
...
Installing collected packages: smart-open, gensim
Successfully installed gensim-3.7.3 smart-open-1.8.3
0

Python Modules installing properly but dont exist in editors

I want to start with I am new to the python language and am learning on my own. My issue is that I am installing the modules that I'm trying to use. The installs are successful but do not exist when I try to import them. I am using Visual Studio Code, Python 3.6, and Anaconda on Windows 10.
I am trying to use pyttsx3, SpeechRecognition and pocketsphinx.
$pip list
This gives me a very large list but does not contain either of my installed modules.
$python -m pip list
Gives me smaller output with them.
Package Version
----------------- ---------
beautifulsoup4 4.6.0
bs4 0.0.1
certifi 2018.4.16
chardet 3.0.4
click 6.7
gTTS 2.0.0
gTTS-token 1.1.1
idna 2.6
pip 10.0.1
pocketsphinx 0.1.3 #Here
pypiwin32 223
pyttsx 1.1
pyttsx3 2.7 #Here
pywin32 223
requests 2.18.4
setuptools 39.1.0
six 1.11.0
SpeechRecognition 3.8.1 #Here
urllib3 1.22
wheel 0.31.0
I know their installed but when I try to use any of the imports.
$python
Python 3.6.5 (v3.6.5:f59c0932b4, Mar 28 2018, 16:07:46) [MSC v.1900 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyttsx3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pyttsx3'
I am fine with this being something small on my part due to my lack of knowledge with this language, but I have done a full days worth of research trying to solve this myself.
Edit:
$pip --version
pip 9.0.1 from C:\Users\Keiro\Anaconda3\lib\site-packages (python 3.6)
You have two versions of pip perhaps two different installations of python as well. One with Anaconda and other one by main site.
Stick to one version and if you install in that use that environment in Visual Studio(Or VS Code)
You have installed those in anaconda which the standard installtion of Python wont recognize..
Open Python prompt from Anaconda Navigator >> Choose the environment (Select base for default) >> Open with Terminal [Or Open with Python]
>>python
>>import pyttsx3
In Visual Studio code set the same environment
Ctrl+Shift+P >> Select Python interpreter
Choose the anaconda location.
Volla. Now it should work
I had the same problem and found the answer in an old forum.
Write "pip3" instead of "pip".
So the code is:
pip3 install pyttsx3
(Source: bash: pip: command not found)

python miniconda install sklearn but not found it

To install sklearn, I installed miniconda in my win10 computer, and in the Anaconda command line, I input:
(C:\Python\Miniconda3) C:\Users\apple>conda install sklearn
and it shows that install successfully
(C:\Python\Miniconda3) C:\Users\apple>conda install scikit-learn
Fetching package metadata ...........
Solving package specifications: .
# All requested packages already installed.
# packages in environment at C:\Python\Miniconda3:
#
scikit-learn 0.18.1 np113py36_1
There are python 2.7 and python 3.6 in my computer, and the default environment is python 3.6. However, I found that there is no sklearn in my python 3.6, when I input this order in python 3.6 command line:
Python 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 08:06:12) [MSC v.1900 64
bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import sklearn
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
import sklearn
ModuleNotFoundError: No module named 'sklearn'
and I input the same order in python 2.7 command line, there is also:
>>> import sklearn
Traceback (most recent call last):
File "<pyshell#2>", line 1, in <module>
import sklearn
ImportError: No module named sklearn
however in conda command line, it shows the sklearn has been installed successfully.
Could you please tell me the reason and solve it?
NOTE: It is normally inadvisable to have multiple installations of Python; use virtual environments instead if you require the use of different Python versions.
First, check which python version are you using in the command prompt.
python --verison
It should display Miniconda somewhere in your version information. If not, then it would mean that your python is executing in a different path. You would have to set the PATH variable to point to the location where the Miniconda executables are stored.
Explanation:
Usually, when a fresh installation takes place in Windows, the path for the executables is appended to the PATH variable. Using conda would use the Miniconda (or Anaconda) executables because the normal Python installation would not contain it. If you have multiple installations, your python command would point to the executables of the distribution initially installed and not Miniconda (where you have successfully installed sklearn).
You would have to go to the Environment Variables settings to change the order of the installation.

Python 2.7.9: No module named requests

I am writing a code in Python 2.7.9 for which I need the requests module. I installed the module using sudo pip install requests but still in python 2.7.9 I am getting an error as follows:
Python 2.7.9 (default, Jan 5 2016, 18:47:14)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named requests
I checked the installation location /usr/local/lib/python2.7/dist-packages, it contains the requests package directory. The same thing works fine in python3, so I am guessing it's an installation error. How can I fix it ?
EDIT:
On executing pip lists, I could see requests (2.1.9) in the list. After I ran pip uninstall requests it shows requests (2.2.1) in pip list.
Based on the comments, it seems you have installed python 2.7.9 using a method Ubuntu doesn't like. Because of this the dist-packages folder is not added to your sys.path. You could set the PYTHONPATH variable in your .bashrc (or other zshrc, etc) to add that folder to your sys.path by default.
Better method would be to use a library like pyenv (It handles all dependency issues flawlessly for multiple python versions) or a better supported ppa for the latest python where this problem shouldn't arise at all.
Also, you have 2 versions of requests. This seems to be because one is installed using apt-get (sudo apt-get install python-requests) and the other is from pip (sudo pip install requests). It would be good to remove one of them to avoid confusion.

Categories