I want to use Jug for parallel processing. I have a Canopy installed and I also installed Jug using command pip install jug according to the documentation online.
In order to find where jug is installed, I installed jug again using the same command as above, it showed
me:
Requirement already satisfied (use --upgrade to upgrade): jug in
c:\users[userfolder]\appdata\local\enthought\canopy\user\lib\site-packages
(from jug)
Requirement already satisfied (use --upgrade to upgrade): six in
c:\users[userfolder]\appdata\local\enthought\canopy\user\lib\site-packages
(from jug)
Requirement already satisfied (use --upgrade to upgrade): redis in
c:\users[userfolder]\appdata\local\enthought\canopy\user\lib\site-packages
(from jug)
Now, I thought my jug is in the path of c:\users\[userfolder]\appdata\local\enthought\canopy\user\lib\site-package and it is there since I listed all files under this folder and I saw it.
I am not sure this jug is a exe or py or something else, but I tried to run a command: jug C:\primes.py under this folder, it gave me error message said jug is not a recognized as the name of cmdlet, function, script file....
I also tried the command ./jug C:\primes.py and .\jug C:\primes.py, but none of them works.
In addition, I tried python jug status C:\primes.py and this one gave me message of cannot find '_main_' module in 'jug'.
Now I have no idea how to run jug. Has someone ever tried jug on windows could help me with it?
I figured it out myself. It is related to python's relative import.
Related
I have been through the various Stack Overflow questions and non of the answers resolve the issue for me.
It is likely a path 'thing' that I am not seeing.
Have installed pandas, it says its there, but numpy will import and pandas won't.
Any ideas? What am I missing?
If I enter:
!pip3 install pandas
I get:
Requirement already satisfied: pandas in c:\users\ron mcmillan\appdata\local\programs\python\python38\lib\site-packages (1.0.3)
Requirement already satisfied: pytz>=2017.2 in c:\users\ron mcmillan\appdata\local\programs\python\python38\lib\site-packages (from pandas) (2020.1)
Requirement already satisfied: numpy>=1.13.3 in c:\users\ron mcmillan\appdata\local\programs\python\python38\lib\site-packages (from pandas) (1.18.4)
Requirement already satisfied: python-dateutil>=2.6.1 in c:\users\ron mcmillan\appdata\local\programs\python\python38\lib\site-packages (from pandas) (2.8.1)
Requirement already satisfied: six>=1.5 in c:\users\ron mcmillan\appdata\local\programs\python\python38\lib\site-packages (from python-dateutil>=2.6.1->pandas) (1.14.0)
Okay, I'm going to give a completely new, opinionated, answer.
It looks like you've got at least two installations of Python, one in ..python\python38, and one in ..\python\python38-32. pandas is installed on the first, but jupyter is running on the second. You've got a version of pip for each, plus a third one in Microsoft\WindowsApps.
This happens.
On Windows, you are probably safe to just uninstall all the versions of Python on your computer. Then download Miniconda for Windows. Miniconda is, like pip, a package manager for Python; but unlike pip, is able to also install non-Python files, which makes all sorts of things much easier. It's called 'Miniconda' to distinguish it from 'Anaconda', which is conda plus almost all the packages a data scientist might need -- Miniconda just gives you the package manager conda, and you can then choose to install what you actually want.
Once conda is installed, create a new conda environment (conda create -n MYNEWENV -- if you have twenty minutes then read the quickstart explanation) and then try to download everything with conda install...
You can specify which 'channel' you download packages from, but you probably don't need to. Just try conda install jupyter numpy pandas; or search the Conda cloud for the specifics of whatever you want.
Make sure that the python installation which is running jupyter is the one that has pandas. Run where jupyter (which on linux/mac), and make sure it matches the paths above. Also try where pip to make sure the installations of pip, python, and jupyter match.
To be sure you're using a specific version of python+pip to install a package, you can call pip the module rather than the script, e.g. with c:\users\ron mcmillan\appdata\local\programs\python\python38\python -m pip install [pkg]
I'd consider using an environment manager such as virtualenv or conda to simplify environments & installation.
Try this:
# Install a pip package in the current Jupyter kernel
import sys
!{sys.executable} -m pip install pandas
There is a lengthy explanation here.
Going through the lengthy explanation above - will need to go through it again.
import sys
!{sys.executable} -m pip install numpy
returns:
'c:\users\ron' is not recognized as an internal or external command,
operable program or batch file.
import sys
sys.path
returns:
['C:\\Users\\Ron McMillan',
'c:\\users\\ron mcmillan\\appdata\\local\\programs\\python\\python38-
32\\python38.zip',
'c:\\users\\ron mcmillan\\appdata\\local\\programs\\python\\python38-32\\DLLs',
'c:\\users\\ron mcmillan\\appdata\\local\\programs\\python\\python38-32\\lib',
'c:\\users\\ron mcmillan\\appdata\\local\\programs\\python\\python38-32',
'',
'c:\\users\\ron mcmillan\\appdata\\local\\programs\\python\\python38-32\\lib\\site-packages',
'c:\\users\\ron mcmillan\\appdata\\local\\programs\\python\\python38-32\\lib\\site-packages\\win32',
'c:\\users\\ron mcmillan\\appdata\\local\\programs\\python\\python38-32\\lib\\site-packages\\win32\\lib',
'c:\\users\\ron mcmillan\\appdata\\local\\programs\\python\\python38-32\\lib\\site-packages\\Pythonwin',
'c:\\users\\ron mcmillan\\appdata\\local\\programs\\python\\python38-32\\lib\\site-packages\\IPython\\extensions',
'C:\\Users\\Ron McMillan\\.ipython']
!type python
returns
The system cannot find the file specified.
And in the Command Prompt:
where jupyter
C:\users\ron mcmillan\appdata\local\programs\python\python38-32\Scripts\jupyter.exe
and
where pip
C:\users\ron mcmillan\appdata\local\programs\python\python38\Scripts\pip.exe
C:\users\ron mcmillan\appdata\local\programs\python\python38-32\Scripts\pip.exe
C:\users\ron mcmillan\appdata\local\Microsoft\WindowsApps\pip.exe
So, Jupyter note finding Python?
Ok, thank you.
Yes, I knew it was messy - to the point I din't get it. The picture makes sense.
Using Python 3.8 on Windows, having installed a number of other modules I have tried to install gastrodon with
(property) C:\Users\andyt>pip install gastrodon
The result is this:
Requirement already satisfied: gastrodon in c:\users\andyt\anaconda3\envs\property\lib\site-packages (0.9.3)
Requirement already satisfied: pandas in c:\users\andyt\anaconda3\envs\property\lib\site-packages (from gastrodon) (1.0.3)
Requirement already satisfied: IPython in c:\users\andyt\anaconda3\envs\property\lib\site-packages (from gastrodon) (7.13.0)
WARNING: No metadata found in c:\users\andyt\anaconda3\envs\property\lib\site-packages
ERROR: Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory: 'c:\\users\\andyt\\anaconda3\\envs\\property\\lib\\site-packages\\ipython-7.13.0.dist-info\\METADATA'
Does anyone know how to fix this? I am able to install it in base...
Have you tried turning it on and off again? I mean potentially uninstall and reinstall it. I had a similar problem downloading numpy a while back and that seemed to fix it.
so I renamed metadata.json to metadata in a few similar positions where the install failed sequentially, and it appeared to have been successful, except I am now dealing with another error when trying to import gastrodon...
I've written something in TensorFlow that makes use of some nice group theory functions that work very easily in Sage (and seem prohibitively difficult to code from scratch). The Sage part works on its own, and the TensorFlow part works on its own, but I can't figure out how to get them working together.
Specifically: I can make a file test.py using Sage functions and run it from the command line using:
sage --python test.py
with no problem. But calling a function defined in test.py from a .py file using TensorFlow fails ("Import error, no module named Sage"), presumably because Sage (6.x) uses Python 2.6.x, while TensorFlow uses Python 2.7 or 3.3+.
Is there a way around this?
Thanks!
EDIT: I'm not sure if this is relevant, but if I fire up normal Python (the kind TensorFlow uses), I get this:
from sage.env import SAGE_LOCAL
SAGE_LOCAL
which outputs '$SAGE_ROOT/local'.
However if I fire up Sage first I get this:
sage
SAGE_LOCAL
which outputs ''/usr/lib/sagemath/local'.
I just upgraded to Sage 7.0 if that matters (this didn't work in 6.10 either, though).
Here's something NOT to do (yet); don't just take whatever Sage install you happen to have and do:
$ sage -pip install https://storage.googleapis.com/tensorflow/mac/tensorflow-0.7.1-cp27-none-any.whl
Even though this "works", it also had several worrying messages about upgrading numpy and six, which completely broke the numpy part of my Sage installation. This was with Sage-6.9.
Which means you have to make sure you have a Sage that has the right versions of Numpy and six. With the latest development version, we do, apparently:
$ cd /path/to/my/bleeding/edge/sage/directory
$ ./sage -pip install https://storage.googleapis.com/tensorflow/mac/tensorflow-0.7.1-cp27-none-any.whl
Collecting tensorflow==0.7.1 from https://storage.googleapis.com/tensorflow/mac/tensorflow-0.7.1-cp27-none-any.whl
Using cached https://storage.googleapis.com/tensorflow/mac/tensorflow-0.7.1-cp27-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): six>=1.10.0 in ./local/lib/python2.7/site-packages/six-1.10.0-py2.7.egg (from tensorflow==0.7.1)
Collecting protobuf==3.0.0b2 (from tensorflow==0.7.1)
Using cached protobuf-3.0.0b2-py2.py3-none-any.whl
Collecting wheel (from tensorflow==0.7.1)
Using cached wheel-0.29.0-py2.py3-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): numpy>=1.10.1 in ./local/lib/python2.7/site-packages (from tensorflow==0.7.1)
Requirement already satisfied (use --upgrade to upgrade): setuptools in ./local/lib/python2.7/site-packages/setuptools-20.1.1-py2.7.egg (from protobuf==3.0.0b2->tensorflow==0.7.1)
Installing collected packages: protobuf, wheel, tensorflow
Successfully installed protobuf-3.0.0b2 tensorflow-0.7.1 wheel-0.29.0
You are using pip version 8.0.2, however version 8.1.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
And then I don't get any failures.
So one has to be careful, but apparently it's possible. However, you definitely have to use it from "within" Sage; Sage-as-distribution wouldn't allow you to use your other tensorflow with it.
I'm trying to run the Flask-Bootstrap sample application from within Spyder, but I'm not seeing anything when I hit run. It sits for a long time, but I'm pretty sure it's supposed to bring up a webpage, correct? I don't get any error message. It just never does anything as far as I can tell.
When I run via python run_sample_app.py, I get an error message No module named flask_appconfig despite having installed Flask using conda install flask.
I tried installing via pip install flask-appconfig, but I have apparently already installed:
Requirement already satisfied (use --upgrade to upgrade): flask-appconfig in /Applications/Anaconda/anaconda/lib/python2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): flask in /Applications/Anaconda/anaconda/lib/python2.7/site-packages (from flask-appconfig)
Requirement already satisfied (use --upgrade to upgrade): six in /Applications/Anaconda/anaconda/lib/python2.7/site-packages (from flask-appconfig)
Cleaning up...
Any ideas?
Thanks for the help, guys! I was just being dumb. I thought surely the application should pull up a browser for me, but I was wrong. It just posts the HTML/CSS to my localhost (127.0.0.1), and I didn't realize that I had to navigate there on my own. Everything is working now. Thanks again!
I'm trying to install a module called Scrapy. I installed it using
pip install Scrapy
I see the 'scrapy' folder in my /usr/local/lib/python2.7/site-packages, but when I try to import it in a Python program, is says there is no module by that name. Any ideas as to why this might be happening?
EDIT: Here is the output of the pip command:
Downloading/unpacking Scrapy
Downloading Scrapy-0.20.0.tar.gz (745kB): 745kB downloaded
Running setup.py egg_info for package Scrapy
no previously-included directories found matching 'docs/build'
Requirement already satisfied (use --upgrade to upgrade): Twisted>=10.0.0 in /usr/local/lib/python2.7/site-packages (from Scrapy)
Requirement already satisfied (use --upgrade to upgrade): w3lib>=1.2 in /usr/local/lib/python2.7/site-packages (from Scrapy)
Requirement already satisfied (use --upgrade to upgrade): queuelib in /usr/local/lib/python2.7/site-packages (from Scrapy)
Requirement already satisfied (use --upgrade to upgrade): lxml in /usr/local/lib/python2.7/site-packages (from Scrapy)
Requirement already satisfied (use --upgrade to upgrade): pyOpenSSL in /usr/local/lib/python2.7/site-packages (from Scrapy)
Requirement already satisfied (use --upgrade to upgrade): cssselect>=0.9 in /usr/local/lib/python2.7/site-packages (from Scrapy)
Requirement already satisfied (use --upgrade to upgrade): zope.interface>=3.6.0 in /usr/local/lib/python2.7/site-packages (from Twisted>=10.0.0->Scrapy)
Requirement already satisfied (use --upgrade to upgrade): six>=1.4.1 in /usr/local/lib/python2.7/site-packages (from w3lib>=1.2->Scrapy)
Requirement already satisfied (use --upgrade to upgrade): setuptools in /usr/local/lib/python2.7/site-packages/setuptools-1.1.6-py2.7.egg (from zope.interface>=3.6.0->Twisted>=10.0.0->Scrapy)
Installing collected packages: Scrapy
Running setup.py install for Scrapy
changing mode of build/scripts-2.7/scrapy from 644 to 755
no previously-included directories found matching 'docs/build'
changing mode of /usr/local/bin/scrapy to 755
Successfully installed Scrapy
Cleaning up...
When I run /usr/local/bin/scrapy I get the usage for the command and the available commands. I noticed that I have a python2.7 and python2.7-32 in my /usr/local/bin, and I remember installing the 32 bit version because of a problem with Mavericks.
Here is the output of python /usr/local/bin/scrapy:
Traceback (most recent call last): File "/usr/local/bin/scrapy", line 3, in <module> from scrapy.cmdline import execute ImportError: No module named scrapy.cmdline
And head /usr/local/bin/scrapy:
#!/usr/local/opt/python/bin/python2.7 from scrapy.cmdline import execute execute()
Are you using Homebrew or MacPorts or something? As #J.F.Sebastian said, it sounds like you are having issues mixing the default python that comes with OS X, and one that is installed via a package manager... Try /usr/local/opt/python/bin/python2.7 -m scrapy and see if that throws an ImportError.
If that works, then you may want to consider making that python executable your default. Something like alias python2.7=/usr/local/opt/python/bin/python2.7 and then always use python2.7 instead of the default python. You can likewise just point python to the /urs/local... bit, but then you won't have easy access to the system (OS X-supplied) python if you ever needed it for some reason.
EDIT: You can force pip to install to an alternate location. The details are here: Install a Python package into a different directory using pip?. If you do indeed have extra Python folders on your system, maybe you can try directing scrapy to those, even if just for a temporary solution.
Can you post the output of the pip command? Perhaps it is failing somewhere?
Also, is it possible you have two versions of Python on your machine? Pip only installs to one location, but perhaps the version of Python on your path is different.
Finally, sometimes package names given to pip are not exactly the same as the name used to import. Check the documentation of the package. I took a quick look and the import should be lowercase:
import scrapy
When all else fails you can always set the environment variable PYTHONPATH (see Permanently add a directory to PYTHONPATH for help) to the path where you installed Scrapy. (pending you're not using virtualenv -- and if you are please specify so we can help, it's generally a good idea to provide OS too)
if you run on Ubuntu:
use the official Ubuntu Packages, which already solve all dependencies for you and are continuously updated with the latest bug fixes.
Optionally, even if it solves your problem, it is always better to install python libraries on a virtual environment, using virtualenvwrapper to keep the libraries separated, try to examine the apt-get installation log to find out what tools where added, then remove scrapy python library and reinstall it in the virtual env. using pip
It appears that the scrapy module that is installed on the Python path is an executable file that will bootstrap a Scrapy project directory for you.
The Python code in the scrapy executable looks like this:
#!/usr/bin/env python
from scrapy.cmdline import execute
execute()
That's intended to be run from the command line rather than imported into your own Python project module.
According to the documentation for the project, running the scrapy executable with this syntax:
scrapy startproject <your-project-name>
will bootstrap a Scrapy project that has the following directory structure:
your-project-name/
scrapy.cfg
tutorial/
__init__.py
items.py
pipelines.py
settings.py
spiders/
__init__.py
...
There are a number of examples in the documentation that demonstrate how you create and run your own spiders, link extractors, etc., and how to manipulate the data that you retrieve with the application. They each demonstrate the appropriate Python imports from subdirectories in the scrapy package to get you up and running.
Hope that this helps.