I am trying to run a python-script on an aws ec2-instance using jenkins.
I get the following error:
[ProdTest] $ /bin/sh -xe /tmp/jenkins14047325752732522807.sh
+ python3 prod.py
Traceback (most recent call last):
File "prod.py", line 2, in <module>
import base
File "/home/jenkins-slave-strat/workspace/ProdTest/base.py", line 3, in <module>
import boto3
ModuleNotFoundError: No module named 'boto3'
Build step 'Execute shell' marked build as failure
Finished: FAILURE
on the ec2 instance:
$ python3 --version
Python 3.7.9
$ pip3 install boto3 --user
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
Requirement already satisfied: boto3 in /usr/local/lib/python3.7/site-packages (1.16.25)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /usr/local/lib/python3.7/site-packages (from boto3) (0.10.0)
Requirement already satisfied: s3transfer<0.4.0,>=0.3.0 in /usr/local/lib/python3.7/site-packages (from boto3) (0.3.3)
Requirement already satisfied: botocore<1.20.0,>=1.19.25 in /usr/local/lib/python3.7/site-packages (from boto3) (1.19.25)
Requirement already satisfied: urllib3<1.27,>=1.25.4; python_version != "3.4" in /usr/local/lib/python3.7/site-packages (from botocore<1.20.0,>=1.19.25->boto3) (1.26.2)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /usr/local/lib/python3.7/site-packages (from botocore<1.20.0,>=1.19.25->boto3) (2.8.1)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.20.0,>=1.19.25->boto3) (1.15.0)
How do i get python to access the packages in /usr/local/lib/python3.7/site-packages ?
Thanks to everyone taking the time!
Before I give a short answer, a few cautions:
I don't know what version of Linux (I assume) the EC2 instance is running.
As a result, it's unclear if Python 2 is installed, but since you used pip3 and not just pip, it seems like it might be.
You could try altering your $PATH variable, but it's generally considered a good practice to use virtual environments for Python. I'm personally a fan of Conda, but you can find guides for Pipenv as well. My advice is; install Conda via Miniconda, then do the following:
$ conda create --name myAppEnv python=your.python.version
$ conda activate myAppEnv
$ conda install [your libraries]
Related
I have a Python tox project where I run the tox for running the test case and I came across one error a few hours back and am unable to resolve it till now. My module is using boto3 library and is installed using both the commands:
pip3 install boto3
pip install boto3 //for venv environments
When I try to install it again it gives me the below stack trace:
Requirement already satisfied: boto3 in ./venv/lib/python3.8/site-packages (1.26.34)
Requirement already satisfied: botocore<1.30.0,>=1.29.34 in ./venv/lib/python3.8/site-packages (from boto3) (1.29.34)
Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in ./venv/lib/python3.8/site-packages (from boto3) (1.0.1)
Requirement already satisfied: s3transfer<0.7.0,>=0.6.0 in ./venv/lib/python3.8/site-packages (from boto3) (0.6.0)
Requirement already satisfied: urllib3<1.27,>=1.25.4 in ./venv/lib/python3.8/site-packages (from botocore<1.30.0,>=1.29.34->boto3) (1.26.13)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in ./venv/lib/python3.8/site-packages (from botocore<1.30.0,>=1.29.34->boto3) (2.8.2)
Requirement already satisfied: six>=1.5 in ./venv/lib/python3.8/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.30.0,>=1.29.34->boto3) (1.16.0)
But when I run tox it gives me the below error:
File "/Users/tony/IdeaProjects/abc/provisioner/.tox/py38/lib/python3.8/site-packages/api/lambda_handler.py", line 1, in <module>
import boto3
ModuleNotFoundError: No module named 'boto3'
Is there some path issue? I am using Python 3.8.10. I tried uninstalling and installing the packages but nothing changed.
Any help is much appreciated.
tox creates an isolated environment for building and testing. So, when you install a package somewhere, here boto3 with pip/pip3, maybe system-wide, tox ignores that - and that is a good thing, otherwise it would be impossible to have a clean test environment.
You need to provide the necessary dependencies to tox.
tox automatically installs a package, ie. when there is a setup.py or a pyproject.toml. You could also provide your dependencies directly via the deps configuration key:
[testenv]
commands = ...
deps = boto3
There is another way. You could "break" the test isolation by using the tox-current-env plugin, see https://pypi.org/project/tox-current-env/
Disclaimer
I am on of the tox maintainers.
I need help understanding whats wrong here.
I will try to explain this as best I can, but Im a bit new to PythonAnywhere. I get this error:
Traceback (most recent call last):
File "/home/HelliottChip/mysite/app/__init__.py", line 6, in <module>
from flask_migrate import Migrate
ModuleNotFoundError: No module named 'flask_migrate'
When trying to run my main.py file. But i have them all installed as shown in this code:
17:14 ~/.local/bin $ pip install Flask-SQLAlchemy email_validator flask-mail pyjwt flask-bootstrap flask-moment
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2
.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Looking in links: /usr/share/pip-wheels
Requirement already satisfied: Flask-SQLAlchemy in /usr/local/lib/python2.7/dist-packages (2.4.1)
Requirement already satisfied: email_validator in /home/HelliottChip/.local/lib/python2.7/site-packages (1.1.2)
Requirement already satisfied: flask-mail in /usr/local/lib/python2.7/dist-packages (0.9.1)
Requirement already satisfied: pyjwt in /usr/local/lib/python2.7/dist-packages (1.7.1)
Requirement already satisfied: flask-bootstrap in /usr/local/lib/python2.7/dist-packages (3.3.7.1)
Requirement already satisfied: flask-moment in /home/HelliottChip/.local/lib/python2.7/site-packages (0.11.0)
Requirement already satisfied: Flask>=0.10 in /usr/local/lib/python2.7/dist-packages (from Flask-SQLAlchemy) (1.1.1)
Requirement already satisfied: SQLAlchemy>=0.8.0 in /usr/local/lib/python2.7/dist-packages (from Flask-SQLAlchemy) (1.3.10)
Requirement already satisfied: idna>=2.0.0 in /usr/local/lib/python2.7/dist-packages (from email_validator) (2.8)
Requirement already satisfied: dnspython>=1.15.0 in /home/HelliottChip/.local/lib/python2.7/site-packages (from email_validator) (1.16.0)
Requirement already satisfied: blinker in /usr/local/lib/python2.7/dist-packages (from flask-mail) (1.4)
Requirement already satisfied: dominate in /usr/local/lib/python2.7/dist-packages (from flask-bootstrap) (2.4.0)
Requirement already satisfied: visitor in /usr/local/lib/python2.7/dist-packages (from flask-bootstrap) (0.1.3)
Requirement already satisfied: itsdangerous>=0.24 in /usr/local/lib/python2.7/dist-packages (from Flask>=0.10->Flask-SQLAlchemy) (1.1.0)
Requirement already satisfied: Jinja2>=2.10.1 in /usr/local/lib/python2.7/dist-packages (from Flask>=0.10->Flask-SQLAlchemy) (2.10.3)
Requirement already satisfied: click>=5.1 in /usr/local/lib/python2.7/dist-packages (from Flask>=0.10->Flask-SQLAlchemy) (7.0)
Requirement already satisfied: Werkzeug>=0.15 in /usr/local/lib/python2.7/dist-packages (from Flask>=0.10->Flask-SQLAlchemy) (0.16.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/lib/python2.7/dist-packages (from Jinja2>=2.10.1->Flask>=0.10->Flask-SQLAlchemy) (0.23)
I have gone down about every website I can find on how to fix this, but I'm lost. This is for a Flask app I have worked on for a while on Replit.com and it worked fine until the transfer to PythonAnywhere. I have been trying to get this out for around two weeks now ^~^ but yet still no luck to my avail. I used a lot of this guide to help me with it, as I am also practicing my web deployment skills. (Don't need judgment on the fact I'm using flask -3-) But could anyone take thier time to help me with this? If you can, thanks a lot.
Check if your web app on PythonAnywhere is set up to be run by Python 2.7 -- I see you're using pip install outside of a virtual environment which by default will use pip for Python 2.7 as you can see in the output (btw. it won't work on PythonAnywhere, you need to provide --user option). If you want to install packages outside of virtual environment for different version of Python, use pipX.X install --user ... (where X.X should be replaced by required Python version). Also, remember to reload the web app every time the setup is changed. And if you're really stuck, maybe try contacting PythonAnywhere support (support#pythonanywhere.com) or use their forums?
Hello #TutorialWarrior9776 my suggestion is:
Step 1:
Install virtualenv,
pip install virtualenv
Step 2:
Create env
virtualenv -p python3.8 NAME_ENV
Step 3:
Activate env and Install requirements.txt (file generated local machine contains module flask_migrate)
source NAME_ENV/bin/active
pip install -r requirements.txt
Step 4:
Set your env path in the section (web) of pythonanywhere, that's all;
https://help.pythonanywhere.com/pages/Virtualenvs/
(Section: Step 3: Configure your app to use this virtualenv)
Hope this helps ;)
I can't seem to import panda package. I use Visual Studio code to code. I use a mac and have osX 10.14 Majove.
The code that i am trying to compile is :
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
house_data = pd.read_csv('house.csv')
plt.plot(house_data['surface'], house_data['loyer'], 'ro', markersize=4)
plt.show()
When I try to use pip install pandas i get on my terminal :
(base) pip install pandas
Requirement already satisfied: pandas in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (0.24.0)
Requirement already satisfied: pytz>=2011k in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from pandas) (2018.9)
Requirement already satisfied: python-dateutil>=2.5.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from pandas) (2.7.5)
Requirement already satisfied: numpy>=1.12.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from pandas) (1.15.3)
Requirement already satisfied: six>=1.5 in /Users/Library/Python/3.7/lib/python/site-packages (from python-dateutil>=2.5.0->pandas) (1.12.0)
(base) Thibaults-MBP-5d47:ML_folder thibaultmonsel$
Then when i execute my code i get :
Traceback (most recent call last):
File "ML1.py", line 5, in <module>
import pandas as pd
ImportError: No module named pandas
After if i try sudo pip install pandas i get :
(base) MBP-5d47:ML_folder $ sudo pip3 install pandas --upgrade
Password:
The directory '/Users/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory.If executing pip with sudo, you may want sudo's -H flag.
The directory '/Users/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting pandas
Downloading https://files.pythonhosted.org/packages/34/63/529fd1391044051514f2f22d61754245db2133cd37c4dad7150a1cbe2ece/pandas-0.24.1-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (15.9MB)
100% |████████████████████████████████| 15.9MB 901kB/s
Requirement already satisfied, skipping upgrade: python-dateutil>=2.5.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from pandas) (2.7.5)
Requirement already satisfied, skipping upgrade: numpy>=1.12.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from pandas) (1.15.3)
Requirement already satisfied, skipping upgrade: pytz>=2011k in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from pandas) (2018.9)
Requirement already satisfied, skipping upgrade: six>=1.5 in /Users/Library/Python/3.7/lib/python/site-packages (from python-dateutil>=2.5.0->pandas) (1.12.0)
Installing collected packages: pandas
Found existing installation: pandas 0.24.0
Uninstalling pandas-0.24.0:
Successfully uninstalled pandas-0.24.0
Successfully installed pandas-0.24.1
However, i still get no modules named pandas
Lastly, when i try pip3 install pandas i get :
Requirement already satisfied: pandas in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (0.24.0)
Requirement already satisfied: pytz>=2011k in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from pandas) (2018.9)
Requirement already satisfied: numpy>=1.12.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from pandas) (1.15.3)
Requirement already satisfied: python-dateutil>=2.5.0 in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (from pandas) (2.7.5)
Requirement already satisfied: six>=1.5 in /Users/Library/Python/3.7/lib/python/site-packages (from python-dateutil>=2.5.0->pandas) (1.12.0)
When i try to execute the program i get the same error mentioned above after using pip3 install pandas....
I also did an import.sys if can help :
base)-MBP-5d47:ML_folder $ python help1.py
2.7.10 (default, Aug 17 2018, 17:41:52)
[GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.0.42)]
Here is also my sys.path :
['/Users/Desktop/ML_folder', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', '/Library/Python/2.7/site-packages', '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python', '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC']
You need to install pandas with:
pip install pandas
If you run into issues with privileges, you may need to run:
sudo pip install pandas
It is also possible on Python 3 that you may need to run:
pip3 install pandas (although pip may be pointing to pip3 already). You can read about differences between pip versions on this SO post.
If you don't have pip installed, see here for installation.
Check pandas package path from your env with:
jupyter kernelspec list
If you see the path:
/Users/yourname/Library/Jupyter/kernels/yourenv
Delete that Jupyter folder from Library and run again.
if you see such this in your IDE and the error "no module named pandas" when you run your code, it means that pandas has not been installed although you have done "pip install pandas" or whatever.
Go to file > settings > project interpreter and see if pandas is available in the list of packages. if not simply click + (plus), choose pandas and install it in your project environment .
see picture
then wait for you IDE update your project skeletons ... voila , the error disappears !
When entering the command to run your file, make sure you specify which version of python you're using. For example, instead of python filename.py, use python3 filename.py or python2 filename.py
your pandas is installed in python3 (3.7):
Requirement already satisfied: pandas in /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages (0.24.0)
but you are running python2.7 and pandas isn't in your path 2.7:
['/Users/thibaultmonsel/Desktop/ML_folder',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old',
'/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload',
'/Library/Python/2.7/site-packages',
'/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python',
'/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC']
try to simply run your script using python3:
python3 help1.py
or add python3 header, example:
#!/usr/bin/env python3
or
#!/usr/local/bin/python3
and if that doesn't work (like I had the same problem because I was importing pandas from jupyter notebook, macos), you can ultimately import from your --user path, example:
sys.path.append("/Users/<USER>/Library/Python/3.7/lib/python/site-packages")
but make sure you have pandas installed there (..python/site-packages/pandas) using
pip3 install pandas --user
Check your virtual environment (you can see it at the left corner of VS code) and install the package (e.g. pandas) in your virtual environment like this:
conda install -n yourenvname [package]
install pandas outside the project, I wanted to download it only for an env environment but I got the same error so I did it from outside.
Code > Preferences > Settings
In Search, type "interpreter"
You will see a bar : Python: Default Interpreter Path
Paste your correct path to Python (something like "/usr/local/bin/python3" in Mac), it will automatically save
Then go back to your python file and try to run
For me, the below command works in MAC
sudo -H pip3 install pandas --upgrade
I tested a python script to send anythink with Pushover. But I get the error "ImportError: No module named pushover"
My installed Versions:
# pip install python-pushover
Collecting python-pushover
Using cached https://files.pythonhosted.org/packages/6f/3d/144a0137c749bd152c3ab7f4d3ce8fe1455168dab36c2fcd900d3fab16ad/python-pushover-0.4.tar.gz
Requirement already satisfied: requests>=1.0 in /usr/local/lib/python3.5/dist-packages (from python-pushover) (2.21.0)
Requirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/local/lib/python3.5/dist-packages (from requests>=1.0->python-pushover) (1.24.1)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.5/dist-packages (from requests>=1.0->python-pushover) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.5/dist-packages (from requests>=1.0->python-pushover) (2018.11.29)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.5/dist-packages (from requests>=1.0->python-pushover) (2.8)
Installing collected packages: python-pushover
Running setup.py install for python-pushover ... done
Successfully installed python-pushover-0.4
# python -V
Python 2.7.13
# python3 -V
Python 3.5.3
The scriptheader:
#!/usr/bin/env python
import pushover
I have tried with pip(3) to install python-pushover but with no success.
I faced this exact error today. This is due to co-existing of different versions of python in your system.
Do, /usr/bin/python3 if the module was installed for py3 and just /usr/bin/python for py2 before running the script.
Refer Installed module using pip, not found for more.
In my case, you should :
uninstall that package:
pip uninstall pushover
and install the correct package:
pip install python-pushover
And your code will work fine.
I have only very rudimentary experience in Python. I am trying to install the package pyslim (see here on the pypi website). I did
$ pip install pyslim
Requirement already satisfied: pyslim in ./Library/Python/2.7/lib/python/site-packages/pyslim-0.1-py2.7.egg (0.1)
Requirement already satisfied: msprime in /usr/local/lib/python2.7/site-packages (from pyslim) (0.6.1)
Requirement already satisfied: attrs in /usr/local/lib/python2.7/site-packages (from pyslim) (16.3.0)
Requirement already satisfied: svgwrite in /usr/local/lib/python2.7/site-packages (from msprime->pyslim) (1.1.12)
Requirement already satisfied: jsonschema in /usr/local/lib/python2.7/site-packages (from msprime->pyslim) (2.6.0)
Requirement already satisfied: six in /usr/local/lib/python2.7/site-packages (from msprime->pyslim) (1.10.0)
Requirement already satisfied: numpy>=1.7.0 in /usr/local/lib/python2.7/site-packages/numpy-1.10.4-py2.7-macosx-10.11-x86_64.egg (from msprime->pyslim) (1.10.4)
Requirement already satisfied: h5py in /usr/local/lib/python2.7/site-packages (from msprime->pyslim) (2.8.0)
Requirement already satisfied: pyparsing>=2.0.1 in /usr/local/lib/python2.7/site-packages (from svgwrite->msprime->pyslim) (2.2.0)
Requirement already satisfied: functools32; python_version == "2.7" in /usr/local/lib/python2.7/site-packages (from jsonschema->msprime->pyslim) (3.2.3.post2)
But when I open python and try to import pyslim, it fails
> import pyslim
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/remi/Library/Python/2.7/lib/python/site-packages/pyslim-0.1-py2.7.egg/pyslim/__init__.py", line 4, in <module>
from pyslim.slim_metadata import * # NOQA
File "/Users/remi/Library/Python/2.7/lib/python/site-packages/pyslim-0.1-py2.7.egg/pyslim/slim_metadata.py", line 1, in <module>
import attr
ImportError: No module named attr
So, I did
$ pip install attr
Requirement already satisfied: attr in /usr/local/lib/python2.7/site-packages (0.3.1)
and
$ pip install attrs
Requirement already satisfied: attrs in /usr/local/lib/python2.7/site-packages (16.3.0)
I restarted python and tried to import pyslim again but I keep receiving the same error message. I also tried to download and install the files from github by doing
$ git clone https://github.com/tskit-dev/pyslim.git
$ cd pyslim
$ python setup.py install --user
as indicated here on the pypi website. On this last line of code, I get a long output ending with
Download error on https://pypi.python.org/simple/: [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590) -- Some packages may not be found!
No local packages or download links found for attrs
error: Could not find suitable distribution for Requirement.parse('attrs')
I am using Python 2.7.10 on a MAC OSX 10.11.6. Not sure if it matter but I usually install things with homebrew. I am using pip 18.1 from /usr/local/lib/python2.7/site-packages/pip (python 2.7).
Edit
$ which python
/usr/bin/python
$ which pip
/usr/local/bin/pip
Take a look to this other question that is related to the attrs package.
In your case, you have attr and attrs installed at the same time, and they are incompatible between them, so python is unable to resolve the package name on the import correctly.
You should use attrs only, so try uninstalling attr and try it again:
python -m pip uninstall attr
If, in the future, you need to have some packages incompatibles between them, take a look about using virtual environments in Python, which are really easy to use and will be very useful to play with packages and packages versions without break anything.
First uninstall pyslim. Use "pip uninstall pyslim". Then try installing again using
"conda install -c conda-forge pyslim"
Refer https://anaconda.org/conda-forge/pyslim