I have a Python tox project where I run the tox for running the test case and I came across one error a few hours back and am unable to resolve it till now. My module is using boto3 library and is installed using both the commands:
pip3 install boto3
pip install boto3 //for venv environments
When I try to install it again it gives me the below stack trace:
Requirement already satisfied: boto3 in ./venv/lib/python3.8/site-packages (1.26.34)
Requirement already satisfied: botocore<1.30.0,>=1.29.34 in ./venv/lib/python3.8/site-packages (from boto3) (1.29.34)
Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in ./venv/lib/python3.8/site-packages (from boto3) (1.0.1)
Requirement already satisfied: s3transfer<0.7.0,>=0.6.0 in ./venv/lib/python3.8/site-packages (from boto3) (0.6.0)
Requirement already satisfied: urllib3<1.27,>=1.25.4 in ./venv/lib/python3.8/site-packages (from botocore<1.30.0,>=1.29.34->boto3) (1.26.13)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in ./venv/lib/python3.8/site-packages (from botocore<1.30.0,>=1.29.34->boto3) (2.8.2)
Requirement already satisfied: six>=1.5 in ./venv/lib/python3.8/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.30.0,>=1.29.34->boto3) (1.16.0)
But when I run tox it gives me the below error:
File "/Users/tony/IdeaProjects/abc/provisioner/.tox/py38/lib/python3.8/site-packages/api/lambda_handler.py", line 1, in <module>
import boto3
ModuleNotFoundError: No module named 'boto3'
Is there some path issue? I am using Python 3.8.10. I tried uninstalling and installing the packages but nothing changed.
Any help is much appreciated.
tox creates an isolated environment for building and testing. So, when you install a package somewhere, here boto3 with pip/pip3, maybe system-wide, tox ignores that - and that is a good thing, otherwise it would be impossible to have a clean test environment.
You need to provide the necessary dependencies to tox.
tox automatically installs a package, ie. when there is a setup.py or a pyproject.toml. You could also provide your dependencies directly via the deps configuration key:
[testenv]
commands = ...
deps = boto3
There is another way. You could "break" the test isolation by using the tox-current-env plugin, see https://pypi.org/project/tox-current-env/
Disclaimer
I am on of the tox maintainers.
Related
I want to install this library: https://github.com/stefanfoulis/django-phonenumber-field
I have done like this:
(venv) michael#michael:~/PycharmProjects/client$ pip install django-phonenumber-field[phonenumbers]
Requirement already satisfied: django-phonenumber-field[phonenumbers] in ./venv/lib/python3.8/site-packages (5.1.0)
Requirement already satisfied: Django>=2.2 in ./venv/lib/python3.8/site-packages (from django-phonenumber-field[phonenumbers]) (3.2.3)
Collecting phonenumbers>=7.0.2; extra == "phonenumbers"
Downloading phonenumbers-8.12.23-py2.py3-none-any.whl (2.6 MB)
|████████████████████████████████| 2.6 MB 2.8 MB/s
Requirement already satisfied: sqlparse>=0.2.2 in ./venv/lib/python3.8/site-packages (from Django>=2.2->django-phonenumber-field[phonenumbers]) (0.4.1)
Requirement already satisfied: asgiref<4,>=3.3.2 in ./venv/lib/python3.8/site-packages (from Django>=2.2->django-phonenumber-field[phonenumbers]) (3.3.4)
Requirement already satisfied: pytz in ./venv/lib/python3.8/site-packages (from Django>=2.2->django-phonenumber-field[phonenumbers]) (2021.1)
Installing collected packages: phonenumbers
Successfully installed phonenumbers-8.12.23
We can see that this library has installed its dependency: phonenumbers library.
But now I want to write in requirements.txt that I need django-phonenumber-field of a certain release. So that its dependency should be installed as appropriate.
In other words how to write requirements.txt for future pip install -r requirements.txt.
Could you help me here?
=====ADDED LATER=======
pip freeze > requirements.txt doesn't seem to be appropriate as it will contain all the dependencies. It will become overburden with dependencies. I will forget why a particular library is installed. Whether it is a dependency at all. If it is, what library depends on it.
I want to keep requirements.txt as clear as possible.
In this particular example I want just one directive: install django-phonenumber-field==5.1.0 and manage dependencies yourself.
Wrong way (bloated and unclear what to delete if I stop using the library):
django-phonenumber-field==5.1.0
phonenumbers==8.12.23
What I want:
django-phonenumber-field==5.1.0 [and all dependencies]
As #jonrsharpe said, you can use pip freeze which lists all your dependencies, and redirect the output to your requirements.txt file to update it.
pip freeze > requirements.txt
The simple way is to create a text file, type the shell commands in, save it and the rename it from .txt to .bat. Now you can double click the file and it installs the requirements. You could probably even execute the .bat
I need help understanding whats wrong here.
I will try to explain this as best I can, but Im a bit new to PythonAnywhere. I get this error:
Traceback (most recent call last):
File "/home/HelliottChip/mysite/app/__init__.py", line 6, in <module>
from flask_migrate import Migrate
ModuleNotFoundError: No module named 'flask_migrate'
When trying to run my main.py file. But i have them all installed as shown in this code:
17:14 ~/.local/bin $ pip install Flask-SQLAlchemy email_validator flask-mail pyjwt flask-bootstrap flask-moment
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2
.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Looking in links: /usr/share/pip-wheels
Requirement already satisfied: Flask-SQLAlchemy in /usr/local/lib/python2.7/dist-packages (2.4.1)
Requirement already satisfied: email_validator in /home/HelliottChip/.local/lib/python2.7/site-packages (1.1.2)
Requirement already satisfied: flask-mail in /usr/local/lib/python2.7/dist-packages (0.9.1)
Requirement already satisfied: pyjwt in /usr/local/lib/python2.7/dist-packages (1.7.1)
Requirement already satisfied: flask-bootstrap in /usr/local/lib/python2.7/dist-packages (3.3.7.1)
Requirement already satisfied: flask-moment in /home/HelliottChip/.local/lib/python2.7/site-packages (0.11.0)
Requirement already satisfied: Flask>=0.10 in /usr/local/lib/python2.7/dist-packages (from Flask-SQLAlchemy) (1.1.1)
Requirement already satisfied: SQLAlchemy>=0.8.0 in /usr/local/lib/python2.7/dist-packages (from Flask-SQLAlchemy) (1.3.10)
Requirement already satisfied: idna>=2.0.0 in /usr/local/lib/python2.7/dist-packages (from email_validator) (2.8)
Requirement already satisfied: dnspython>=1.15.0 in /home/HelliottChip/.local/lib/python2.7/site-packages (from email_validator) (1.16.0)
Requirement already satisfied: blinker in /usr/local/lib/python2.7/dist-packages (from flask-mail) (1.4)
Requirement already satisfied: dominate in /usr/local/lib/python2.7/dist-packages (from flask-bootstrap) (2.4.0)
Requirement already satisfied: visitor in /usr/local/lib/python2.7/dist-packages (from flask-bootstrap) (0.1.3)
Requirement already satisfied: itsdangerous>=0.24 in /usr/local/lib/python2.7/dist-packages (from Flask>=0.10->Flask-SQLAlchemy) (1.1.0)
Requirement already satisfied: Jinja2>=2.10.1 in /usr/local/lib/python2.7/dist-packages (from Flask>=0.10->Flask-SQLAlchemy) (2.10.3)
Requirement already satisfied: click>=5.1 in /usr/local/lib/python2.7/dist-packages (from Flask>=0.10->Flask-SQLAlchemy) (7.0)
Requirement already satisfied: Werkzeug>=0.15 in /usr/local/lib/python2.7/dist-packages (from Flask>=0.10->Flask-SQLAlchemy) (0.16.0)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/lib/python2.7/dist-packages (from Jinja2>=2.10.1->Flask>=0.10->Flask-SQLAlchemy) (0.23)
I have gone down about every website I can find on how to fix this, but I'm lost. This is for a Flask app I have worked on for a while on Replit.com and it worked fine until the transfer to PythonAnywhere. I have been trying to get this out for around two weeks now ^~^ but yet still no luck to my avail. I used a lot of this guide to help me with it, as I am also practicing my web deployment skills. (Don't need judgment on the fact I'm using flask -3-) But could anyone take thier time to help me with this? If you can, thanks a lot.
Check if your web app on PythonAnywhere is set up to be run by Python 2.7 -- I see you're using pip install outside of a virtual environment which by default will use pip for Python 2.7 as you can see in the output (btw. it won't work on PythonAnywhere, you need to provide --user option). If you want to install packages outside of virtual environment for different version of Python, use pipX.X install --user ... (where X.X should be replaced by required Python version). Also, remember to reload the web app every time the setup is changed. And if you're really stuck, maybe try contacting PythonAnywhere support (support#pythonanywhere.com) or use their forums?
Hello #TutorialWarrior9776 my suggestion is:
Step 1:
Install virtualenv,
pip install virtualenv
Step 2:
Create env
virtualenv -p python3.8 NAME_ENV
Step 3:
Activate env and Install requirements.txt (file generated local machine contains module flask_migrate)
source NAME_ENV/bin/active
pip install -r requirements.txt
Step 4:
Set your env path in the section (web) of pythonanywhere, that's all;
https://help.pythonanywhere.com/pages/Virtualenvs/
(Section: Step 3: Configure your app to use this virtualenv)
Hope this helps ;)
C:\Users\sbhandari>python --version
Python 3.9.1
C:\Users\sbhandari>pip install pytest
Collecting pytest
Using cached pytest-6.2.1-py3-none-any.whl (279 kB)
Requirement already satisfied: toml in c:\users\sbhandari\appdata\local\packages\pythonsoftwarefoundation.python.3.9_qbz5n2kfra8p0\localcache\local-packages\python39\site-packages (from pytest) (0.10.2)
Requirement already satisfied: pluggy<1.0.0a1,>=0.12 in c:\users\sbhandari\appdata\local\packages\pythonsoftwarefoundation.python.3.9_qbz5n2kfra8p0\localcache\local-packages\python39\site-packages (from pytest) (0.13.1)
Requirement already satisfied: iniconfig in c:\users\sbhandari\appdata\local\packages\pythonsoftwarefoundation.python.3.9_qbz5n2kfra8p0\localcache\local-packages\python39\site-packages (from pytest) (1.1.1)
Requirement already satisfied: py>=1.8.2 in c:\users\sbhandari\appdata\local\packages\pythonsoftwarefoundation.python.3.9_qbz5n2kfra8p0\localcache\local-packages\python39\site-packages (from pytest) (1.10.0)
Requirement already satisfied: packaging in c:\users\sbhandari\appdata\local\packages\pythonsoftwarefoundation.python.3.9_qbz5n2kfra8p0\localcache\local-packages\python39\site-packages (from pytest) (20.8)
Requirement already satisfied: attrs>=19.2.0 in c:\users\sbhandari\appdata\local\packages\pythonsoftwarefoundation.python.3.9_qbz5n2kfra8p0\localcache\local-packages\python39\site-packages (from pytest) (20.3.0)
Requirement already satisfied: atomicwrites>=1.0; sys_platform == "win32" in c:\users\sbhandari\appdata\local\packages\pythonsoftwarefoundation.python.3.9_qbz5n2kfra8p0\localcache\local-packages\python39\site-packages (from pytest) (1.4.0)
Requirement already satisfied: colorama; sys_platform == "win32" in c:\users\sbhandari\appdata\local\packages\pythonsoftwarefoundation.python.3.9_qbz5n2kfra8p0\localcache\local-packages\python39\site-packages (from pytest) (0.4.4)
Requirement already satisfied: pyparsing>=2.0.2 in c:\users\sbhandari\appdata\local\packages\pythonsoftwarefoundation.python.3.9_qbz5n2kfra8p0\localcache\local-packages\python39\site-packages (from packaging->pytest) (2.4.7)
Installing collected packages: pytest
WARNING: The scripts py.test.exe and pytest.exe are installed in 'C:\Users\sbhandari\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pytest-6.2.1
WARNING: You are using pip version 20.2.3; however, version 20.3.3 is available.
You should consider upgrading via the 'C:\Users\sbhandari\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\python.exe -m pip install --upgrade pip' command.
C:\Users\sbhandari>pytest
'pytest' is not recognized as an internal or external command,
operable program or batch file.
it worked on my computer,i think its not working on your computer because you haven't added it to system variables(PATH),here's how you can do so
pytest and other scripts are mostly present in a directory(in python folder) named 'Scripts'.maybe yours is somewhere else and if so then type pytest.exe in start menu and copy the file location and add it to system variables.if this doesnt work try re-installing pytest or python.
Edit:if you check the output pip is telling you that WARNING: The scripts py.test.exe and pytest.exe are installed in 'C:\Users\sbhandari\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\Scripts' which is not on PATH. Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.just add the directory above on your path and it should solve your problem
I am trying to run a python-script on an aws ec2-instance using jenkins.
I get the following error:
[ProdTest] $ /bin/sh -xe /tmp/jenkins14047325752732522807.sh
+ python3 prod.py
Traceback (most recent call last):
File "prod.py", line 2, in <module>
import base
File "/home/jenkins-slave-strat/workspace/ProdTest/base.py", line 3, in <module>
import boto3
ModuleNotFoundError: No module named 'boto3'
Build step 'Execute shell' marked build as failure
Finished: FAILURE
on the ec2 instance:
$ python3 --version
Python 3.7.9
$ pip3 install boto3 --user
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
Requirement already satisfied: boto3 in /usr/local/lib/python3.7/site-packages (1.16.25)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in /usr/local/lib/python3.7/site-packages (from boto3) (0.10.0)
Requirement already satisfied: s3transfer<0.4.0,>=0.3.0 in /usr/local/lib/python3.7/site-packages (from boto3) (0.3.3)
Requirement already satisfied: botocore<1.20.0,>=1.19.25 in /usr/local/lib/python3.7/site-packages (from boto3) (1.19.25)
Requirement already satisfied: urllib3<1.27,>=1.25.4; python_version != "3.4" in /usr/local/lib/python3.7/site-packages (from botocore<1.20.0,>=1.19.25->boto3) (1.26.2)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /usr/local/lib/python3.7/site-packages (from botocore<1.20.0,>=1.19.25->boto3) (2.8.1)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.20.0,>=1.19.25->boto3) (1.15.0)
How do i get python to access the packages in /usr/local/lib/python3.7/site-packages ?
Thanks to everyone taking the time!
Before I give a short answer, a few cautions:
I don't know what version of Linux (I assume) the EC2 instance is running.
As a result, it's unclear if Python 2 is installed, but since you used pip3 and not just pip, it seems like it might be.
You could try altering your $PATH variable, but it's generally considered a good practice to use virtual environments for Python. I'm personally a fan of Conda, but you can find guides for Pipenv as well. My advice is; install Conda via Miniconda, then do the following:
$ conda create --name myAppEnv python=your.python.version
$ conda activate myAppEnv
$ conda install [your libraries]
I have a recurrent problem I have yet been unable to find an answer to.
Whenever I install a package in PyCharm through the terminal, whenever I try importing the module the package appears to not be installed. Or, like just now, the package may already be installed but when importing the module I get the error "No module named x":
C:\Users\TimStack\PycharmProjects\API>pip install requests
Requirement already satisfied: requests in c:\users\TimStack\appdata\local\programs\python\python37\lib\site-packages (2.23.0)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\TimStack\appdata\local\programs\python\python37\lib\site-packages (from requests) (2019.11.28)
Requirement already satisfied: idna<3,>=2.5 in c:\users\TimStack\appdata\local\programs\python\python37\lib\site-packages (from requests) (2.9)
Requirement already satisfied: chardet<4,>=3.0.2 in c:\users\TimStack\appdata\local\programs\python\python37\lib\site-packages (from requests) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in c:\users\TimStack\appdata\local\programs\python\python37\lib\site-packages (from requests) (1.25.8)
Looking at these directories, it seems to refer to an old Python 3.7 installation. However, my environment uses 3.8.
What's the issue at hand here, and how do I go about solving it?
You need to uninstall the old 3.7 version as you have 2 conflicting Python versions.
OR
Specify your Python version when installing packages:
pip3.8 install [package name]
(3.8 for Python 3.8)
pip install takes --python-version <python_version> as argument.
You can pass 3.8 as argument and it will pick up the right one for you.
from the docs -
The Python interpreter version to use for wheel and "Requires-
Python" compatibility checks. Defaults to a version derived from
the running interpreter. The version can be specified using up
to three dot-separated integers (e.g. "3" for 3.0.0, "3.7" for
3.7.0, or "3.7.3"). A major-minor version can also be given as a
string without dots (e.g. "37" for 3.7.0).