We Keep Coding

tox unable to find boto3 even though it is installed

I have a Python tox project where I run the tox for running the test case and I came across one error a few hours back and am unable to resolve it till now. My module is using boto3 library and is installed using both the commands:
pip3 install boto3
pip install boto3 //for venv environments
When I try to install it again it gives me the below stack trace:
Requirement already satisfied: boto3 in ./venv/lib/python3.8/site-packages (1.26.34)
Requirement already satisfied: botocore<1.30.0,>=1.29.34 in ./venv/lib/python3.8/site-packages (from boto3) (1.29.34)
Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in ./venv/lib/python3.8/site-packages (from boto3) (1.0.1)
Requirement already satisfied: s3transfer<0.7.0,>=0.6.0 in ./venv/lib/python3.8/site-packages (from boto3) (0.6.0)
Requirement already satisfied: urllib3<1.27,>=1.25.4 in ./venv/lib/python3.8/site-packages (from botocore<1.30.0,>=1.29.34->boto3) (1.26.13)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in ./venv/lib/python3.8/site-packages (from botocore<1.30.0,>=1.29.34->boto3) (2.8.2)
Requirement already satisfied: six>=1.5 in ./venv/lib/python3.8/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.30.0,>=1.29.34->boto3) (1.16.0)
But when I run tox it gives me the below error:
File "/Users/tony/IdeaProjects/abc/provisioner/.tox/py38/lib/python3.8/site-packages/api/lambda_handler.py", line 1, in <module>
import boto3
ModuleNotFoundError: No module named 'boto3'
Is there some path issue? I am using Python 3.8.10. I tried uninstalling and installing the packages but nothing changed.
Any help is much appreciated.

tox creates an isolated environment for building and testing. So, when you install a package somewhere, here boto3 with pip/pip3, maybe system-wide, tox ignores that - and that is a good thing, otherwise it would be impossible to have a clean test environment.
You need to provide the necessary dependencies to tox.
tox automatically installs a package, ie. when there is a setup.py or a pyproject.toml. You could also provide your dependencies directly via the deps configuration key:
[testenv]
commands = ...
deps = boto3
There is another way. You could "break" the test isolation by using the tox-current-env plugin, see https://pypi.org/project/tox-current-env/
Disclaimer
I am on of the tox maintainers.

Install dependencies via pip -r requirements.txt

I want to install this library: https://github.com/stefanfoulis/django-phonenumber-field
I have done like this:
(venv) michael#michael:~/PycharmProjects/client$ pip install django-phonenumber-field[phonenumbers]
Requirement already satisfied: django-phonenumber-field[phonenumbers] in ./venv/lib/python3.8/site-packages (5.1.0)
Requirement already satisfied: Django>=2.2 in ./venv/lib/python3.8/site-packages (from django-phonenumber-field[phonenumbers]) (3.2.3)
Collecting phonenumbers>=7.0.2; extra == "phonenumbers"
Downloading phonenumbers-8.12.23-py2.py3-none-any.whl (2.6 MB)
|████████████████████████████████| 2.6 MB 2.8 MB/s
Requirement already satisfied: sqlparse>=0.2.2 in ./venv/lib/python3.8/site-packages (from Django>=2.2->django-phonenumber-field[phonenumbers]) (0.4.1)
Requirement already satisfied: asgiref<4,>=3.3.2 in ./venv/lib/python3.8/site-packages (from Django>=2.2->django-phonenumber-field[phonenumbers]) (3.3.4)
Requirement already satisfied: pytz in ./venv/lib/python3.8/site-packages (from Django>=2.2->django-phonenumber-field[phonenumbers]) (2021.1)
Installing collected packages: phonenumbers
Successfully installed phonenumbers-8.12.23
We can see that this library has installed its dependency: phonenumbers library.
But now I want to write in requirements.txt that I need django-phonenumber-field of a certain release. So that its dependency should be installed as appropriate.
In other words how to write requirements.txt for future pip install -r requirements.txt.
Could you help me here?
=====ADDED LATER=======
pip freeze > requirements.txt doesn't seem to be appropriate as it will contain all the dependencies. It will become overburden with dependencies. I will forget why a particular library is installed. Whether it is a dependency at all. If it is, what library depends on it.
I want to keep requirements.txt as clear as possible.
In this particular example I want just one directive: install django-phonenumber-field==5.1.0 and manage dependencies yourself.
Wrong way (bloated and unclear what to delete if I stop using the library):
django-phonenumber-field==5.1.0
phonenumbers==8.12.23
What I want:
django-phonenumber-field==5.1.0 [and all dependencies]

As #jonrsharpe said, you can use pip freeze which lists all your dependencies, and redirect the output to your requirements.txt file to update it.
pip freeze > requirements.txt

The simple way is to create a text file, type the shell commands in, save it and the rename it from .txt to .bat. Now you can double click the file and it installs the requirements. You could probably even execute the .bat

Why don't my modules install in my current python environment?

I have a recurrent problem I have yet been unable to find an answer to.
Whenever I install a package in PyCharm through the terminal, whenever I try importing the module the package appears to not be installed. Or, like just now, the package may already be installed but when importing the module I get the error "No module named x":
C:\Users\TimStack\PycharmProjects\API>pip install requests
Requirement already satisfied: requests in c:\users\TimStack\appdata\local\programs\python\python37\lib\site-packages (2.23.0)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\TimStack\appdata\local\programs\python\python37\lib\site-packages (from requests) (2019.11.28)
Requirement already satisfied: idna<3,>=2.5 in c:\users\TimStack\appdata\local\programs\python\python37\lib\site-packages (from requests) (2.9)
Requirement already satisfied: chardet<4,>=3.0.2 in c:\users\TimStack\appdata\local\programs\python\python37\lib\site-packages (from requests) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in c:\users\TimStack\appdata\local\programs\python\python37\lib\site-packages (from requests) (1.25.8)
Looking at these directories, it seems to refer to an old Python 3.7 installation. However, my environment uses 3.8.
What's the issue at hand here, and how do I go about solving it?

You need to uninstall the old 3.7 version as you have 2 conflicting Python versions.
OR
Specify your Python version when installing packages:
pip3.8 install [package name]
(3.8 for Python 3.8)

pip install takes --python-version <python_version> as argument.
You can pass 3.8 as argument and it will pick up the right one for you.
from the docs -
The Python interpreter version to use for wheel and "Requires-
Python" compatibility checks. Defaults to a version derived from
the running interpreter. The version can be specified using up
to three dot-separated integers (e.g. "3" for 3.0.0, "3.7" for
3.7.0, or "3.7.3"). A major-minor version can also be given as a
string without dots (e.g. "37" for 3.7.0).

why is the pip install process stuck on ''Installing collected packages" step?

I'm trying to pip install some python libraries in a virtual environment created by conda create, but for some packages, the installation were stuck on the step "Installing collected packages: .
Take pandas as an example:
My command and output are as follows:
pip install pandas --no-cache-dir
Collecting pandas
Downloading https://files.pythonhosted.org/packages/99/12/bf4c58eea94cea4f91ff931f284146337814fb8546e6eb0b52584446fd52/pandas-0.24.1-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (16.3MB)
100% |████████████████████████████████| 16.3MB 11.4MB/s
Requirement already satisfied: numpy>=1.12.0 in /anaconda/envs/testctds2/lib/python3.6/site-packages (from pandas) (1.16.1)
Requirement already satisfied: pytz>=2011k in /anaconda/envs/testctds2/lib/python3.6/site-packages (from pandas) (2018.9)
Requirement already satisfied: python-dateutil>=2.5.0 in /anaconda/envs/testctds2/lib/python3.6/site-packages (from pandas) (2.8.0)
Requirement already satisfied: six>=1.5 in /anaconda/envs/testctds2/lib/python3.6/site-packages (from python-dateutil>=2.5.0->pandas) (1.12.0)
Installing collected packages: pandas
The process just hang there (at least for 30 min) until I control+z to quit (control+c got no response).
What I have tried:
conda install pandas worked well, which is also the recommended way to install pandas. I just don't understand why pip install didn't work, as it's assumed to and this situation also happened to some other libraries such as numpy, scipy, and scikit-learn.
I also tried without --no-cache-dir or -vvv to see more details, but in either case there were no more information or error code after the line "Installing collected packages: pandas"
I tried the command in a new terminal window. Magically numpy can be installed very quickly by "pip install numpy", but it didn't work with pandas or scipy.

I see this may be a problem other users are having. Here is a github link describing the same problem. There are a few others on the Conda GitHub page. Some of the answers that come from that post are:
Make sure you are up to date on your root conda environment. try: conda upgrade conda
Create a brand new virtual env
Micheal Grant, who is a Director for Technical Consulting at Anaconda replied to that thread with this:
That said, when I look at the debug output, I'm finding that it's not able to prune back the package list very well. The more "old" packages it has to consider the higher the likelihood that this kind of solver stall happens. Thankfully it is a lot less likely than it used to be.

Python with Pushover: No module named pushover

I tested a python script to send anythink with Pushover. But I get the error "ImportError: No module named pushover"
My installed Versions:
# pip install python-pushover
Collecting python-pushover
Using cached https://files.pythonhosted.org/packages/6f/3d/144a0137c749bd152c3ab7f4d3ce8fe1455168dab36c2fcd900d3fab16ad/python-pushover-0.4.tar.gz
Requirement already satisfied: requests>=1.0 in /usr/local/lib/python3.5/dist-packages (from python-pushover) (2.21.0)
Requirement already satisfied: urllib3<1.25,>=1.21.1 in /usr/local/lib/python3.5/dist-packages (from requests>=1.0->python-pushover) (1.24.1)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /usr/local/lib/python3.5/dist-packages (from requests>=1.0->python-pushover) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.5/dist-packages (from requests>=1.0->python-pushover) (2018.11.29)
Requirement already satisfied: idna<2.9,>=2.5 in /usr/local/lib/python3.5/dist-packages (from requests>=1.0->python-pushover) (2.8)
Installing collected packages: python-pushover
Running setup.py install for python-pushover ... done
Successfully installed python-pushover-0.4
# python -V
Python 2.7.13
# python3 -V
Python 3.5.3
The scriptheader:
#!/usr/bin/env python
import pushover
I have tried with pip(3) to install python-pushover but with no success.

I faced this exact error today. This is due to co-existing of different versions of python in your system.
Do, /usr/bin/python3 if the module was installed for py3 and just /usr/bin/python for py2 before running the script.
Refer Installed module using pip, not found for more.

In my case, you should :
uninstall that package:
pip uninstall pushover
and install the correct package:
pip install python-pushover
And your code will work fine.