I am creating a docker containing python and php. I am writing a python script to connect to a MQTT broker residing in another docker.
In my dockerfile I ensure that I install the paho client by using the following commands:
RUN apt-get install -y python3-dev
RUN apt-get install -y libffi-dev
RUN apt-get install -y libssl-dev
ADD https://bootstrap.pypa.io/get-pip.py /tmp/get-pip.py
RUN cat /tmp/get-pip.py | python3
RUN pip install paho-mqtt
RUN pip install python-etcd
However when I run the python script I get the following error:
ImportError: No module named paho.mqtt.client
The docker installation does not show any error with regards to paho-mqtt installation.
It will be great if someone can guide on this.
I think I have found the problem,
You have installed Python3 but for some reason the interpreter defaults to version 2.7 in Linux.
Try using pip3 install paho-mqtt python-etcd instead.
Or if it does not work, you can literally copy and paste the paho folder from your Python2.7 site-packages folder to your Python3 site-packages folder. I have just verified paho-mqtt 1.2 for Python2 is exactly the same as paho-mqtt 1.2 for Python3 using a Meld diff tool. Please note, when you directly copy and paste pip list will not display the package you copied.
site-packages are usually inside your system lib folder. It depends upon how Python is installed. In my case everything is inside $HOME/.pyenv folder.
Remember Python2 has it's own site-packages folder and Python3 has it's own site-packages folder where Python searches for the packages. Sometimes if you are using a Debian based Linux distro please make sure to check inside the dist-packages folder as well to see if you can find the package you are looking for.
You can try install Paho lib:
git clone https://github.com/eclipse/paho.mqtt.python
Once you have the code, it can be installed from your repository as well:
cd paho.mqtt.python
python setup.py install
Related
I am new to several concepts/tools here but have been working my way through tutorials to get this issue figured out. (I've spent a majority of my career writing embedded software or windows apps using C/C++/C#)
So, I'm getting started working in an AWS environment that another member of my team spun up a few months before I joined. I've noticed several lambdas have code just copy/pasted in them and so wanted to build a python package and import it as a layer then use it in all of our lambdas where the code is copied.
Through some reading I understand I can't just build my python package on mac os and found that I should build the package inside docker.
So, this is what I have tried so far:
Docker File
FROM amazonlinux:2
RUN yum update -y && yum install -y initscripts
RUN yum install python3 -y
RUN python3 -m venv myenv
RUN source myenv/bin/activate
RUN yum install python3-pip -y
RUN yum install zip -y
RUN pip3 install wheel
COPY . .
The last line in the docker file just dumps all of my python code/shell scripts into the docker container.
Now, from terminal:
docker build -t myimage .
docker run -it myimage /bin/bash
Once I have the container prompt:
./build-package.sh
Where build-package.sh looks like this:
pip3 install setuptools
pip3 install wheel
python3 setup.py sdist
python3 setup.py bdist_wheel
cd dist
pip3 install geocalc-1.0.0-py3-none-any.whl
cd ..
Now I am verifying my package is installed:
pip3 list
From which I get this output:
Package Version
geocalc 1.0.0
pip 20.2.2
setuptools 49.1.3
wheel 0.38.4
So I know my geocalc pacakge is installed. I find out where it is located with pip3 show geocalc. It is located under /usr/local/lib/python3.7/site-packages
Also ran a few tests inside the docker container to use the package and was able to confirm it was working as expected.
Next:
cd /usr/local/lib/python3.7/site-packages
zip -r9 ${OLDPWD}/function.zip .
In a separate terminal window:
docker ps
I get my container's ID from the ps command output and then run
docker cp bb87bf25976f:/function.zip .
So now I have my funciton.zip file from my docker container. Time to deploy to AWS:
aws lambda publish-layer-version \
--layer-name layerNameHere \
--description "Geospatial Calculations" \
--zip-file fileb://function.zip \
--compatible-runtimes python3.7
This command outputs the arn of the newly created layer. I copy this and go over to my lambda and add the layer using the arn.
At the top of my lambda:
from geocalc import geocalculator
Now when I run the lambda I get this error:
"errorMessage": "Unable to import module 'lambda_function': No module named 'geocalc'"
A couple of other details regarding python code:
geocalc.py
import math
class geocalculator:
.
.
.
setup.py
from setuptools import setup
setup(
name='geocalc',
version='1.0.0',
description='Package with a few commonly used geospatial calculations',
...
packages=['geocalc'],
)
If you've read this far, thanks for taking the time. Any suggestions about why my layer is not working properly?
I am new on linux and i tried to change the Symbolic link of python3 in /usr/bin/ ,
and i accidentally remove the python3.9 file !
But i know i didn't delete it completely Because there is still a lot of file called python3.9 .
After that 'apt' didn't work anymore and i got this error :
E: Problem executing scripts APT::Update::Post-Invoke-Success 'if /usr/bin/test -w /var/lib/command-not-found/ -a -e /usr/lib/cnf-update-db; then /usr/lib/cnf-update-db > /dev/null; fi'
Now , I only have version 2.7 of python and i can't install another because apt don't work !!
This is the result of
:
So I hope someone can help me and I wish you a good afternoon
Edit: As you've lost apt command meaning you can't install or remove anything using apt command.
The possible way to fix this is by reinstalling respective apt-package of your architecture and then do the below python installation.
To install apt-package again download the .deb file from the “/etc/apt/sources.list” file. Lots of links for installation and upgrades for packages will be in this file.
Now find the downloading source using $cat /etc/apt/sources.list command.
Find /pool/main/a/apt/ directory under downloading source and then download the .deb file which matches your architecture and download it.
Thereafter install this using dpkg command like this
sudo dpkg -i PackageName.deb
Replace the PackageName with your file name (e.g- apt_1.6.13_arm64/apt_1.9.3_i386).
Restart the PC and then check the /usr/bin/ directory to ensure if it had properly installed .
If you get nothing there then run locate apt-get command to locate it. If you can't get it then there is no other way than reinstalling the OS itself.
If you have reinstalled apt then
use the following commands to freshly install Python.
Note that all the commands below will be for Python3 as you're concerned with version 3.9.
# To uninstall the Python only
sudo apt-get remove python3.9
# To uninstall the Python with all the packages also
sudo apt-get remove --auto-remove python3.9
# To remove all the dependencies and configuration files
sudo apt-get purge --auto-remove python3.9
Now to install the Python3. The following command will install the latest version of python3. which to this date is python3.9.
sudo apt-get install python3
You can use pip to manage the python packages also.
To install pip use the following command
sudo apt install python3-pip
Now to manage the python packages using pip
# To install package; replace PackageName with the name of package(like flask)
sudo pip install PackageName
# To uninstall package
sudo pip uninstall PackageName
If you get trouble with pip get list of all the commands pip uses with
sudo pip help
You can list all the Python version installed (on default location)
ls /usr/bin/python*
Hope this will help in resolving the problem.
I'm working on a Raspberry Pi. For some complex reasons I'd rather not get into, I had to remove Python2 and Python3 from the system, and compile Python 3.9.2 from scratch. Works great.
Unfortunately, now, I need Ansible to configure the system, which requires python3-apt if you plan to use a task that uses the apt module, which I do. python3-apt appears to just be python-apt, which I can install with pip.
Unfortunately, when I try that (pip install python-apt), I get this:
ModuleNotFoundError: No module named 'DistUtilsExtra'
If my python was installed with apt, I could just do:
sudo apt-get install python3-distutils-extra
But that depends on the python packages from the apt repos, so it will install Python from apt, which I don't want.
python3-distutils-extra appears to just be a python package, but I can't seem to find it on pypi. Is there a way to install it without using apt?
Or should I just tell apt to install the package, and ignore installing dependencies?
A possible workaround is to rebuild the wheel from the source code of the apt package
apt-get source python3-distutils-extra
cd python-distutils-extra-2.40
python3.9 -m pip install wheel # You'll need this to run setup.py
python3.9 setup.py bdist_wheel # Create the wheel
python3.9 -m pip install dist/python_distutils_extra-2.39-py3-none-any.whl # Install the wheel
But when i run command sudo apt-get install python3-pip. It is installed but inot as python3.6. It is installed in python3.5.I have also specified the version in command i.e sudo apt-get install python3.6-pip.But it gives me an error.
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package python3.6-pip
E: Couldn't find any package by glob 'python3.6-pip'
E: Couldn't find any package by regex 'python3.6-pip'
You can install python packages using python3.6 -m pip install [Package_to_install] straight away.
I use PipEnv for working in different environments with different python versions.
the usage would be something like pipenv --python 3.X and it will install that version of python if its not installed.
It also makes things really helpful if you are collaborating with other developers, because it creates a PipFile (alt to requirements.pip) which will contain the version of python used for the project so all developers will use the same version of python on a project without doing any prep work.
If you don't want to use pipenv you can use pip3.5 or create a python3.5 environment like virtualenv [NAME] --python=/usr/lib/python3.5 and then install the package using pip install somepackage
This will help you fix the above problem
You need to add following personal package repository:
sudo add-apt-repository ppa:deadsnakes/ppa
Check for updates
sudo apt-get update
install python using following command
sudo apt-get install python3.6
It was easy. I just changed my default python3 version:
sudo gedit ~/.bashrc alias python3="python3.6"
Now I can install the packages using python3.6 with
-m pip install 'packagenae'
My laptop has been formatted and new OS was installed, and since then I get this error:
ImportError: No module named git
This refers to a python code which simply imports git.
Location of git before my laptop was formatted: /usr/local/bin/git
Location of git after the laptop was formatted: /usr/bin/git
How / what do I change in my python code to refer to right path ?
The git module is not the same thing as the git command line executable. They happen to have the same name and cover related tasks, but they are distinct software packages.
I'm going to assume that the git module your code is importing is the one provided by the GitPython project. You'll need to install that project, see their installation instructions:
# pip install gitpython
In my cas, I installed pythong2-git, it solved my problem.
sudo apt-get install python3-git
In my case apt install python-git fixed the issue.
You can try this.
GitPython’s git repo is available on GitHub, which can be browsed at:
https://github.com/gitpython-developers/GitPython
and cloned using:
$ git clone https://github.com/gitpython-developers/GitPython git-python
Initialize all submodules to obtain the required dependencies with:
$ cd git-python
$ git submodule update --init --recursive
In Centos7 (might also work in Redhat System), if you are using Python2.7 :
sudo yum install epel-release
sudo yum install python-pip
sudo yum install GitPython.noarch
The above command is to install gitpython in Python2