We Keep Coding

Make Docker container use newest version of Python installed

I have a couple of Python modules that I use inside my Docker container and they require a higher version of Python that what's being used. I install Python and install the modules using:
RUN apt-get update || : && apt-get install python3 -y
RUN apt-get install -y python3-pip
COPY requirements.txt /project
RUN pip3 install -r requirements.txt
Expecting I would be using the latest version of Python in my Docker container but when I go into it's shell and run python3 --version is comes as 3.4.2 which is incredibly old for my program. How do I make the default Python to be the latest I installed above without messing over the System-level python?
The image runtime I'm using for the Docker container is: node:9-slim

I don't think you can find a prebuilt python3.9 package on a debian 8 distribution as your environment is pretty old.
The only solution is you build the python3.9 out from source code in your base container. A full workable Dockerfile as next:
FROM node:9-slim
RUN apt update; \
apt install -y build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev; \
wget https://www.python.org/ftp/python/3.9.7/Python-3.9.7.tgz; \
tar -zxvf Python-3.9.7.tgz; \
cd Python-3.9.7; \
./configure --prefix=/usr/local/python3; \
make && make install; \
ln -sf /usr/local/python3/bin/python3.9 /usr/bin/python3; \
ln -sf /usr/local/python3/bin/pip3.9 /usr/bin/pip3
Verify it:
$ docker build -t myimage:1 .
$ docker run --rm -it myimage:1 python3 --version
Python 3.9.7
$ docker run --rm -it myimage:1 pip3 --version
pip 21.2.3 from /usr/local/python3/lib/python3.9/site-packages/pip (python 3.9)

python3.7 No module named pip

i had a problem with installing packages to new upgraded python to version 3.7
When i type:
python3.7 -m pip install pip -d
/usr/local/bin/python3.7: No module named pip
I make easy_install like this: sudo easy_install pip what solve previous problems, but now it create pip3.7 in a weird way. Whem i calling for pip3.8 version, this returns me a message:
pip3.7 -V
pip 19.2.3 from /usr/local/lib/python3.5/dist-packages/pip (python 3.5)
Someone know what i am doing wrong? I tried many things to resolve it. Reinstall, purge, install with symbolic link from python2 etc.
There is a script which i used to install python3.7 (i had also a problem with zlib):
sudo cd /home
sudo wget https://www.python.org/ftp/python/3.7.3/Python-3.7.3.tar.xz
sudo tar xf Python-3.7.3.tar.xz
sudo cd ./Python-3.7.3/
sudo ./configure
sudo make
sudo make install
sudo update-alternatives --install /usr/bin/python3 python3 /usr/local/bin/python3 10
python3 --version
Where is a mistake?

Answer finally i find myself. There: https://linuxize.com/post/how-to-install-python-3-7-on-debian-9/
There with similar errors you can recompile it, even without removal (on secound machine i checked without removal) failed installation before.
If someone want to good upgrade with everywith working on debian 9. I recommend that script:
NOW_DIR=$(pwd)
apt update
apt install build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev wget
mkdir ~/python_upgrade
cd ~/python_upgrade
curl -O https://www.python.org/ftp/python/3.7.3/Python-3.7.3.tar.xz
tar -xf Python-3.7.3.tar.xz
cd Python-3.7.3
./configure --enable-optimizations
make -j $(nproc)
make altinstall
python3.7 --version
cd $NOW_DIR
rm -rf ~/python_upgrade
There are minor changes from that commands in tutorial. Run above file.sh as root.
It can take some times, on my virtual machine with 2 cores during another job it takes 40 minutes. I hope someone find there good answer, not totally green comments like 'apt install' which is first try action before looking to web.

Install PyQt5 on Raspberry for Python3.6

Since I found no answer for my question, neither in older posts nor in other forums, I want to ask the stackoverflow community for advice.
I am using a raspberry pi 3B+, version 9.4 (lite) with kernel version 4.14.71-v7.
I use python3.6. I installed it as follows:
sudo apt-get install build-essential checkinstall
sudo apt-get install libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev
cd /usr/src
sudo wget https://www.python.org/ftp/python/3.6.0/Python-3.6.0.tgz
sudo tar xzf Python-3.6.0.tgz
sudo -s
cd Python-3.6.0
bash configure
make altinstall
exit
Installation was without any trouble and everything works perfectly.
Now I wanted to install the PyQt5 modul for python3.6. I usually use
sudo python3.6 -m pip install ...
for installing a modul for python3.6. Trying
sudo python3.6 -m pip install pyqt5
gave me the error message
Could not find a version that satisfies the requirement PyQt5 (from versions: )
No matching distribution found for PyQt5
So I tried
sudo apt-get update
sudo apt-get install qt5-default pyqt5-dev pyqt5-dev-tools
But it installed PyQt5 for python3.5 (which is preinstalled) on the raspberry.
So does anybody know how to use or install PyQt5 for the subsequently installed
python3.6?
Edit 08.03.2019:
Thanks FlyingTeller. I started to build from source. I followed the steps from
your link.
sudo apt-get update
cd /usr/src
sudo wget https://www.riverbankcomputing.com/static/Downloads/sip/sip-4.19.14.tar.gz
sudo tar xzf sip-4.19.14.tar.gz
cd sip-4.19.14
sudo -s
python3.6 configure.py --sip-module=PyQt5.sip
make
make install
cd /usr/src
sudo wget https://www.riverbankcomputing.com/static/Downloads/PyQt5/PyQt5_gpl-5.12.tar.gz
sudo tar xzf PyQt5_gpl-5.12.tar.gz
cd PyQt5_gpl-5.12
python3.6 configure.py
Then I received the following error
Error: Use the --qmake argument to explicitly specify a working Qt qmake.
I think I am on the right way, but I do not understand what qmake is or what it
means.
Edit 10.03.2019:
I could solve the last error message. I installed
sudo apt-get install qt5-default
Then I did the same procedure as already mentioned. Now I get the error
fatal error: sip.h: File or directory not found
#include <sip.h>
PyQt5 config.py is in: /usr/src/PyQt5_gpl-5.12
SIP sip.h is in: /usr/src/sip-4.19.14
Anybody an idea? Thanks guys.

Today I found the solution. The steps below worked for me, without any error. The whole process took almost two hours.
sudo apt-get update
sudo apt-get install qt5-default
sudo apt-get install sip-dev
cd /usr/src
sudo wget https://www.riverbankcomputing.com/static/Downloads/sip/sip-4.19.14.tar.gz
sudo tar xzf sip-4.19.14.tar.gz
cd sip-4.19.14
sudo python3.6 configure.py --sip-module PyQt5.sip
sudo make
sudo make install
cd /usr/src
sudo wget https://www.riverbankcomputing.com/static/Downloads/PyQt5/PyQt5_gpl-5.12.tar.gz
sudo tar xzf PyQt5_gpl-5.12.tar.gz
cd PyQt5_gpl-5.12
sudo python3.6 configure.py
sudo make
sudo make install

Seems like they moved some things around. This seems to work, as far as getting things, and compiling them. It takes a long time to build.
For the associated designer, look at:
QtDesigner for Raspberry Pi
sudo apt-get update
sudo apt-get install qt5-default
sudo apt-get install sip-dev
cd /usr/src
sudo wget https://www.riverbankcomputing.com/static/Downloads/sip/4.19.23/sip-4.19.23.tar.gz
sudo tar xzf sip-4.19.23.tar.gz
cd sip-4.19.23
sudo python3 configure.py --sip-module PyQt5.sip
sudo make
sudo make install
cd /usr/src
sudo wget https://www.riverbankcomputing.com/static/Downloads/PyQt5/5.13.2/PyQt5-5.13.2.tar.gz
sudo tar xzf PyQt5-5.13.2.tar.gz
cd PyQt5-5.13.2
sudo python3 configure.py
sudo make
sudo make install

In my case it helped to update pip from verion 18 to the newest, in my case 20.2 (python -m pip install --upgrade pip) and then do a pip install PyQt5.

The instructions used in the accepted answer did not work for me. I think it's simply because they are outdated. I wanted to post the list of commands that did work for me. I'm running a Pi 400 with the latest version of Raspbian as of 12/20/2020. I used the default python3 rather than python3.6.
Here is the modified list of commands that worked for me:
sudo apt-get install qt5-default
sudo apt-get install sip-dev
cd /usr/src
sudo wget https://www.riverbankcomputing.com/static/Downloads/sip/sip-5.5.1.dev2011271026.tar.gz
sudo tar xzf sip-5.5.1.dev2011271026.tar.gz
cd sip-5.5.1.dev2011271026
sudo python3 setup.py build
sudo python3 setup.py install
sudo wget https://files.pythonhosted.org/packages/28/6c/640e3f5c734c296a7193079a86842a789edb7988dca39eab44579088a1d1/PyQt5-5.15.2.tar.gz
sudo tar xzf PyQt5-5.15.2.tar.gz
cd PyQt5-5.15.2
sudo python3 configure.py
sudo make
sudo make install

Can't comment due to reputation but I would add to Christ Troutner's useful updated answer, that in case users get a No module named 'PyQt5.sip' error, try --sip-module PyQt5.sip during configure, per the docs:
Note
When building PyQt5 v5.11 or later you must configure SIP to create a
private copy of the sip module using a command line similar to the
following:
python configure.py --sip-module PyQt5.sip
If you already have SIP installed and you just want to build and
install the private copy of the module then add the --no-tools option.

The accepted answers did not work for me, below is the code that worked for me.
sudo apt-get update
sudo apt-get install qt5-default
sudo apt-get install qtcreator
This code not only installs Qtcreator but also installs Qt5 Assistant, Qt5 Designer, and Qt5 Linguist

How to install xapian with Python 3.6 on Ubuntu 16.04?

I installed Python 3.6 on Ubuntu 16.04 on Docker using the ppa:jonathonf/python-3.6 repository. Now I'd like to install xapian so I can use it with Python. I have not found any ready-made packages, so I am trying to build it from sources. I set PYTHON3 and PYTHON3_LIB parameters to point to Python 3.6. During the build process I get the following error:
ImportError: libxapian.so.30: cannot open shared object file: No such file or directory
I tried xapian versions 1.3.7 and 1.4.5 without luck.
How can I install xapian?
Here's a Dockerfile to reproduce my error:
FROM ubuntu:16.04
RUN apt-get update \
&& apt-get install -y software-properties-common python-software-properties
RUN add-apt-repository ppa:jonathonf/python-3.6
RUN apt-get update \
&& apt-get install -y python3-pip docker.io python3.6 python3.6-dev software-properties-common \
python-software-properties build-essential wget unzip cmake python3-sphinx \
&& cd /usr/local/bin \
&& ln -s /usr/bin/python3.6 python
RUN python -m pip install --upgrade pip
# install xapian 1.4.5
RUN apt-get update && apt-get install -y curl uuid-dev zlib1g-dev
WORKDIR /root
RUN curl --silent --show-error --fail --next -O https://oligarchy.co.uk/xapian/1.4.5/xapian-core-1.4.5.tar.xz
RUN curl --silent --show-error --fail --next -O https://oligarchy.co.uk/xapian/1.4.5/xapian-bindings-1.4.5.tar.xz
RUN tar xvf xapian-core-1.4.5.tar.xz
RUN tar xvf xapian-bindings-1.4.5.tar.xz
WORKDIR /root/xapian-core-1.4.5
RUN ./configure && make && make install
WORKDIR /root/xapian-bindings-1.4.5
RUN ./configure PYTHON3=/usr/bin/python3.6 PYTHON3_LIB=/usr/lib/python3.6 --with-python3 && make && make install
RUN python -c "import xapian"

The problem is that the Xapian library (libxapian.so.30) is being installed into /usr/local/lib by default, but Ubuntu doesn't know that it's been put there yet. You can tell it by adding:
RUN ldconfig
after installing the core (so before you change WORKDIR to build the bindings).
There's some helpful information about ldconfig and library search paths on Ubuntu in the answers to this Unix Stackexchange question.

pyconfig.h missing during "pip install cryptography"

I wanna set up scrapy cluster follow this link scrapy-cluster,Everything is ok before I run this command:
pip install -r requirements.txt
The requirements.txt looks like:
cffi==1.2.1
characteristic==14.3.0
ConcurrentLogHandler>=0.9.1
cryptography==0.9.1
...
I guess the above command means to install packages in requirements.txt.But I don't want it to specify the version,So I change it to this:
cat requirements.txt | while read line; do pip install ${line%%[>=]*} --user;done
When install cryptography,it gives me the error:
build/temp.linux-x86_64-2.7/_openssl.c:12:24:fatal error:pyconfig.h:No such file or directory
#include <pyconfig.h>
I don't know how to solved this , I have tried a lot of methods ,but failed. my system is centos 7, and the version of python is 2.7.5(default).
Besides, Is there any other scrapy frame which is appliable for a large number of urls . Thanks in advance

For Ubuntu, python2
apt-get install python-dev
For Ubuntu, python3
apt-get install python3-dev

I have solved it by myself. for the default python of centos, there is only a file named pyconfg-64.h in usr/include/python2.7/,So run the command
yum install python-devel
Then it works.

for python3.6,
apt-get install python3.6-dev
and
apt-get install libssl-dev libffi-dev

i use python 2 on ubuntu and got the same problem when installing cryptography.
after i run this command
apt-get install python-dev libssl-dev libffi-dev
then it works.

For Python 3.7 on Debian, the following works for me.
apt-get install python3.7-dev
and
apt-get install libssl-dev
You may also need:
apt-get install libffi-dev

On a Debian based distro (AntiX distro), together with apt-get install python3-dev, I also installed rust, to complete successfully the pip3 install cryptography command. So, I gave:
$ sudo apt-get install build-essential curl python3-dev libssl-dev libffi-dev
$ sudo curl https://sh.rustup.rs -sSf | sh
When prompted (Figure 1), type 1 and hit Enter on your keyboard.
Once it completes, you have to give following commands:
$ source $HOME/.cargo/env
$ source ~/.profile
$ pip3 install cryptography