We Keep Coding

Create Pyinstaller app on Docker running Mac

I try to create a standalone python app using Pyinstaller which should run and can be reproducibly recreated on Windows, Linux, and Mac.
The idea is to use Docker to have a fixed environment that creates the app and exports it again. For Linux and Windows, I could make it work using https://github.com/cdrx/docker-pyinstaller.
the idea is to let docker create a fully functioning pyinstaller app (with GUI) within and export this app. Since pyinstaller depends on the package versions etc. this should be fixed in the docker and only the new source code should be supplied to compile and export a new version of the software.
in the ideal scenario (and how it already works with Linux and Windows), the user can create the docker and compile the software itself:
docker build -t docker_pyinstaller_linux https://raw.githubusercontent.com/loipf/calimera_docker_export/main/linux/Dockerfile
docker run --rm -v "/path/to/app_source_code:/code/" -v "${PWD}:/code/dist/" docker_pyinstaller_linux
For Mac, however, there is no simple straightforward solution. There is one Mac docker image out there https://github.com/sickcodes/Docker-OSX, but the docker creation code is not that simple.
my idea:
take https://github.com/sickcodes/Docker-OSX/blob/master/Dockerfile.auto
add to the end the download of miniconda:
RUN chmod -R 777 /Users/user
### install miniconda to /miniconda
RUN curl -LO "http://repo.continuum.io/miniconda/Miniconda3-4.4.10-MacOSX-x86_64.sh"
RUN bash Miniconda3-4.4.10-MacOSX-x86_64.sh -p /miniconda -b
ENV PATH=/miniconda/bin:${PATH}
RUN conda update -y conda
### install packages from conda
RUN conda install -c anaconda -y python=3.8
RUN apt-get install -y python3-pyqt5
...
but already the first command fails due to chmod: cannot access '/Users/user': No such file or directory since I am in a different "environment" than the interactive session (/home/arch/OSX-KVM). Can somebody help me out?
I know these asking for code pieces questions are not the best, and I could show you all the things I tried, but I doubt they will help anybody. I would like to have a minimum Mac example without user login or gui etc (which should be possible using https://github.com/sickcodes/osx-optimizer). It should only have miniconda installed (with pyinstaller and a few packages).
other info:
I can run the previous commands in the mac environment interactively in the docker environment but would like to fix these commands permanently in the image:
docker pull sickcodes/docker-osx:auto ### replace with docker with pyinstaller and packages
docker run -it \
--device /dev/kvm \
-p 50922:10022 \
sickcodes/docker-osx:auto
ideally in the end, we can run the above command with OSX_commands pyinstaller file.spec
related questions but without solution:
Using Docker and Pyinstaller to distribute my application
How to create OS X app with Python on Windows

How to install additional dependencies in Tensorman

I am on popos 20.04 LTS and I want to use Tensorman for tenserflow/python. I'm new into docker and I want to install additional dependencies for example using default Image I can run jupyter notebook using these commands -
tensorman run -p 8888:8888 --gpu --python3 --jupyter bash
jupyter notebook --ip=0.0.0.0 --no-browser
but now I have to install additional dependencies for example if I want to install jupytertheme how can I change that ? I have tried to install it directly inside docker container but its not working that way.
this issue is looking similar to my problem but there was no explanation exactly how I have to make custom image in tensorman.

There are two ways to install dependencies.
Create a custom image, install dependencies and save it.
Use the --root tag to gain root access to the container, install dependencies and use them.
Build your own custom image
If you are working on a project and want some dependencies for that project or just want to save all your favourite dependencies, you can create a custom image according to that project and save it and later use that image for your project.
Now make a list of all packages once you are ready use this command
tensorman run -p 8888:8888 --root --python3 --gpu --jupyter --name CONTAINER_NAME bash
Where CONTAINER_NAME is the name of the container you can give any name you want and -p sets the port (you can search about port forwarding in docker)
Now you are running container as root, now in the container shell use.
# its always a good idea to update the apt, you can install packages from apt also
apt update
# install jupyterthemes
pip install jupyterthemes
# check if all your desired packages are installed
pip list
Now it's time to save your image
Open a new terminal and use this command to save your image
tensorman save CONTAINER_NAME IMAGE_NAME
CONTAINER_NAME should be the one which was used earlier, and for IMAGE_NAME you can choose according to your preferences.
Now you can close the terminals use tensorman list to check your custom image is there or not. To use your custom image use
tensorman =IMAGE_NAME run -p 8888:8888 --gpu bash
# to use jupyter
jupyter notebook --ip=0.0.0.0 --no-browser
Use --root and install dependencies
Now you might be wondering that in a normal Jupyter notebook, you can install dependencies even inside the notebook, but that's not the case with tensorman; it's because we're not running it as root because if we run it as root, exported files in host machine would also be using root permissions that's why it is good to avoid using --root tag but we can use that to install dependencies. After installing, you have to save that image (it's not necessary though you can also install them every time) otherwise, installed dependencies will be lost.
In the last step of the custom image building, use these commands instead
# notice --root
tensorman =IMAGE_NAME run -p 8888:8888 --gpu --root bash
# to use jupyter, notice --allow-root
jupyter notebook --allow-root --ip=0.0.0.0 --no-browser

How to install a python module in a docker container

My Docker knowledge is very poor, I have Docker installed only because I would use freqtrade, so I followed this simple HOWTO
https://www.freqtrade.io/en/stable/docker_quickstart/
Now , all freqtrade commands run using docker , for example this
D:\ft_userdata\user_data>docker-compose run --rm freqtrade backtesting --config user_data/cryptofrog.config.json --datadir user_data/data/binance --export trades --stake-amount 70 --strategy CryptoFrog -i 5m
Well , I started to have problems when I would had try this strategy
https://github.com/froggleston/cryptofrog-strategies
for freqtrade . This strategy requires Python module finta .
I understood that the Python module finta should be installed in my Docker container
and NOT in my Windows system (it was easy "pip install finta" from console!).
Even if I tried to find a solution over stackoverflow and google, I do no understand how to do this step (install finta python module in freqtrade container).
after several hours I am really lost.
Someone can explain me in easy steps how to do this ?
Freqtrade mount point is
D:\ft_userdata\user_data

You can get bash from your container with this command:
docker-compose exec freqtrade bash
and then:
pip install finta
OR
run only one command:
docker-compose exec freqtrade pip install finta
If the above solutions didn't work, You can run docker ps command and get container id of your container. Then
docker exec -it CONTAINER_ID bash
pip install finta

You need to make your own docker image that has finta installed. Luckily you can build on top of the standard freqtrade docker image.
First make a Dockerfile with these two lines in it
FROM freqtradeorg/freqtrade:stable
RUN pip install finta
Then build the image (calling the new image myfreqtrade) by running the command
docker build -t myfreqtrade .
Finally change the docker-compose.yml file to run your image by changing the line
image: freqtradeorg/freqtrade:stable
to
image: myfreqtrade
And that should be that.

The way to get our Python code running in a container is to pack it as a Docker image and then run a container based on it.
To generate a Docker image we need to create a Dockerfile that contains instructions needed to build the image. The Dockerfile is then processed by the Docker builder which generates the Docker image. Then, with a simple docker run command, we create and run a container with the Python service.
An example of a Dockerfile containing instructions for assembling a Docker image for Python service installing finta is the following
# set base image (host OS)
FROM python:3.8
# install dependencies
RUN pip install finta
# command to run on container start
CMD [ "python", "-V" ]
For each instruction or command from the Dockerfile, the Docker builder generates an image layer and stacks it upon the previous ones. Therefore, the Docker image resulting from the process is simply a read-only stack of different layers.
docker build -t myimage .
Then, we can check the image is in the local image store:
docker images
Please refer to the freqtrade DockerFile https://github.com/freqtrade/freqtrade/blob/develop/Dockerfile

Docker Python Image

I've a RHEL host with docker installed, it has default Py 2.7. My python scripts needs a bit more modules which
I can't install due to lack of sudo access & moreover, I dont want to screw up with the default Py which is needed for host to function.
Now, I am trying to get a python in docker container where I get to add few modules do the needfull.
Issue - docker installed RHEL is not connected to internet and cant be connected as well
The laptop i have doesnt have the docker either and I can't install docker here (no admin acccess) to create the docker image and copy them to RHEL host
I was hoping if docker image with python can be downloaded from Internet I might be able to use that as is!,
Any pointers in any approprite direction would be appreciated.
what have I done - tried searching for the python images, been through the dockers documentation to create the image.
Apologies if the above question sounds silly, I am getting better with time on docker :)

If your environment is restricted enough that you can't use sudo to install packages, you won't be able to use Docker: if you can run any docker run command at all you can trivially get unrestricted root access on the host.
My python scripts needs a bit more modules which I can't install due to lack of sudo access & moreover, I dont want to screw up with the default Py which is needed for host to function.
That sounds like a perfect use for a virtual environment: it gives you an isolated local package tree that you can install into as an unprivileged user and doesn't interfere with the system Python. For Python 2 you need a separate tool for it, with a couple of steps to install:
export PYTHONUSERBASE=$HOME
pip install --user virtualenv
~/bin/virtualenv vpy
. vpy/bin/activate
pip install ... # installs into vpy/lib/python2.7/site-packages

you can create a docker image on any standalone machine and push the final required image to docker registry ( docker hub ). Then in your laptop you can pull that image and start working :)
Below are some key commands that will be required for the same.
To create a image, you will need to create a Dockerfile with all the packages installed
Or you can also do sudo docker run -it ubuntu:16.04 then install python and other packages as required.
then sudo docker commit container_id name
sudo docker tag SOURCE_IMAGE[:TAG] TARGET_IMAGE[:TAG]
sudo docker push IMAGE_NAME
Then you pull this image in your laptop and start working.
You can refer to this link for more docker commands https://github.com/akasranjan005/docker-k8s/blob/master/docker/basic-commands.md
Hope this helps. Thanks

Set JAVA_HOME for docker in NLTK for Stanford NLP

I am a beginner in using Docker.
I'm using Docker toolbox for Windows 7 , i built an image for my python web app and everything works fine.
However, for this app, i use nltk module which also needs java and java_home setting to the java file.
When running on my computer, i can mannualy set the java_home, but how to do it in the dockerfile so that it wont get error when running on another machine.
Here is my error :
p.s : Answer below

When you are running a container you have the option of passing in environment variables that will be set in your container using the -e flag. This answer explains environment variables nicely: How do I pass environment variables to Docker containers?
docker container run -e JAVA_HOME='/path/to/java' <your image>
Make sure your image actually contains Java as well. You might want to look at something like the openjdk:8 image on docker hub.
It sounds like you need a docker file to build your image. Have a look at the ENV command documented here to set the JAVA_HOME var: https://docs.docker.com/engine/reference/builder/#env and then build your image with docker build /path/to/Dockerfile
I see you've already tried that and didn't have much luck.. run the container and instead of running your application process just run a bash script along the lines of echo $JAVA_HOME so you can at least verify that part is working.
Also make sure you copy in whatever files/binaries needed to the appropriate directories within the image in the docker file as noted below.

i finally found the way to install the java for dockerfile , it is use the java install commandline of ubuntu image.
Below is the docker file . Thanks for your reading.
RUN apt-get update
RUN apt-get install -y python-software-properties
RUN apt-get install -y software-properties-common
RUN add-apt-repository -y ppa:openjdk-r/ppa
RUN apt-get update
RUN apt-get install -y openjdk-8-jdk
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64/
RUN export JAVA_HOME