We Keep Coding

Docker : Add a valid entrypoint for multiple python script

Helli, I have to build a Docker image for the following bioinformatics tool: https://github.com/CAMI-challenge/CAMISIM. Their dockerfile works but takes a long time to build and I would like to build my own, slightly differently, to learn. I face issues: there are several python script that I should be able to choose to run, not only a main. If I add one script in particular as an ENTRYPOINT then the behavior isn't exactly what I shoud have.
The Dockerfile:
FROM ubuntu:20.04
ENV DEBIAN_FRONTEND=noninteractive
USER root
#COPY ./install_docker.sh ./
#RUN chmod +x ./install_docker.sh && sh ./install_docker.sh
RUN apt-get update && \
apt install -y git python3-pip libxml-simple-perl libncursesw5 && \
git clone https://github.com/CAMI-challenge/CAMISIM.git && \
pip3 install numpy ete3 biom-format biopython matplotlib joblib scikit-learn
ENTRYPOINT ["python3"]
ENV PATH="/CAMISIM/:${PATH}"
This yields :
sudo docker run camisim:latest metagenomesimulation.py --help
python3: can't open file 'metagenomesimulation.py': [Errno 2] No such file or directory
Adding that script as an ENTRYPOINT after python3 allows me to use it with 2 drawbacks: I cannot use another script (I could build a second docker image but that would be a bad solution), and it outputs:
ERROR: 0
usage: python metagenomesimulation.py configuration_file_path
#######################################
# MetagenomeSimulationPipeline #
#######################################
Pipeline for the simulation of a metagenome
optional arguments:
-h, --help show this help message and exit
-silent, --silent Hide unimportant Progress Messages.
-debug, --debug_mode more information, also temporary data will not be deleted
-log LOGFILE, --logfile LOGFILE
output will also be written to this log file
optional config arguments:
-seed SEED seed for random number generators
-s {0,1,2}, --phase {0,1,2}
available options: 0,1,2. Default: 0
0 -> Full run,
1 -> Only Comunity creation,
2 -> Only Readsimulator
-id DATA_SET_ID, --data_set_id DATA_SET_ID
id of the dataset, part of prefix of read/contig sequence ids
-p MAX_PROCESSORS, --max_processors MAX_PROCESSORS
number of available processors
required:
config_file path to the configuration file
You can see there is an error that should'nt be there, it actually does not use the help flag. The original Dockerfile is:
FROM ubuntu:20.04
RUN apt update
RUN apt install -y python3 python3-pip perl libncursesw5
RUN perl -MCPAN -e 'install XML::Simple'
ADD requirements.txt /requirements.txt
RUN cat requirements.txt | xargs -n 1 pip install
ADD *.py /usr/local/bin/
ADD scripts /usr/local/bin/scripts
ADD tools /usr/local/bin/tools
ADD defaults /usr/local/bin/defaults
WORKDIR /usr/local/bin
ENTRYPOINT ["python3"]
It works but shows the error as above, so not so much. Said error is not present when using the tool outside of docker. Last time I made a Docker image I just pulled the git repo and added the main .sh script as an ENTRYPOINT and everything worked despite being more complex (see https://github.com/Louis-MG/Metadbgwas).
Why would I need ADD and moving everything ? I added the git folder to the path, why can't I find the scripts ? How is it different from the Metadbgwas image ?

In your first setup, you start in the image root directory / and run git clone to check out the repository into /CAMISIM. You never change the current directory, though, so when you try to run python3 metagenomesimulation.py --help it's looking in / and not /CAMISIM, hence the "not found" error.
You can fix this just by changing the current directory. At any point after you check out the repository, run
WORKDIR /CAMISIM
You should also delete the ENTRYPOINT line. For each of the scripts you could run as a top-level entry point, check two things:
Is it executable; if you ls -l metagenomesimulation.py are there x in the permission listing? If not, on the host system, run chmod +x metagenomesimulation.py and commit to source control. (Or you could RUN chmod ... in the Dockerfile if you really can't change the repository.)
Does it have a "shebang" line? The very first line of the script should be
#!/usr/bin/env python3
If both of these things are true, then you can just run ./metagenomesimulation.py without explicitly saying python3; since you add the directory to $PATH as well, you can probably run it without specifying the ./... file location.
(Probably deleting the ENTRYPOINT line on its own is enough, given that ENV PATH setting, but your script still might be confused by starting up in the wrong directory.)
The long "help" output just suggests to me that the script is expecting a configuration file name as a parameter and you haven't provided it, or else you've repeated the script name in both the entrypoint and command parts of the container command string.

In the end very little was recquired and the original Dockerfile was correct, the same error is displayed anyway, that is due to the script itself.
What was missing was a link to the interpreter, so I could remove the ENTRYPOINT and actually interpret the script instead of having python look for it in its own path. The Dockerfile:
FROM ubuntu:20.04
ENV DEBIAN_FRONTEND=noninteractive
USER root
RUN ln -s /usr/bin/python3 /usr/bin/python
RUN apt-get update && \
apt install -y git python3-pip libxml-simple-perl libncursesw5 && \
git clone https://github.com/CAMI-challenge/CAMISIM.git && \
pip3 install numpy ete3 biom-format biopython matplotlib joblib scikit-learn
ENV PATH="/CAMISIM:${PATH}"
Trying WORKDIR as suggested instead of the PATH yielded an error.

Run bash script from pydroid inbuilt terminal. Install miniconda on Android. Access sdcard from UserLAnd

chd.sh
#! /bin/bash
cd django/hellodjango
exec bash
python manage.py runserver
chd.py
# a=`python chd.py`;cd $a
import os
new_dir = "django/hellodjango"
os.chdir(new_dir)
are the two ways I have tried.
Also, on terminal I have tried,
. chd.sh
./chd.sh
. ./chd.sh
I have also tried to assign to variable and then run on terminal but no success.
Spent over 4 hours trying multiple methods given on stackoverflow.com but no success yet.
The only thing that has worked yet is,
alias mycd='cd django/hellodjango'
But I will have to copy paste it everytime.
alias myrun = `cd django/hellodjango && python manage.py runserver`
And,
alias myrun = `cd django/hellodjango; python manage.py runserver`
doesn't work.
This is just a sample, there are so many django commands that I have to use repeatedly. Appreciate if you have read all this way.
If you know the link where this is discussed, please attach the link, as I was not able to find after hours of search.
Edit:
/storage/emulated/0 $
This is what the prompt appears like.
/storage/emulated/0/django/hellodjango
This is the path.
/storage/emulated/0 $ cd django/hellodjango
/storage/emulated/0/django/hellodjango $ python manage.py
runserver
Watching for file changes with StatReloader
Performing system checks...
System check identified no issues (0 silenced).
July 25, 2020 - 19:08:42
Django version 3.0.7, using settings 'hellodjango.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
Individually works fine.
Edit:
/storage/emulated/0 $ chmod u+x chd.sh /storage/emulated/0 $ chmod u+x
rn.sh /storage/emulated/0 $ ./chd.sh ./chd.sh: cd: line 2: can't cd t:
No such file or directory /storage/emulated/0 $ chmod u+x chd.py
/storage/emulated/0 $ a=python chd.py;cd $a
~/data/ru.iiec.pydroid3/app_HOME $
Edit:
/data/user/0/tech.ula/files/support/dbclient: Caution, skipping
hostkey check for localhost
subham#localhost's password:
subham#localhost:~$ ls
subham#localhost:~$ cd
subham#localhost:~$ pwd
/home/subham
subham#localhost:~$ pkg install miniconda
-bash: pkg: command not found
subham#localhost:~$ apt install miniconda
Reading package lists... Done Building dependency tree
Reading state information... Done
E: Unable to locate package miniconda
subham#localhost:~$
subham#localhost:~$ cd ..
subham#localhost:/home$ cd ..
subham#localhost:/$ ls
bin dev host-rootfs mnt root srv
sys var boot etc lib opt run storage tmp data home
media proc sbin support usr
subham#localhost:/$ cd ..
subham#localhost:/$ cd sys
subham#localhost:/sys$ ls
ls: cannot open
directory '.': Permission denied
subham#localhost:/sys$ cd..
-bash: cd..: command not found
subham#localhost:/sys$ cd ..
subham#localhost:/$ cd storage
subham#localhost:/storage$ ls internal
subham#localhost:/storage$ cd internal
subham#localhost:/storage/internal$ ls
subham#localhost:/storage/internal$ ls -l total 0
subham#localhost:/storage/internal$ cd 0
-bash: cd: 0: No such file or directory subham#localhost:/storage/internal$
subham#localhost:/$ chmod -R 777 /host-rootfs
chmod: changing permissions of '/host-rootfs': Read-only file system
chmod: cannot read directory '/host-rootfs': Permission denied
subham#localhost:/$
https://github.com/CypherpunkArmory/UserLAnd/issues/46

how to parse volume in dockerfile for an application to be pushed on docker hub

FROM python:3
WORKDIR /Users/vaibmish/Documents/new/graph-report
RUN pip install graphreport==1.2.1
CMD [ cd /Users/vaibmish/Documents/new/graph-report/graphreport_metrics ]
CMD [ graphreport ]
THIS IS PART OF DCOKERFIILE
i wish to remove cd volumes from tha file and have a command like -v there so that whoever runs that can give his or her own volume path in same

The line
CMD [ cd /Users/vaibmish/Documents/new/graph-report/graphreport_metrics ]
is wrong. You achieve the same with WORKDIR:
WORKDIR /Users/vaibmish/Documents/new/graph-report/graphreport_metrics
WORKDIR creates the path if it doesn't exist and then changes the current directory to that path (same as mkdir -p /path/new && cd /path/new)
You can also declare the path as a volume and instruct who runs the container to provide their own path (docker run -v host_path:container_path ...)
VOLUME /Users/vaibmish/Documents/new/graph-report
A final note: It looks like these paths are from the host. Remember that the paths in the Dockerfile are not host paths. They are paths inside the container.

Typical practice here is to pick some fixed path inside the Docker container. It should be a different path from where your application is installed; it does not need to match any particular host path at all.
FROM python:3
RUN pip3 install graphreport==1.2.1
WORKDIR /data
CMD ["graphreport"]
docker build -t me/graphreport:1.2.1 .
docker run --rm \
-v /Users/vaibmish/Documents/new/graph-report:/data \
me/graphreport:1.2.1
(Remember that only the last CMD has an effect, and if it's not a well-formed JSON array, Docker will interpret it as a shell command. What you show in the question would run the test(1) command and not the program you're installing.)
If you're trying to install a single package from PyPI and just run it on local files, a Python virtual environment will be much easier to set up than anything based on Docker, and will essentially work as you expect:
python3 -m venv graphreport
. graphreport/bin/activate
pip3 install graphreport==1.2.1
cd /Users/vaibmish/Documents/new/graph-report
graphreport
deactivate # switch back to system Python/pip
All of the installed Python code is inside the graphreport virtual environment directory, and if you don't need this application again, you can just delete the directory tree.

Run a Python script in Vagrant

I'm learning how to use Vagrant with a Udacity course, and we're asked to run a Python script database_setup.py in our virtual machine.
For this, I created a folder "udacityproject" inside my vagrant folder on my computer. I saved my file database_setup.py there.
Now on Bash, I do
$ vagrant up
$ vagrant ssh
$ cd udacityproject
$ python database_setup.py
The interface returns:
"python: can't open file 'database_setup.py': [Errno 2] No such file
or directory".
It must be a silly mistake, but I cannot see what I am doing wrong... A similar topic was opened here (Run Python script in Vagrant) but the answers are not helping me.

The vagrant folder on your computer, which contains the file VagrantFile, is the folder /vagrant on your vm (It is under /). It's not your home directory. After vagrant ssh you are logged in home directory of user vagrant. It's /home/vagrant/.
$ vagrant ssh
$ pwd
/home/vagrant
The tree looks like that:
/root
/tmp
/usr
/var
/home
|-- vagrant # <-- You are here after logging
/vagrant
|-- udacityproject
|-- database_setup.py # <-- Your script is here
...
To run your script you must go to /vagrant
$ cd /vagrant
With ls * you can check if your file exists. Now go to your created folder and run your script
$ cd udacityproject
$ python database_setup.py
Or simply do that from beginning:
$ vagrant ssh
$ python /vagrant/udacityproject/database_setup.py

Cannot remove virtualenv directory from PC

I've tried rm -rf venvpc and I get the error message:
rm: cannot unlink `venvpc/Scripts/python.exe': Permission denied
rm: cannot remove directory `venvpc/Scripts': Directory not empty
rm: cannot remove directory `venvpc': Directory not empty
I removed the virtualenv name 'venvpc' from PyCharm in case there was a link there.
I tried removing via Windows Explorer and it says the folder or a file in it is open in another program.
What can I try next?

You need to do it as root user , so use sudo (on linux) :
sudo rm -rf venvpc
and if you are in windows run runas /noprofile /user:Administrator cmd to command as admin.