Starting qsub in parent folder with Python fails - python

I'm creating a Python script to setup a series of calculations on a remote calculation cluster. However I have run into a problem.
The calculation on the remote server is started with the command:
qsub Run.sh
Run.sh is located in the folder ./sol/
If I enter ./sol/ and run:
python25 -c "import os;os.system(\"qsub Run.sh\")"
Every thing works like it should.
However, If I am located in ./ and run this command:
python25 -c "import os;os.chdir(\"sol\");os.system(\"qsub Run.sh\")"
qsub failes with this error message:
*** error from copy
Host key verification failed.
lost connection
*** end error output
Does anyone know why this is? I use python version 2.5.1 and unix 2.6.18

Related

Issue in creating a layer for AWS lambda for psycopg2

I am trying to create a layer for aws lambda for psycopg. I am following the answer to the question - psycopg2 folder zipped into python venv lambda function
Screenshot of the process from above link
However I am getting the below error while executing the docker step.
$ docker run "lambci/lambda:build-python3.8" /bin/sh -c "pip install -r requirements.txt -t /python/lib/python3.8/site-packages/; exit"
**docker: Error response from daemon: failed to create shim task: OCI
runtime create failed: runc create failed: unable to start container
process: exec: "C:/Users/aameen/AppData/Local/Programs/Git/usr/bin/sh":
stat C:/Users/aameen/AppData/Local/Programs/Git/usr/bin/sh: no such file
or directory: unknown.**
I am executing this on the git bash on windows. May be the problem lies there.

MLFlow projects; bash: python: command not found

I'm running MLflow Project for a model using following command from my ubuntu 20.04 terminal
mlflow run . --no-conda -P alpha=0.5
My system doesn't have conda or python (It does however have python3). So, I added alias for python using terminal
alias python='python3'
After which I could open python in terminal using python. However, I still got the same error
2021/11/21 08:07:34 INFO mlflow.projects.utils: === Created directory /tmp/tmpp4h595ql for downloading remote URIs passed to arguments of type 'path' ===
2021/11/21 08:07:34 INFO mlflow.projects.backend.local: === Running command 'python tracking.py 0.5 0.1' in run with ID 'e50ca47b3f8848a083906be6220c26fc' ===
bash: python: command not found
2021/11/21 08:07:34 ERROR mlflow.cli: === Run (ID 'e50ca47b3f8848a083906be6220c26fc') failed ===
How to get rid of this error?
Change python to python3 in the MLproject file to the resolve error.
command: "python3 tracking.py {alpha} {l1_ratio}"
Running the command below in the terminal helped me:
$ sudo apt install python-is-python3
After installation, run the ML project and it should work.

"bin/bash: python: command not found" returned when running docker image

Here is the docker run output:
hausey#ubuntu:~/niso2-jxj934$ docker run niso2-jxj934
Test version: 15:59, Mar 24th 2020
Question 1: Evaluation of expression.
Command failed: /bin/bash -c "python /bin/jxj934.py -question 1 -expr \"(ifleq (ifleq -1.11298616747 1.63619642199 (sub -1.11298616747 -1.11298616747) 1.7699684348) (add (exp -0.822479932786) 1.39992604386) (add -1.11298616747 (exp 0.385042309638)) 0.205973267133)\" -n 10 -x \"-0.168958230447 -0.131749160548 0.0971246476126 1.8706205565 -0.464122426299 2.35887369763 -0.375948313434 -0.613901105864 0.411326743135 -0.149276696072\"" Exit status: exited with code 127 stderr: /bin/bash: python: command not found
Here is the Dockerfile:
FROM pklehre/niso2020-lab2-msc
ADD jxj934.py /bin
CMD ["-username","jxj934", "-submission", "python /bin/jxj934.py"]
Here is check for python:
hausey#ubuntu:~/niso2-jxj934$ which python
/usr/bin/python
Is that related to the PATH of python?
Usually, it is related to the value of PATH but, specifically, that image only has python3. In other words, looking through the filesystem with
find / -name -type f "python*"
Look for regular files named "python*" in /
There were only python3 results.
...
/usr/bin/python3.8
/usr/bin/python3.7
...
A quick solution is to specify python3 in your CMD line (python3 /bin/jxj934.py). Another is to add a soft link (ln -s /usr/bin/python /usr/bin/python3.8). The best solution is to solve it using the package manager. Then again, that depends if you're in control of the Dockerfile + image.
When you queried which python, you did so on your local machine. The container runs in a different filesystem namespace than yours and with a completely different terminal. The container will behave differently than your machine and any such investigations will yield relevant results only when run within the container.
A little unrelated to your question but it might serve you.
docker run has a --entrypoint option that allows you to override the image's entrypoint. You can ask for bash and explore the container.
docker run --it --entrypoint=bash pklehre/niso2020-lab2-msc
Note that bash has to be in the $PATH.

docker-compose error: "no such file or directory" on Windows

I am following along with this tutorial for running a Flask server using Docker. I am working on a Windows machine so I am using Docker Toolbox.
Once I enter the command to create the database table:
docker-compose run web /usr/local/bin/python create_db.py
I get the following error:
Error response from daemon: OCI runtime create failed:
container_linux.go:346: starting container process caused "exec:
\"C:/Program Files/Git/usr/local/bin/python\": stat C:/Program
Files/Git/usr/local/bin/python: no such file or directory": unknown
I am not sure why I am getting this error, any suggestions on how to fix this error would be greatly appreciated. Thank you.
The command fails because windows tries to parse the path, this can be circumvented by quoting the path:
docker-compose run web python create_db.py
If the above fails a double dash can be used:
docker-compose run web -- "/usr/local/bin/python create_db.py"
I had to change the current directory, and then to execute the script :
docker-compose run web bash -c "cd /usr/local/bin/; python create_db.py"

How to execute a python configuration that exists in "Run Configurations" Eclipse...from the command line

I would like to execute a run configuration using one parameter from my main .py module and others that are given from a run configuration created in Eclipse,
the configuration is run by another .py file called let's say "database.py" with parameters taken from the configuration like so :
-q
-u
-p
-c
-f
-s oracle
-d
-w yes
-r parameter taken from console of my main run.py
-o
I can run it from Eclipse IDE going to run configurations, but I would like this to be executed if I type "yes" into the console.

Categories