I am trying to run a bash script from my Python code. I am calling the script in a subprocess like so:
preprocessed = subprocess.check_output([
'bash',
'../Paraphrase_Demo/models/processing_utils/preprocess_data_givenBPE.sh',
phrase, src_lang, tgt_lang, this_dir
])
Here is the script preprocess_data_givenBPE.sh:
#!/usr/bin/env bash
REMOTE_PATH=$4/processing_utils
SCRIPTS=$REMOTE_PATH/tiny-moses
TOKENIZER=$SCRIPTS/tokenizer.perl
DETOKENIZER=$SCRIPTS/detokenizer_v2.perl
CLEAN=$SCRIPTS/clean-corpus-n.perl
NORM_PUNC=$SCRIPTS/normalize-punctuation.perl
REM_NON_PRINT_CHAR=$SCRIPTS/remove-non-printing-char.perl
BPEROOT=$4/zhen/tools/subword-nmt/subword_nmt
text=$1
flan=$2
BPE=$4/$2$3/data/bpecodes
echo $text | perl $NORM_PUNC -l $flan | perl $REM_NON_PRINT_CHAR |
perl $TOKENIZER -a -l $flan -q | python3 $BPEROOT/apply_bpe.py -c $BPE | cat
When I run my Python program I get the following output:
Can't open perl script "C:/Users/Administrator/source/repos/Paraphrasing/Paraphrase_Demo/models/processing_utils/tiny-moses/normalize-punctuation.perl": No such file or directory
Can't open perl script "C:/Users/Administrator/source/repos/Paraphrasing/Paraphrase_Demo/models/processing_utils/tiny-moses/remove-non-printing-char.perl": No such file or directory
Can't open perl script "C:/Users/Administrator/source/repos/Paraphrasing/Paraphrase_Demo/models/processing_utils/tiny-moses/tokenizer.perl": No such file or directory
python3: can't open file 'C:/Users/Administrator/source/repos/Paraphrasing/Paraphrase_Demo/models/zhen/tools/subword-nmt/subword_nmt/apply_bpe.py': [Errno 2] No such file or directory
I have copy and pasted each of those paths into powershell and ls'd to verify that they are correct, but it still says it can't find them. What is going on here? This also happens if I run the bash script directly.
Edit: I've tried using backslashes in the path and received the same error. I also found this post: Bash: cannot execute a perl script using an absolute path?, so I tried using a relative path from the directory of my Bash script but it still says "No such file or directory."
After lots of debugging, I found the issue. While the paths I listed exist if I ls them in powershell, typing bash in powershell doesn't just open a bash shell, it actually changes the directory structure. I think this may be related to the Windows Subsystem for Linux, but the result is that C: changes to /mnt/c once inside the bash shell. Replacing this in all my paths, I was able to run my scripts.
Related
I have a folder called TEST with inside :
script.py
script.sh
The bash file is :
#!/bin/bash
# Run the python script
python script.py
If I run the bash file like this :
./TEST/script.sh
I have the following error :
python: can't open file 'script.py': [Errno 2] No such file or directory
How could I do, to tell my script.sh to look in the directory (which may change) and to allow me to run it for inside the TEST directory ?
Tricky, my python file run a sqlite database and I have the same problem when calling the script from outside the folder, it didn't look inside the folder to find the database!
Alternative
You are able to run the script directly by adding this line to the top of your python file:
#!/usr/bin/env python
and then making the file executable:
$ chmod +x script.py
With this, you can run the script directly with ./TEST/script.py
What you asked for specifically
This works to get the path of the script, and then pass that to python.
#!/bin/sh
SCRIPTPATH="$( cd "$(dirname "$0")" ; pwd -P )"
python "$SCRIPTPATH/script.py"
Also potentially useful:
You mentioned having this problem with accessing a sqlite DB in the same folder, if you are running this from a script to solve this problem, it will not work. I imagine this question may be of use to you for that problem: How do I get the path of a the Python script I am running in?
You could use $0 which is the name of the currently executing program, as invoked, combined with dirname which provides the directory component of a file path, to determine the path (absolute or relative) that the shell script was invoked under. Then, you can apply it to the python invocation.
This example worked for me:
$ t/t.sh
Hello, world!
$ cat t/t.sh
#!/bin/bash
python "$(dirname $0)/t.py"
Take it a step farther and change your current working directory which will also be inherited by python, thus helping it to find its database:
$ t/t.sh; cat t/t.sh ; cat t/t.py ; cat t/message.txt
hello, world!
#!/bin/bash
cd "$(dirname $0)"
python t.py
with(open('message.txt')) as msgf:
print(msgf.read())
hello, world!
From the shell script, you can always find your current directory: Getting the source directory of a Bash script from within. While the accepted answer to this question provide a very comprehensive and robust solution, your relatively simple case only really needs something like
#!/bin/bash
dir="$(dirname "${BASH_SOURCE[0]}")"
# Run the python script
python "$(dir)"/script.py
Another way to do it would be to change the directory from which you run the script:
#!/bin/bash
dir="$(dirname "${BASH_SOURCE[0]}")"
# Run the python script
(cd "$dir"; python script.py)
The parentheses ((...)) around cd and python create a subprocess, so that the directory does not change for the rest of your bash script. This may not be necessary if you don't do anything else in the bash portion, but is still useful to have if you ever decide to say source your script instead of running it as a subprocess.
If you do not change the directory in bash, you can do it in Python using a combination of sys.argv\[0\], os.path.dirname and os.chdir:
import sys
import os
...
os.chdir(os.path.dirname(sys.argv[0]))
I'm trying to run a Python script from a shell script within a shell script, but I'm running into some problems.
Imagine my root shell script looks like this:
echo "test0"
sh ./test/test1/test2.sh
and my test2.sh:
echo "test2"
python testme.py
The python file is in the same directory as test2.sh but test2.sh's working directory seems to be the root's so it cannot find the python script. So, if I give the python script's absolute location to test2.sh, the python file runs, but when the python file creates new files, it's created in the root.
If there is a solution so that I don't have to edit the python script to create new files in the directory of the python script, please let me know.
Here is your root script:
echo "test0"
cd test/test1
sh ./test2.sh
Your script test2 is launched with the environment of your root script. So you just have to cd in your root script.
im kinda new to the python world and im having some issues running a bash file that will be automatically from my python script (using linux) .
i set my python script to create both a text file .geo and a Bash file .sh in a directory somewhere in my Desktop like this :
basedirectory="/home/pst2/Desktop/";
*//Writing the .geo file*
file = open(basedirectory+nomdossier+"/"+nomfichier+".geo", 'w');
file.write
..blabla
..blabla
file.close();
//Writing the .sh file
file = open(basedirectory+nomdossier+"/"+nomfichier+".sh", 'w');
file.write
..blabla
..blabla
file.close();
Now at this point my script works perfectly with all the variables set up and working fine and both those files that i created find themselves in this directory (for exemple after running the python script and entering the variables)
/home/pst2/Desktop/test/
(and in here you will find the new test.geo and test.sh that were created via the python script)
basically the test.sh when executed "manually" with Bash test.sh ( whenever i am in its directory on ubuntu) will create another file called test.msh in the same directory
and i cant seem to find the right coding , using the subprocess modules to execute the newly created test.sh file automatically from the script .
is there a way to do so , like with indicating the absolute path to the .sh file
(in our case basedirectory+nomdossier+"/"+nomfichier+".sh ) ?
take a look at the os module.
I believe
os.system("command_line_with_args")
could be what your looking for
Not sure what you are writing into the .sh file.
But to start with:
Have you started your .sh-file with the hashbang? #!/bin/sh
Have you modded your file as executable? chmod +x
After you have done this, you should be able to use subprocess module and do something like the example from the manual for subprocess:
subprocess.call([path_to_script+'/script.sh'])
I might have to update this answer if & when new information comes to my attention
Roughly equivalent to "manually" executing bash test.sh with the current directory being the one where test.sh has been written by your posted code is:
from subprocess import call
call(['bash', 'test.sh'], cwd=basedirectory+nomdossier)
I have written a bash script in MAC , which is placed in a directory say xyz.
Now i have a python file in the same directory xyz
We can also say that this python script will always be in the directory where this bash script is
So i want this bash script to be general
/Library/Frameworks/Python.framework/Versions/3.5/bin/python3.5 /Users/labuser/Desktop/Installer/OpenURL_GenericNotification.py Some Arguments
So i want to replace
/Users/labuser/Desktop/Installer/
so that from wherever this script is run python script is automatically calculated by some system variable like $cd
/Users/labuser/Desktop/Installer/ this like $cd in windows not sure how in MAC
Any comments on how to go about this script ??
In a Bash script, the path to the script is stored in the $0 variable.
The dirname command returns the directory portion of a path (filename removed).
You can use the dirname with $0 to get the script's base directory, and cd to that path, and then run the Python script in the current directory:
cd "$(dirname "$0")"
./OpenURL_GenericNotification.py
I have a python script let's name it script1.py. I can run it in the terminal this way:
python /path/script1.py
...
but I want to run like a command-line program:
arbitraryname
...
how can i do it ?
You use a shebang line at the start of your script:
#!/usr/bin/env python
make the file executable:
chmod +x arbitraryname
and put it in a directory on your PATH (can be a symlink):
cd ~/bin/
ln -s ~/some/path/to/myscript/arbitraryname
There are three parts:
Add a 'shebang' at the top of your script which tells how to execute your script
Give the script 'run' permissions.
Make the script in your PATH so you can run it from anywhere.
Adding a shebang
You need to add a shebang at the top of your script so the shell knows which interpreter to use when parsing your script. It is generally:
#!path/to/interpretter
To find the path to your python interpretter on your machine you can run the command:
which python
This will search your PATH to find the location of your python executable. It should come back with a absolute path which you can then use to form your shebang. Make sure your shebang is at the top of your python script:
#!/usr/bin/python
Run Permissions
You have to mark your script with run permissions so that your shell knows you want to actually execute it when you try to use it as a command. To do this you can run this command:
chmod +x myscript.py
Add the script to your path
The PATH environment variable is an ordered list of directories that your shell will search when looking for a command you are trying to run. So if you want your python script to be a command you can run from anywhere then it needs to be in your PATH. You can see the contents of your path running the command:
echo $PATH
This will print out a long line of text, where each directory is seperated by a semicolon. Whenever you are wondering where the actual location of an executable that you are running from your PATH, you can find it by running the command:
which <commandname>
Now you have two options: Add your script to a directory already in your PATH, or add a new directory to your PATH. I usually create a directory in my user home directory and then add it the PATH. To add things to your path you can run the command:
export PATH=/my/directory/with/pythonscript:$PATH
Now you should be able to run your python script as a command anywhere. BUT! if you close the shell window and open a new one, the new one won't remember the change you just made to your PATH. So if you want this change to be saved then you need to add that command at the bottom of your .bashrc or .bash_profile
Add the following line to the beginning script1.py
#!/usr/bin/env python
and then make the script executable:
$ chmod +x script1.py
If the script resides in a directory that appears in your PATH variable, you can simply type
$ script1.py
Otherwise, you'll need to provide the full path (either absolute or relative). This includes the current working directory, which should not be in your PATH.
$ ./script1.py
You need to use a hashbang. Add it to the first line of your python script.
#! <full path of python interpreter>
Then change the file permissions, and add the executing permission.
chmod +x <filename>
And finally execute it using
./<filename>
If its in the current directory,