I have written a shell in python (using the cmd module) and when using this shell, certain information about commands run ect. is outputted to a text file. I would like to write a piece of code that creates an alias to view the text file in my bash shell (eg as if I had executed the following command:
alias latest.trc='less <path to file>
My best current effort is:
x="alias %s = 'less %s'" % (<alias name>, <path>)
y=shlex.split(x)
atexit.register(subprocess.call, y)
This works to execute some commands on exit, eg if x="echo 'bye'" but I get the error
OSError: [Errno 2] No such file or directory
The path I am entering is certainly valid and if I run x directly from the command line it works.
Any help either with why my code is hitting an error or a better way to achieve the same effect would be greatly appreciated.
You get an error because alias is a bash command, not an executable. You must run subprocess.call inside a shell:
import subprocess
import atexit
import shlex
x="alias %s = 'less %s'" % ('a', 'whatever')
y=shlex.split(x)
atexit.register(lambda args: subprocess.call(args, shell=True), y)
But, given the fact that subprocess spawns a new process, whatever changes are made to the environment it runs into, are lost on exit.
You'd better instead create a file where you put all these aliases and source it by hand when you need to use them.
Related
I have a bash script that I can run flawlessly in my Rpi terminal in its folder:
./veye_mipi_i2c.sh -r -f mirrormode -b 10
it works like this: Usage: ./veye_mipi_i2c.sh [-r/w] [-f] function name -p1 param1 -p2 param2 -b bus
options:
-r read
-w write
-f [function name] function name
-p1 [param1] param1 of each function
-p2 [param1] param2 of each function
-b [i2c bus num] i2c bus number
When I try to run it in Python (2) via my Spyder editor with os.system, I get a "0" return which I interpret as "succesfully executed" but in fact the script has not been executed and the functions have not been performed. I know this because the script is suppose to change the camera functioning and by checking the images I take afterwards, I can see that nothing has changed.
import os
status = os.system('/home/pi/VeyeMipi/Camera_Folder/veye_mipi_i2c.sh -w -f mirrormode -p1 0x04 -b 10')
print status
Any idea, what causes this? The bash script uses two other scripts that lie in the same folder location (read and write). Could it be, that it cannot execute these additional scripts when startet through Python? It does not make sense to me, but so do a lot of things....
Many thanks
Ok, I understand that my question was not exemplary because of the lack of a minimal reproducible example, but as I did not understand what the problem was, I was not able to create one.
I have found out, what the problem was. The script I am calling in bash requires two more scripts that are in the same folder. Namely the "write" script and "read" script. When executing in terminal in the folder, no problem, because the folder was the working directory.
I tried to execute the script within Spyder editor and added the file location to the PATH in the user interface. But still it would not be able to execute the "write" script in the folder.
Simply executing it in the terminal did the trick.
It would help if you fix your scripts so they don't depend on the current working directory (that's a very bad practice).
In the meantime, running
import subprocess
p = subprocess.run(['./veye_mipi_i2c.sh', '-r', '-f', 'mirrormode', '-b', '10'], cwd='/home/pi/VeyeMipi/Camera_Folder')
print(p.returncode)
which changes the directory would help.
Use subprocess and capture the output:
import subprocess
output = subprocess.run(stuff, capture_output=True)
Check output.stderr and output.stdout
I am creating a script to run shell commands for simulation purposes using a web app. I want to run a shell command in a django app and then save the output to a file.
The problem I am facing is that when running the shell command, the output tries to get saved in the url that is invoked (for example: localhost:8000/projects) which is understandable.
I want to save the output to for example:
/home/myoutput/output.txt rather than /projects or /tasks
I have to run a whole script and save it's output to the txt file later but that is easy once this is done.
Tried os.chdir() function to change directory to /desiredpath already
from subprocess import run
#the function invoked from views.py
def invoke_mpiexec():
run('echo "this is a test file" > fahadTest.txt')
FileNotFoundError at /projects
Exception Type: FileNotFoundError
First I want to say that directly calling external programs from a web request in Django is a bit of an anti-pattern. The preferred approach is to use a work queue like Celery or rq, but that comes with a bit of added complexity.
That being said, you can solve your problem with the argument shell=True:
from subprocess import run
#the function invoked from views.py
def invoke_mpiexec():
run('echo "this is a test file" > fahadTest.txt', shell=True)
Here is the documentation:
If shell is True, the specified command will be executed through the
shell. This can be useful if you are using Python primarily for the
enhanced control flow it offers over most system shells and still want
convenient access to other shell features such as shell pipes,
filename wildcards, environment variable expansion, and expansion of ~
to a user’s home directory. However, note that Python itself offers
implementations of many shell-like features (in particular, glob,
fnmatch, os.walk(), os.path.expandvars(), os.path.expanduser(), and
shutil).
Note: Using shell=True can lead to security issues:
If the shell is invoked explicitly, via shell=True, it is the
application’s responsibility to ensure that all whitespace and
metacharacters are quoted appropriately to avoid shell injection
vulnerabilities.
You should use subprocess.call with stdout argument
def invoke_mpiexec():
f = open("fahadTest.txt", "w")
subprocess.call(['echo', '"this is a test file"'], stdout=f)
or use write function
def invoke_mpiexec():
f = open('fahadTest.txt', 'w')
f.write("Now the file has more content!")
f.close()
So I figured it out.
Below is the fix:
run('mkdir -p $HOME/phdata/test/ && echo "this is a test file" > $HOME/phdata/test/fahadTest.txt', shell=True)
mkdir -p creates a directory if it doesn't exist
$HOME is used to go to the home directory and from there you can
navigate to folders.
shell=True argument is required to run it as shell command
You can also create a ssh connection and run the commands/ scripts on the remote server. For this, my approach will be to create a script on the remote server, call it through my app and provide arguments to it. Another workaround which is not that good but works is to create a script on the server using the above line and then call it.
I am creating a bash script which calls a python script that in turn runs other processes in bash using subprocess.run(). However, when the bash script runs the python script within it, in the line where subprocess.run is called, I get an error message:
run_metric = subprocess.run(command, shell=True, stdout = subprocess.PIPE, universal_newlines = True)
AttributeError: 'module' object has no attribute 'run'
1) I made sure I ran the script using python 3 by activating a conda environment with python=3.6, which should not bring me any problem to call subprocess.run. The interesting thing is that if I change subprocess.run() to subprocess.Popen() the script works, but I could not work out how to get run_metric.stdout properly.
2) I do not have any subprocess.py file within any directory I am working in
3) the result of print(subprocess.__file__) is showing me that python is not 3.6: /usr/lib/python2.7/subprocess.pyc
Also, I tried to use something like
from subprocess import run
and making sure in both the python script and the function I had import subprocess
The bash script is as follows:
SWC_FOLDER_PATH=$(pwd)
sudo chmod +x /media/leandroscholz/KINGSTON/Results_article/Tracing_data/run_metrics.py
echo "run /media/leandroscholz/Tracing_data/run_metrics.py ${SWC_FOLDER_PATH} /media/leandroscholz/KINGSTON/Results_article/TREEStoolbox_tree_fixed.swc"
python /media/leandroscholz/Tracing_data/run_metrics.py ${SWC_FOLDER_PATH} /media/leandroscholz/TREEStoolbox_tree_fixed.swc
And the python script I run calls a certain function that uses subprocess.run() this way (just part of the code where the problem arises):
import subprocess
import glob
import numpy as np
def compute_metrics(swc_folder_path, gt_file_path):
# first get list of files in swc_folder_path
swc_files = (glob.glob(swc_folder_path+"/*_fixed.swc"))
n_swc_files = len(swc_files)
workflow_dict = gets_workflow_dict(swc_files)
n_images = get_n_images(swc_files)
n_workflows = len(workflow_dict)
for swc in range(0,n_swc_files):
command = "java -jar /home/leandroscholz/DiademMetric.jar -G " + swc_files[swc] +" -T " + gt_file_path
run_metric = subprocess.run(command, shell=True, stdout = subprocess.PIPE, universal_newlines = True)
I am using subprocess.run within python because, in the end, I want to get a string of the run_metric.stdout after running the process in bash so I can later store it in an array and save it to a txt file.
I hope I was sufficiently clear and provided enough information.
Thanks!
After the comments received, I tested the output of print(subprocess.__file__), which showed that python being used was python2.7,
Thus, I changed the call of the python script from python script.py to python3 script.py. I've found this question, which also shows another way to call python programs from terminal.
Running Python File in Terminal
Not sure if this is possible. I have a set of python scripts and have modified the linux PATH in ~/.bashrc so that whenever I open a terminal, the python scripts are available to run as a command.
export PATH=$PATH:/home/user/pythonlib/
my_command.py resides in the above path.
I can run my_command.py (args) from anywhere in terminal and it will run the python scripts.
I'd like to control this functionality from a different python script as this will be the quickest solution to automating my processing routines. So I need it to open a terminal and run my_command.py (args) from within the python script I'm working on.
I have tried subprocess:
import subprocess
test = subprocess.Popen(["my_command.py"], stdout=subprocess.PIPE)
output = test.communicate()[0]
While my_command.py is typically available in any terminal I launch, here I have no access to it, returns file not found.
I can start a new terminal using os then type in my_command.py, and it works
os.system("x-terminal-emulator -e /bin/bash")
So, is there a way to get the second method to accept a script you want to run from python with args?
Ubuntu 16
Thanks :)
Popen does not load the system PATH for the session you create in a python script. You have to modify the PATH in the session to include the directory to your project like so:
someterminalcommand = "my_command.py (args)"
my_env = os.environ.copy()
my_env["PATH"] = "/home/usr/mypythonlib/:" + my_env["PATH"]
combine = subprocess.Popen(shlex.split(someterminalcommand), env=my_env)
combine.wait()
This allows me to run my "my_command.py" file from a different python session just like I had a terminal window open.
If you're using Gnome, the gnome-terminal command is rather useful in this situation.
As an example of very basic usage, the following code will spawn a terminal, and run a Python REPL in it:
import subprocess
subprocess.Popen(["gnome-terminal", "-e", "python"])
Now, if you want to run a specific script, you will need to concatenate its path with python, for the last element of that list it the line that will be executed in the new terminal.
For instance:
subprocess.Popen(["gnome-terminal", "-e", "python my_script.py"])
If your script is executable, you can omit python:
subprocess.Popen(["gnome-terminal", "-e", "my_script.py"])
If you want to pass parameters to your script, simply add them to the python command:
subprocess.Popen(["gnome-terminal", "-e", "python my_script.py var1 var2"])
Note that if you want to run your script with a particular version of Python, you should specify it, by explicitly calling "python2" or "python3".
A small example:
# my_script.py
import sys
print(sys.argv)
input()
# main.py
import subprocess
subprocess.Popen(["gnome-terminal", "-e", "python3 my_script.py hello world"])
Running python3 main.py will spawn a new terminal, with ['my_script.py', 'hello', 'world'] printed, and waited for an input.
The python script I would use (source code here) would parse some arguments when called from the command line. However, I have no access to the Windows command prompt (cmd.exe) in my environment. Can I call the same script from within a Python console? I would rather not rewrite the script itself.
%run is a magic in IPython that runs a named file inside IPython as a program almost exactly like running that file from the shell. Quoting from %run? referring to %run file args:
This is similar to running at a system prompt python file args,
but with the advantage of giving you IPython's tracebacks, and of
loading all variables into your interactive namespace for further use
(unless -p is used, see below). (end quote)
The only downside is that the file to be run must be in the current working directory or somewhere along the PYTHONPATH. %run won't search $PATH.
%run takes several options which you can learn about from %run?. For instance: -p to run under the profiler.
If you can make system calls, you can use:
import os
os.system("importer.py arguments_go_here")
You want to spawn a new subprocess.
There's a module for that: subprocess
Examples:
Basic:
import sys
from subprocess import Popen
p = Popen(sys.executable, "C:\test.py")
Getting the subprocess's output:
import sys
from subprocess import Popen, PIPE
p = Popen(sys.executable, "C:\test.py", stdout=PIPE)
stdout = p.stdout
print stdout.read()
See the subprocess API Documentation for more details.