I am trying to execute a system executable on UNIX with python. I have used op.system() to do this, but really need to use subprocess.call() instead. My op.System call is below:
os.system('gmsh default.msh_timestep%06d* animation_options.geo' %(timestep));
and works fine. It calls the program gmsh and gmsh reads a series of files specified in default.msh_timestep%06d*. I then try to do the equivalent thing with subprocess, but I get errors saying that the files are not there. Below is the subprocesses call:
call(["gmsh", "default.msh_timestep%06d*" %(timestep), "animation_options.geo"],shell=True);
Does anyone know what could be going on here? I'm admittedly a Python noob, so this might be a silly question.
Globbing is done by the shell for you. In Python, you need to do it yourself. You can use glob.glob to get file list that match the pattern:
import glob
call(["gmsh"] + glob.glob("default.msh_timestep%06d*" % (timestep,)) +
["animation_options.geo"])
If you want to use shell=True, pass a string isntead of a list of strings:
call("gmsh default.msh_timestep%06d* animation_options.geo" % (timestep,), shell=True)
Related
I am trying to execute rm command from python in linux as follows
remove_command = [find_executable(
"rm"), "-rf", "dist/", "python_skelton.egg-info", "build/", "other/*_generated.py"]
print('Removing build, dist, python_skelton.egg-
if subprocess.call(remove_command) != 0:
sys.exit(-1)
The directories gets removed successfully but the regex pattern other/*_generated.py
does not remove the relevant _generated.py files.
How shall I remove those files using regex from python script?
The reason this doesn't work the way you intend it to, is that your pattern is not expanded, but interpreted as the litteral file name "other/*_generated.py". This happens because you are relying on so-called glob pattern expansion.
The glob pattern is typically expanded by the shell, but since you are calling the rm command without using the shell, you will not get this "automatically" done. I can see two obvious ways to handle this.
Expand the glob before calling the subprocess
This can be done, using the Python standard library glob implementation:
import glob
remove_command = [find_executable("rm"), "-rf", "dist/", "python_skelton.egg-info",
"build/"] + glob.glob("other/*_generated.py")
subprocess.call(remove_command)
Use the shell to expand the glob
To do this, you need to pass shell=True to the subprocess.call. And, as always, when using the shell, we should pass the command as a single string and not a list:
remove_command = [find_executable("rm"), "-rf", "dist/", "python_skelton.egg-info",
"build/", "other/*_generated.py"]
remove_command_string = " ".join(remove_command) # generate a string from list
subprocess.call(remove_command_string, shell=True)
Both of these approaches will work. Note that if you allow user input, you should avoid using shell=True though, as it is a security hole, that can be used to execute arbitrary commands. But, in the current use case, it seems to not be the case.
I am trying to make a python script that will open a directory, apply a perl script to every file in that directory and get its out put in either multiple text files or just one.
I currently have:
import shlex, subprocess
arg_str = "perl tilt.pl *.pdb > final.txt"
arg = shlex.split(arg_str)
import os
framespdb = os.listdir("prac_frames")
for frames in framespdb:
subprocess.Popen(arg, stdout=True)
I keep getting *.pdb not found. I am very new to all of this so any help trying to complete this script would help.
*.pdb not found means exactly that - there won't be a *.pdb in whatever directory you're running the script... and as I read the code - I don't see anything to imply it's within 'frames' when it runs the perl script.
you probably need os.chdir(path) before the Popen.
How do I "cd" in Python?
...using a python script to run somewhat dubious syscalls to perl may offend some people but everyone's done it.. aside from that I'd point out:
always specify full paths (this becomes a problem if you will later say, want to run your job automatically from cron or an environment that doesn't have your PATH).
i.e. 'which perl' - put that full path in.
./.pdb would be better but not as good as the fullpath/.pdb (which you could use instead of the os.chdir option).
subprocess.Popen(arg, stdout=True)
does not expand filename wildcards. To handle your *.pdb, use shell=True.
I spend a few hours writing a little script.
Basically what it does is create a new text file and fills it up with whatever.
I zip the text file --using zipfile-- and here's where my problem lies.
I want to run the Windows system command:
copy /b "imgFile.jpg" + "zipFile.zip" newImage.jpg
To merge the image "imgFile.jpg" and the zip "zipFile.zip".
So:
os.system("copy /b \"imgFile.jpg\" + \"zipFile.zip\" newImage.jpg")
When I run my script, it all seems to go fine.
But when it's done and I try to extract the 'newImage.jpg' file, it gives me:
The archive is either in unknown format or damaged
This ONLY happens when I run the system command within the script.
It works fine when I use the shell. It even works if I use a separate script.
I've double checked my zip file. Everything is in good shape.
Is there something I'm doing wrong? Something I'm not seeing?
Have you tried using shutil?
import shutil
shutil.copy(src, dst)
There may be a problem with the way Python is passing the arguments to the shell command. Try using subprocess.call. This method takes arguments as an array and passes them that way to the command:
import subprocess
subprocess.call(["copy", "/b", '"imgFile.jpg" + "zipFile.zip"', "newImage.jpg"])
I'm trying to to a ls -l from python, to check for the last modification date of a file.
os.listdir doesn't show the long list format.
subprocess.call shows the format, but actually prints it, and returns 0. I want to be able to put it in a variable. Any ideas ?
Also, I tried
subprocess.call("ls","*.py")
which answers
ls: cannot access *.py: No such file or directory
it works with shell=True, but if someone could explain why it doesn't work without it, I'll appreciate. If you know how to make it work, even better.
It doesn't work without shell=True because the * is a shell expansion character - going from *.py to a list of files ending in .py is a function performed by the shell itself, not ls or python.
If you want to get the output of a command invoked via subprocess, you should use subprocess.check_output() or subprocess.Popen.
ls_output = subprocess.check_output(['ls', '-l'])
With nice formatting:
import subprocess
print(subprocess.check_output(['ls', '-lh']).decode('utf-8'))
I am trying to use os.system() to call another program that takes an input and an output file. The command I use is ~250 characters due to the long folder names.
When I try to call the command, I'm getting an error: The input line is too long.
I'm guessing there's a 255 character limit (its built using a C system call, but I couldn't find the limitations on that either).
I tried changing the directory with os.chdir() to reduce the folder trail lengths, but when I try using os.system() with "..\folder\filename" it apparently can't handle relative path names. Is there any way to get around this limit or get it to recognize relative paths?
Even it's a good idea to use subprocess.Popen(), this does not solve the issue.
Your problem is not the 255 characters limit, this was true on DOS times, later increased to 2048 for Windows NT/2000, and increased again to 8192 for Windows XP+.
The real solution is to workaround a very old bug in Windows APIs: _popen() and _wpopen().
If you ever use quotes during the command line you have to add the entire command in quoates or you will get the The input line is too long error message.
All Microsoft operating systems starting with Windows XP had a 8192 characters limit which is now enough for any decent command line usage but they forgot to solve this bug.
To overcome their bug just include your entire command in double quotes, and if you want to know more real the MSDN comment on _popen().
Be careful because these works:
prog
"prog"
""prog" param"
""prog" "param""
But these will not work:
""prog param""
If you need a function that does add the quotes when they are needed you can take the one from http://github.com/ssbarnea/tendo/blob/master/tendo/tee.py
You should use the subprocess module instead. See this little doc for how to rewrite os.system calls to use subprocess.
You should use subprocess instead of os.system.
subprocess has the advantage of being able to change the directory for you:
import subprocess
my_cwd = r"..\folder\"
my_process = subprocess.Popen(["command name", "option 1", "option 2"], cwd=my_cwd)
my_process.wait() # wait for process to end
if my_process.returncode != 0:
print "Something went wrong!"
The subprocess module contains some helper functions as well if the above looks a bit verbose.
Assuming you're using windows, from the backslashes, you could write a .bat file from python and then os.system() on that. It's a hack.
Make sure when you're using '\' in your strings that they're being properly escaped.
Python uses the '\' as the escape character, so the string "..\folder\filename" evaluates to "..folderfilename" since an escaped f is still an f.
You probably want to use
r"..\folder\filename"
or
"..\\folder\\filename"
I got the same message but it was strange because the command was not that long (130 characters) and it used to work, it just stopped working one day.
I just closed the command window and opened a new one and it worked.
I have had the command window opened for a long time (maybe months, it's a remote virtual machine).
I guess is some windows bug with a buffer or something.