Consider this command
strings --radix=d --encoding={b,l} abc.exe >> xyz.txt
When I run this on the Ubuntu terminal it works without any problems.
However when I use it through a python code:
import os
os.system("strings --radix=d --encoding={b,l} abc.exe >> xyz.txt")
Its not working.
If I remove the "encoding" then it works fine in both cases.
However, I need to get Unicode strings so that part is necessary.
Anyone have any solutions?
ubuntu uses dash by default as /bin/sh, and bash for login shells.
So in your terminal --encoding={b,l} is propably expanded by bash to --encoding=b --encoding=l, while dash (probably called by os.system as /bin/sh) does no such expansion and it remains --encoding={b,l}
The easiest way is to explicitly expand the encoding parameter and don't leave that to the shell, then it will work with any shell.
And you should use the subprocess module instead of os.system(). Just be aware that when using the shell=True argument, it will also call the default /bin/sh, which isn't guaranteed to be bash.
You don't need shell=True, you can pass a list of args and write the stdout to a file:
from subprocess import check_call
with open('xyz.txt',"a") as out:
check_call(['strings', '--radix=d', '--encoding={b,l}', 'abc.exe'],stdout=out)
A good explanation here of what shell=True does
os.system is outdated, use subprocess instead. u also should use shell=True to have sh like behaviour:
import subprocess
cmd = "strings --radix=d --encoding={b,l} abc.exe >> xyz.txt"
subprocess.check_call(cmd, shell=True)
It also throws exception in case of call failure!
Related
I am a beginner in Python and can you kindly help me understand the following concept.
If I do the following,
import subprocess
subprocess.run(['ls'])
Here we know that the key word argument, shell is set to False by default so that the 'ls' does not run on the shell. But my question is if it does not run on the shell, on where does it run and how can it give me an output?
I have a windows system but it should work the same.
For getting the output of subprocess you can use check_output.
On windows -
import subprocess
subprocess.check_output(["dir"], shell=True)
running this code without shell=True will result in an error.
If I want to run the code above with shell=False
I would do something like this -
subprocess.check_output(["cmd","/c","dir"], shell=False)
Notice -
On Unix with shell=True, the shell defaults to /bin/sh.
That means when you pass command and use shell=True
it will use /bin/sh to run that command.
I know you can run Linux terminal commands through Python scripts using subprocess
subprocess.call(['ls', '-l']) # for linux
But I can't find a way to do the same thing on windows
subprocess.call(['dir']) # for windows
is it possible using Python without heavy tinkering?
Should I stick to good old fashioned batch files?
dir is not a file, it is an internal command, so the shell keyword must be set to True.
subprocess.call(["dir"], shell=True)
Try this
import os
os.system("windows command")
ex: for date
os.system("date")
Almost everyone's answers are right but it seems I can do what I need using os.popen -- varStr = os.popen('dir /b *.py').read()
First of all, to get a directory listing, you should rather use os.listdir(). If you invoke dir instead, you'll have to parse its output to make any use of it, which is lots of unnecessary work and is error-prone.
Now,
dir is a cmd.exe built-in command, it's not a standalone executable. cmd.exe itself is the executable that implements it.
So, you have two options (use check_output instead of check_call if you need to get the output instead of just printing it):
use cmd's /C switch (execute a command and quit):
subprocess.check_call(['cmd','/c','dir','/s'])
use shell=True Popen() option (execute command line through the system shell):
subprocess.check_call('dir /s', shell=True)
The first way is the recommended one. That's because:
In the 2nd case, cmd, will do any shell transformations that it normally would (e.g. splitting the line into arguments, unquoting, environment variable expansion etc). So, your arguments may suddenly become something else and potentially harmful. In particular, if they happen to contain any spaces and cmd special characters and/or keywords.
shell=True uses the "default system shell" (pointed to via COMSPEC environment variable in the case of Windows), so if the user has redefined it, your program will behave unexpectedly.
I wrote a simple piece of code:
import subprocess
p=subprocess.Popen('mkdir -p ./{a,b,c}', shell=True, stderr=subprocess.STDOUT)
p.wait()
Unfortunately, it not always behaves the way I'd expect. I.e, when I run it on my PC, everything is OK (ls -l gives me three dirs: a, b and c). But when my colleague runs it on his desktop, he gets... one dir named: '{a,b,c}' ... We both use Python 2.7.3. Why is that? How would you fix it?
I tried to find the answer by myself. According to manual:
"args should be a sequence of program arguments or else a single string. By default, the program to execute is the first item in args if args is a sequence. If args is a string, the interpretation is platform-dependent and described below. See the shell and executable arguments for additional differences from the default behavior. Unless otherwise stated, it is recommended to pass args as a sequence."
So I tried to execute the code in shell:
python -c "import subprocess; p=subprocess.Popen(['mkdir', '-p', './{ea,fa,ga}'], shell=True, stderr=subprocess.STDOUT); p.wait()"
And I got:
mkdir: missing operand
I will be thankful for any advice
Thanks!
The ./{a,b,c} syntax is bash syntax, not supported by all shells.
The documentation says:
On Unix with shell=True, the shell defaults to /bin/sh. If args is a
string, the string specifies the command to execute through the shell.
So your command only works if /bin/sh is symlinked to a shell that supports that syntax, like bash or zsh. Your colleague is probably using dash or another shell that doesn't support this.
You should no be relying in something like a user's default shell. Instead, write the full command with the full expansion:
p = subprocess.Popen('mkdir -p ./a ./b ./c', shell=True, stderr=subprocess.STDOUT)
There's several problems here.
First: if you are using a sequence of arguments, do not set "shell = True" (this is recommended in the Popen manual). Set it to False, and you'll see that your mkdir command will be accepted.
"./{a,b,c}" is AFAIK a specific syntax in bash. If your colleague is using a different shell, it will probably not work, or behave differently.
You should use the python "mkdir" command instead of calling a shell command, it will work whatever the server / shell / OS.
Thank you all for your answers.
It seems, that the best way is simply use /bin/sh syntax. I changed my code to use:
'mkdir -p ./a ./b ./c'
as you suggested.
I avoided to use mkdir() function, because I am writing a scripts with plenty of system calls, and I wanted to provide elegant --dry-run option (so I could list all of the commands).
Problem solved - thank you!
The os.mkdir(path,[mode]) method are as far as I understand safer to use when working on multiplatform projects.
os.mkdir(os.getcwd()/a)
However its not as elegant as taking the subprocess approach.
I am looking to open a Python script in edit mode in IDLE by passing in a directory from another Python script. I understand that I can use os.system to execute idle.py but I then don't know how to pass the appropriate -e parameter and get it to use a specific directory i.e. open something.py in diredtory C:\Python all from the original Python script.
Thanks for your help,
Ben
You can use subprocess.call(). By setting shell = True, the function treats the string as a literal shell command. Modify the paths as you see fit.
import subprocess
subprocess.call(r'C:\Python27\Lib\idlelib\idle.py -e C:\Python27\something.py',
shell=True)
I would like to invoke multiple commands from my python script.
I tried using the os.system(), however, I'm running into issues when the current directory is changed.
example:
os.system("ls -l")
os.system("<some command>") # This will change the present working directory
os.system("launchMyApp") # Some application invocation I need to do.
Now, the third call to launch doesn't work.
os.system is a wrapper for the C standard library function system(). Its argument can be any valid shell command as long as it fits into the memory reserved for environment and argument lists of a process.
So, delimit each command with a semicolon or a newline and they will be executed one after another in the same environment.
os.system(" ls -l; <some command>; launchMyApp")
os.system('''
ls -l
<some command>
launchMyApp
''')
Try this
import os
os.system("ls -l")
os.chdir('path') # This will change the present working directory
os.system("launchMyApp") # Some application invocation I need to do.
Each process has its own current working directory. Normally, child processes can't change parent's directory that is why cd is a builtin shell command: it runs in the same (shell) process.
Each os.system() call creates a new shell process. Changing the directory inside these processes has no effect on the parent python process and therefore on the subsequent shell processes.
To run multiple commands in the same shell instance, you could use subprocess module:
#!/usr/bin/env python
from subprocess import check_call
check_call(r"""set -e
ls -l
<some command> # This will change the present working directory
launchMyApp""", shell=True)
If you know the destination directory; use cwd parameter suggested by #Puffin GDI instead.
It’s simple, really.
For Windows separate your commands with &, for Linux, separate them with ;.
str.replace is a very good way to approach the problem, used in the example below:
import os
os.system('''cd /
mkdir somedir'''.replace('\n', ';')) # or use & for Windows
When you call os.system(), every time you create a subshell - that closes immediately when os.system returns (subprocess is the recommended library to invoke OS commands). If you need to invoke a set of commands - invoke them in one call.
BTW, you may change working director from Python - os.chdir
Try to use subprocess.Popen and cwd
example:
subprocess.Popen('launchMyApp', cwd=r'/working_directory/')
os.system("ls -l && <some command>")
You can change back to the directory you need to be in with os.chdir()
Just use
os.system("first command\nsecond command\nthird command")
I think you have got the idea what to do
Note: This is not a very reliable approach if you are doing a complex
job using CLI tools. Popen and subprocess methods are better there.
Although small task link copy, move, list will work fine
.