We Keep Coding

Calling a command line utility from Python

I am currently trying to utilize strace to automatically trace a programm 's system calls. To then parse and process the data obtained, I want to use a Python script.
I now wonder, how would I go about calling strace from Python?
Strace is usually called via command line and I don't know of any C library compiled from strace which I could utilize.
What is the general way to simulate an access via command line via Python?
alternatively: are there any tools similar to strace written natively in Python?
I'm thankful for any kind of help.
Nothing, as I'm clueless

You need to use the subprocess module.
It has check_output to read the output and put it in a variable, and check_call to just check the exit code.
If you want to run a shell script you can write it all in a string and set shell=True, otherwise just put the parameters as strings in a list.
import subprocess
# Single process
subprocess.check_output(['fortune', '-m', 'ciao'])
# Run it in a shell
subprocess.check_output('fortune | grep a', shell=True)
Remember that if you run stuff in a shell, if you don't escape properly and allow user data to go in your string, it's easy to make security holes. It is better to not use shell=True.

You can use commands as the following:
import commands
cmd = "strace command"
result = commands.getstatusoutput(cmd)
if result[0] == 0:
print result[1]
else:
print "Something went wrong executing your command"
result[0] contains the return code, and result[1] contains the output.

Python 2 and Python 3 (prior 3.5)
Simply execute:
subprocess.call(["strace", "command"])
Execute and return the output for processing:
output = subprocess.check_output(["strace", "command"])
Reference: https://docs.python.org/2/library/subprocess.html
Python 3.5+
output = subprocess.run(["strace", "command"], caputure_output=True)
Reference: https://docs.python.org/3.7/library/subprocess.html#subprocess.run

scp with Python3 subprocess [duplicate]

When using subprocess.Popen(args, shell=True) to run "gcc --version" (just as an example), on Windows we get this:
>>> from subprocess import Popen
>>> Popen(['gcc', '--version'], shell=True)
gcc (GCC) 3.4.5 (mingw-vista special r3) ...
So it's nicely printing out the version as I expect. But on Linux we get this:
>>> from subprocess import Popen
>>> Popen(['gcc', '--version'], shell=True)
gcc: no input files
Because gcc hasn't received the --version option.
The docs don't specify exactly what should happen to the args under Windows, but it does say, on Unix, "If args is a sequence, the first item specifies the command string, and any additional items will be treated as additional shell arguments." IMHO the Windows way is better, because it allows you to treat Popen(arglist) calls the same as Popen(arglist, shell=True) ones.
Why the difference between Windows and Linux here?

Actually on Windows, it does use cmd.exe when shell=True - it prepends cmd.exe /c (it actually looks up the COMSPEC environment variable but defaults to cmd.exe if not present) to the shell arguments. (On Windows 95/98 it uses the intermediate w9xpopen program to actually launch the command).
So the strange implementation is actually the UNIX one, which does the following (where each space separates a different argument):
/bin/sh -c gcc --version
It looks like the correct implementation (at least on Linux) would be:
/bin/sh -c "gcc --version" gcc --version
Since this would set the command string from the quoted parameters, and pass the other parameters successfully.
From the sh man page section for -c:
Read commands from the command_string operand instead of from the standard input. Special parameter 0 will be set from the command_name operand and the positional parameters ($1, $2, etc.) set from the remaining argument operands.
This patch seems to fairly simply do the trick:
--- subprocess.py.orig 2009-04-19 04:43:42.000000000 +0200
+++ subprocess.py 2009-08-10 13:08:48.000000000 +0200
## -990,7 +990,7 ##
args = list(args)
if shell:
- args = ["/bin/sh", "-c"] + args
+ args = ["/bin/sh", "-c"] + [" ".join(args)] + args
if executable is None:
executable = args[0]

From the subprocess.py source:
On UNIX, with shell=True: If args is a string, it specifies the
command string to execute through the shell. If args is a sequence,
the first item specifies the command string, and any additional items
will be treated as additional shell arguments.
On Windows: the Popen class uses CreateProcess() to execute the child
program, which operates on strings. If args is a sequence, it will be
converted to a string using the list2cmdline method. Please note that
not all MS Windows applications interpret the command line the same
way: The list2cmdline is designed for applications using the same
rules as the MS C runtime.
That doesn't answer why, just clarifies that you are seeing the expected behavior.
The "why" is probably that on UNIX-like systems, command arguments are actually passed through to applications (using the exec* family of calls) as an array of strings. In other words, the calling process decides what goes into EACH command line argument. Whereas when you tell it to use a shell, the calling process actually only gets the chance to pass a single command line argument to the shell to execute: The entire command line that you want executed, executable name and arguments, as a single string.
But on Windows, the entire command line (according to the above documentation) is passed as a single string to the child process. If you look at the CreateProcess API documentation, you will notice that it expects all of the command line arguments to be concatenated together into a big string (hence the call to list2cmdline).
Plus there is the fact that on UNIX-like systems there actually is a shell that can do useful things, so I suspect that the other reason for the difference is that on Windows, shell=True does nothing, which is why it is working the way you are seeing. The only way to make the two systems act identically would be for it to simply drop all of the command line arguments when shell=True on Windows.

The reason for the UNIX behaviour of shell=True is to do with quoting. When we write a shell command, it will be split at spaces, so we have to quote some arguments:
cp "My File" "New Location"
This leads to problems when our arguments contain quotes, which requires escaping:
grep -r "\"hello\"" .
Sometimes we can get awful situations where \ must be escaped too!
Of course, the real problem is that we're trying to use one string to specify multiple strings. When calling system commands, most programming languages avoid this by allowing us to send multiple strings in the first place, hence:
Popen(['cp', 'My File', 'New Location'])
Popen(['grep', '-r', '"hello"'])
Sometimes it can be nice to run "raw" shell commands; for example, if we're copy-pasting something from a shell script or a Web site, and we don't want to convert all of the horrible escaping manually. That's why the shell=True option exists:
Popen(['cp "My File" "New Location"'], shell=True)
Popen(['grep -r "\"hello\"" .'], shell=True)
I'm not familiar with Windows so I don't know how or why it behaves differently.

Execute external program with command line arguments

I want to execute a Python script from AutoIt using ShellExecuteWait(). My Attempt:
$x = ShellExecuteWait("E:/Automation/Python/Scripts/ReadLog.py", '-f "file.log" -k "key" -e "errMsg" ')
MsgBox(0,"x=",String($x))
If #error Then
MsgBox(0,"Error=",String(#error))
EndIf
I can see some process id in $x, and #error also gets set to 0 (means AutoIt executed the script). But my Python script is not producing results (it writes to a txt file when executed independently). Seems the problem is with passing command line arguments like:
ShellExecuteWait("E:/Automation/Python/Scripts/ReadLog.py", '-f "file.log" -k "key" -e "errMsg" ')
How can I pass command line arguments using ShellExecuteWait()? Syntax:
ShellExecuteWait ( "filename" [, "parameters" [, "workingdir" [,"verb" [, showflag]]]] )
Parameters:
filename :- The name of the file to run (EXE, .txt, .lnk, etc).
parameters :- [optional] Any parameters for the program. Blank ("") uses none.
This misses examples for use of parameters. There are no problems with the Python script (it requires 3 command line arguments, strings with options -f, -k and -e).
Related: How to run or execute python file from autoit.

Check the path to your Python binary (e.g. Python.exe, wherever your Python program/binary is located) in your Window's system environment/path.
Execute Python script from AutoIt If path is there then your code must work. In $x you will receive return exit code of the Python script.
Also you can try:
RunWait('full_path\Python.exe ReadLog.py -f "file.log" -k "key" -e "errMsg"', 'full_path_of_working_directory')

AutoIt does not execute external programs/scripts until you pass the working directory (optional parameter to all the execute and run commands). So pass the working directory as a separate parameter and it will work:
RunWait('full_path\Python.exe ReadLog.py -f "file.log" -k "key" -e "errMsg"', 'full_path_of_working_directory')

Fetch PowerShell script from GitHub and execute it

Running into issues executing a PowerShell script from within Python.
The Python itself is simple, but it seems to be passing in \n when invoked and errors out.
['powershell.exe -ExecutionPolicy Bypass -File', '$Username = "test";\n$Password = "password";\n$URL
This is the code in full:
import os
import subprocess
import urllib2
fetch = urllib2.urlopen('https://raw.githubusercontent.com/test')
script = fetch.read()
command = ['powershell.exe -ExecutionPolicy Bypass -File', script]
print command #<--- this is where I see the \n.
#\n does not appear when I simply 'print script'
So I have two questions:
How do I correctly store the script as a variable without writing to disk while avoiding \n?
What is the correct way to invoke PowerShell from within Python so that it would run the script stored in $script?

How do I correctly store the script as a variable without writing to disk while avoiding \n?
This question is essentially a duplicate of this one. With your example it would be okay to simply remove the newlines. A safer option would be to replace them with semicolons.
script = fetch.read().replace('\n', ';')
What is the correct way to invoke PowerShell from within Python so that it would run the script stored in $script?
Your command must be passed as an array. Also you cannot run a sequence of PowerShell statements via the -File parameter. Use -Command instead:
rc = subprocess.call(['powershell.exe', '-ExecutionPolicy', 'Bypass', '-Command', script])

I believe this is happening because you are opening up PowerShell and it is automatically formatting it a specific way.
You could possibly do a for loop that goes through the command output and print without a /n.

Attempting to circumvent pysvn, can I use the subprocess function to write svn calls in shell script?

I'm having some trouble understanding the subprocess function in Python 2.7. I have some commands in shell script that I'm trying to convert into Python, svn export -r 5 ... for example, but I don't want to depend on a library such as pysvn to do this. The solution to that (to my understanding) is to use a subprocess and just run each individual command that would be in a shell script. Should this be achieved by subprocess.call("svn export -r 5", shell=True)? Or is Popen what I should be looking at? I know that it's been said you should avoid shell=True, but there is no security concern or possible user error in my case. Any advice would be appreciated.

subprocess.call is just a thin wrapper around subprocess.Popen that waits for the process to complete:
def call(*args, **kwargs):
return Popen(*args, **kwargs).wait()
The only reason to use the shell to run your command is if you want to run some more or less complicated shell command. With a single simple command and its arguments, it is better to pass a single list of strings consisting of the command name and its arguments.
subprocess.call(["svn", "export", "-r", "5"])
If you were writing a function that could, for example, take a revision number as an argument, you can pass that to svn export as long as you ensure that it is a string:
def svn_export(r):
subprocess.call(["svn", "export", "-r", str(r)])