python subprocess and empty strings - python

I am trying to use the .subprocess.check_output() function to issue commands to a crypto-coin wallet (like bitcoin, but not). Some of the commands work great, but one does not return the correct results. I suspect the use of an empty string ("") is the problem, but need help finding a solution.
I am using Python 2.7 on a Raspberry Pi B+.
My code is:
import subprocess
command = 'rimbitd getaddressesbyaccount ""'
print "The command is: ", command
results = subprocess.check_output(command.split())
print "The Results are: ", results
My output is:
pi#Pi ~ $ python Demo2.py
The command is: rimbitd getaddressesbyaccount ""
The Results are: [
]
Obviously the results are an empty set of brackets. But, if I use the exact same command directly (at the $ prompt) by copying/pasting the rimbitd getaddressesbyaccount "" from the output (i.e. "The command is: ...") then I get the following (correct) output:
pi#Pi ~ $ rimbitd getaddressesbyaccount ""
[
"RPSXNdNu4TsfyrytMSFz5RvxeSqrRyQMg6",
"RGyFKcrYR6NGr82Gqsjamj9DgEj7ZBUyfM",
"RLwQ9XGPesV35KK84oUupUh62tNori9Ahe",
"RQ2HRzX3U9NGP14dLAM7TPagNPhYKbTaVM",
"RNvR2vvaWmUnij88tTiGTXFLgZU1daKVvz"
]
pi#Pi ~ $
Other commands work as described (sending coins, checking the balance, etc.). I just can't get this one to work and the only difference is that the non-working command needs to send an empty string. I have tried using n = "" and then concatenating: commands + n (no joy). I have tried using:
results = subprocess.check_output("rimbitd", "getaddressesbyaccount", " \"\"").
I have tried using a number of methods to get the empty string through, but none have worked.

Splitting 'rimbitd getaddressesbyaccount ""' doesn't give you an empty third parameter; it gives you a parameter consisting of two quotes:
>>> 'rimbitd getaddressesbyaccount ""'.split()
['rimbitd', 'getaddressesbyaccount', '""']
You want to use
['rimbitd', 'getaddressesbyaccount', '']
for this if the last parameter needs to be empty (for which you would use a pair of double quotes when invoked from the shell prompt).
As J.F.Sebastian indicated, you can also use the shlex module:
>>> import shlex
>>> shlex.split('rimbitd getaddressesbyaccount ""')
['rimbitd', 'getaddressesbyaccount', '']
subprocess.check_output() by default doesn't run the commands in a shell. You could also have used your original .split() and provide check_output() with the shell=True keyword argument.

Related

Python subprocess.call() gives an error but same call in shell executes correctly [duplicate]

For example I am using ffplay and want to run this command -bufsize[:stream_specifier] integer (output,audio,video)
At the moment I have this:
subprocess.call(["ffplay", "-vn", "-nodisp","-bufsize 4096", "%s" % url])
But this says it is invalid.
As JBernardo mentioned in a comment, separate the "-bufsize 4096" argument into two, "-bufsize", "4096". Each argument needs to be separated when subprocess.call is used with shell=False (the default). You can also specify shell=True and give the whole command as a single string, but this is not recommended due to potential security vulnerabilities.
You should not need to use string formatting where you have "%s" % url. If url is a string, pass it directly, otherwise call str(url) to get a string representation.
This is the way to go:
url = 'http://www.whatever.com'
cmd = 'ffplay -vn -nodisp -bufsize 4096 '.split()
subprocess.call(cmd + [str(url)], shell=False)
While using shlex.split() is overkill for your use case, many of the comments seem to be asking about the use of spaces in parameters in cases where a CLI allows you to pass in quoted strings containing spaces (i.e. git commit -m "Commit message here").
Here is a quick python function that can be used to run commands including parameters with spaces:
import shlex, subprocess
def run_command( command ):
subprocess.call(shlex.split(command))

PYTHON Script not using OS Commands

Within a UBUNTU VM, using GNS3 I created code that is an attempt to after the user's input perform one of 3 different outcomes, however, the if statements don't work, the python files can't be found which I was trying to point to this the cd/home.. command. And the curl commands are apparently the incorrect syntax even though that is what I would enter for them to work. please help me out and get this working.
This is what I tried:
#!/usr/bin/python3
import os
import subprocess
Code = input("Enter RYU, ONOS or CURL:")
print("Command entered was: " + Code)
if input == 'RYU':
os.system('rest_router.py')
os.system('gui_topology.py')
elif input == "ONOS":
os.system('sudo /opt/onos/bin/onos-service start')
You are using single quotes to quote something that already has single quotes. By doing so, what should be an opening quote in your curl command is now effectively a closing quote in your Python, and Python doesn't understand why there is now a random ( there where Python code should continue.
I underlined what is quoted in the following examples. Note that even syntax highlighting in most any editor (and also here on Stack Overflow) is helping you see what is inside a string and what is not, colouring them differently (though syntax highlighting can be fallible):
echo 'foo' bar
---
But:
os.system('echo 'foo' bar')
----- ----
To fix, you can escape the inner quotes, so Python treats them as any regular character inside the string:
os.system('echo \'foo\' bar')
----------------
Or you can change the outer quotes. Python has several sets of quotes; you can't use ' or " since you're already using both of them inside the string, but you can use ''' or """:
os.system('''echo 'foo' bar''')
--------------

How to send a string with spaces from Python to Bash sub-process as a single value?

I'm trying to send a variable from a python script to a bash script. I'm using popen like as shown below:
subprocess.Popen(["bash", "-c", ". mainGui_functions.sh %d %s" % (commandNum.get(), entryVal)])
However, entryVal can sometimes contain one or more white space characters. In that case I divide the string into multiple arguments ($2,$3..)
How can i get it in one argument?
Simple solution #1: You do it the exact same way you'd do it if you were typing the input on the commandline; put it in quotes:
subprocess.Popen(["bash", "-c", ". mainGui_functions.sh {} '{}'".format(commandNum.get(), entryVal)])
Simple solution #2: If mainGui_functions.sh is already executable, then you can just omit the bash part and pas args to it directly. In this case, subprocess takes care of making sure an entry with whitespace winds up as a single arg for you:
subprocess.Popen(["mainGui_functions.sh", str(commandNum.get()), entryVal])
Put quotes around it in your bash command line – e.g.,
subprocess.Popen(['bash', '-c', '. mainGui_functions.sh %d "%s"' % (commandNum.get(), entryVal)])

How to return multiple variables from python to bash

I have a bash script that calls a python script. At first I was just returning one variable and that is fine, but now I was told to return two variables and I was wondering if there is a clean and simple way to return more than one variable.
archiveID=$(python glacier_upload.py $archive_file_name $CURRENTVAULT)
Is the call I make from bash
print archive_id['ArchiveId']
archive_id['ArchiveId']
This returns the archive id to the bash script
Normally I know you can use a return statement in python to return multiple variables, but with it just being a script that is the way I found to return a variable. I could make it a function that gets called but even then, how would I receive the multiple variables that I would be passing back?
From your python script, output one variable per line. Then from you bash script, read one variable per line:
Python
print "foo bar"
print 5
Bash
#! /bin/bash
python main.py | while read line ; do
echo $line
done
Final Solution:
Thanks Guillaume! You gave me a great starting point out the soultion. I am just going to post my solution here for others.
#! /bin/bash
array=()
while read line ; do
array+=($line)
done < <(python main.py)
echo ${array[#]}
I found the rest of the solution that I needed here
The safest and cleanest way to parse any input effectively in bash is to map into an array,
mapfile -t -d, <<<"$(python your_script.py)"
Now you just need to make sure you script outputs the data you want to read with the chosen delimiter, "," in my example (-d selects a delimiter, -t trims input like newlines). The quotes are non-optional to ensure the shell doesn't separate things with spaces.
If you have a tuple of things that do not contain commas, this would be enough:
print(str(your_tuple).strip('()'))
Below some easy ways for easy input, before I was more familiar with Bash:
My favorite way is reading straight into a list:
x=($(python3 -c "print('a','b','c')"))
echo ${x[1]}
b
echo ${x[*]}
a b c
for this reason if my_python_function returns a tuple, I would use format to make sure I just get space delimited results:
#Assuming a tuple of length 3 is returned
#Remember to quote in case of a space in a single parameter!
print('"{}" "{}" "{}"'.format(*my_python_function())
If you want this to be generic you would need to construct the format string:
res = my_python_function()
print(("{} "*len(res)).format(*res))
is one way. No need to worry about the extra space, but you could [:-1] on the format string to get rid of it.
Finally, if you are expecting multi-word arguments (i.e. a space in a single argument, you need to add quotes, and a level of indirection (I am assuming you will only be running your own, "safe", scripts):
#myfile.py
res = my_python_function()
print(('"{}" '*len(res)).format(*res))
#my file.bash
eval x=($(python3 myfile.py))

python subprocess calling bash script - need to print the quotes out too

I'm having a problem with subprocess and printing quotes.
My Python script takes user input, mashes it around a bit - and I need it to send it's results to a bash script in this manner.
myscript.sh 'var1 == a var2 == b; othervar == c' /path/to/other/files
Where I'm getting hung up on is the single quotes. Python tries to rip them out.
I used this for my test.
subprocess.Popen([myscript.sh 'var=11; ignore all' /path/to/files], shell=True, executable="/bin/bash")
which returns an invalid syntax pointing at the 2nd single quote. I've also tried the above without the brackets and using single quotes outside and double quotes inside, etc.
Other - would-like.
As I was saying above the 'var == a var == b; othervar == c' is derived from the python script (in string format) - and I'll need to call that in the subprocess like this.
subprocess.Popen([myscript.sh myvariables /path/to/files], shell=True, executable="/bin/bash")
I just have to put the single quotes around the value of myvariables like the first example.
Any pointers as to where I'm going off the correct method?
Thank you.
When shell=True is passed to Popen, you pass whatever you would send on the command line. That means your list should only have one element. So for example:
subprocess.Popen(['myscript.sh "var=11; ignore all" /path/to/files'], shell=True, executable="/bin/bash")
Or if /path/to/files is a variable in your Python environment:
subprocess.Popen(['myscript.sh "var=11; ignore all" %s' % path_to_files], shell=True, executable="/bin/bash")
Having said that I STRONGLY encourage you not to use the shell argument. The reason is fragility. You'll get a much more robust way of doing it like this:
subprocess.Popen(["/bin/bash", "myscript.sh", "var=11; ignore all", path_to_files])
Note that "var=11; ignore all" is passed as one argument to your script. If those are separate arguments, make them separate list elements.
I haven't checked why this works, but it does and without the need for shell=True.
subprocess.Popen(["/bin/bash", "myscript.sh", '""' + string_to_be_quoted + '""', path_to_files])
That's a list, those are strings in it, so they need quotes:
["myscript.sh", "var=11; ignore all", "/path/to/files"]
That should work. If your script really somehow relies on quotes, then try this (I don't know the details of how subprocess works):
["myscript.sh", "'var=11; ignore all'", "/path/to/files"]

Categories