Here is a simple script running subprocess that retrieves IP from the ifconfig command output from the terminal.
I have noticed that subprocess.check_output() always returns a value with \n.
I desire to get a return value without \n.
How can this be done?
$ python
>>> import subprocess
>>> subprocess.check_output("ifconfig en0 | awk '{ print $2}' | grep -E -o '([0-9]{1,3}[\.]){3}[0-9]{1,3}'", shell=True)
'129.194.246.245\n'
For a generic way :
subprocess.check_output("echo hello world", shell=True).strip()
subprocess.check_output() does not add a newline. echo does. You can use the -n switch to suppress the newline, but you have to avoid using the shell built-in implementation (so use /bin/echo):
>>> import subprocess
>>> subprocess.check_output('/bin/echo -n hello world', shell=True)
'hello world'
If you use echo -n instead, you could get the string '-n hello world\n', as not all sh implementations support the -n switch support echo (OS X for example).
You could always use str.rstrip() or str.strip() to remove whitespace, of course, but don't blame subprocess here:
>>> subprocess.check_output('echo hello world', shell=True).rstrip('\n')
'hello world'
Your question update added a more complex example using awk and grep:
subprocess.check_output("ifconfig en0 | awk '{ print $2}' | grep -E -o '([0-9]{1,3}[\.]){3}[0-9]{1,3}'", shell=True)
Here grep adds the (final) newline. grep -o may print just the matching text, but still adds a newline to separate matches. See the grep manual:
-o
--only-matching
Print only the matched (non-empty) parts of matching lines, with each such part on a separate output line.
Emphasis mine.
You can add a tr -d '\n' at the end to remove any newlines from the output of your pipe:
>>> subprocess.check_output(
... "ifconfig en0 | awk '{ print $2}' | "
... "grep -E -o '([0-9]{1,3}[\.]){3}[0-9]{1,3}' | "
... "tr -d '\n'", shell=True)
'172.17.174.160'
You can str.rstrip any newline or use what Martijn suggests, you can also parse the output using python with the need to awk or grep which won't add any newlines:
You can split:
out = subprocess.check_output(["ifconfig", "en0"])
for line in out.splitlines():
if line.lstrip().startswith("inet "):
print(line.split()[1].split(":", 2)[1])
print(ip.search(line))
break
Or use your own regex:
import re
out = subprocess.check_output(["ifconfig", "en0"])
print(re.search('([0-9]{1,3}[\.]){3}[0-9]{1,3}', out).group())
The point being you don't need awk or grep.
If you want to match ipv4 or ipv6 and also catch when there is an error returned i.e no such interface you can catch a CalledProcessError which will be raised for any non zero exit status, it is easy use the regex for ipv4 but for ipv6 it is simpler to use inet6 to grab the ipv6 address.
from subprocess import check_output, CalledProcessError
import re
def get_ip(iface, ipv="ipv4"):
try:
out = check_output(["ifconfig", iface])
except CalledProcessError as e:
print(e.message)
return False
try:
if ipv == "ipv4":
return re.search('([0-9]{1,3}[\.]){3}[0-9]{1,3}', out).group()
return re.search("(?<=inet6 addr:)(.*?)(?=/)", out).group().lstrip()
except AttributeError as e:
print("No {} address for interface {}".format(ipv, iface))
return False
Demo:
In [2]: get_ip("wlan0")
Out[2]: '192.168.43.168'
In [3]: get_ip("wlan0","ipv6")
Out[3]: 'fe80::120b:a9ff:fe03:bb10'
In [4]: get_ip("wlan1","ipv6")
wlan1: error fetching interface information: Device not found
Out[4]: False
This is what you have :
$ python
>>> import subprocess
>>> subprocess.check_output("ifconfig eth0 | awk '{ print $2}' | grep -E -o '([0-9]{1,3}[\.]){3}[0-9]{1,3}'", shell=True)
'172.31.94.116\n'
Try this instead :
$ python
>>> import subprocess
>>> subprocess.check_output("ifconfig eth0 | awk '{ print $2}' | grep -E -o '([0-9]{1,3}[\.]){3}[0-9]{1,3}'", shell=True).strip()
'172.31.94.116'
Related
This is a bash command that I run in python and get the expected result:
count = subprocess.Popen("ps -ef | grep app | wc -l", stdout=subprocess.PIPE, shell=True)
but when I'd like to pass an argument (count in this case) cannot figure out how to do it.
I tried:
pid = subprocess.call("ps -ef | grep app | awk -v n=' + str(count), 'NR==n | awk \'{print $2}\'", shell=True)
and
args = shlex.split('ps -ef | grep app | awk -v n=' + str(count), 'NR==n | awk \'{print $2}\'')
pid = subprocess.Popen(args, stdout=subprocess.PIPE, shell=True)
among other attempts, from various posts here, but still cannot make it.
You're mixing opening and closing quotations and you pass a colon by mistake on your other attempts among other things.
Try this for a fix:
pid = subprocess.call("ps -ef | grep app | awk -v n=" + str(count) + " NR==n | awk '{print $2}'", shell=True)
You opened the command parameter with " and there for you need to close it before you do + str() with a " and not a '. Further more i swapped the , 'NR= with + "NR= since you want to append more to your command and not pass a argument to subprocess.call().
As pointed out in the comments, there's no point in splitting the command with shlex since piping commands isn't implemented in subprocess, I would however like to point out that using shell=True is usually not recommended because for instance one of the examples given here.
An other vay is using format:
pid = subprocess.call("ps -ef | grep app | awk -v n={} NR==n | awk '{{print $2}}'".format(str(count)), shell=True)
Your Awk pipeline could be simplified a great deal - if the goal is to print the last match, ps -ef | awk '/app/ { p=$2 } END { print p }' does that. But many times, running Awk from Python is just silly, and performing the filtering in Python is convenient and easy, as well as obviously more efficient (you save not only the Awk process, but also the pesky shell=True).
for p in subprocess.check_output(['ps', '-ef']).split('\n'):
if 'app' in p:
pid = p.split()[1]
I'm using python to script a functional script and I can't handler the result of this command line:
os.system("ps aux -u %s | grep %s | grep -v 'grep' | awk '{print $2}'" % (username, process_name)
It shows me pids but I can't use it as List.
If I test:
pids = os.system("ps aux -u %s | grep %s | grep -v 'grep' | awk '{print $2}'" % (username, process_name)
print type(pids)
#Results
29719
30205
31037
31612
<type 'int'>
Why is pids an int? How can I handle this result as List?
Stranger part:
print type(os.system("ps aux -u %s | grep %s | grep -v 'grep' | awk '{print $2}'" % (username, process_name))
There is nothing. Not any type written on my console..
os.system does not capture the output of the command it runs. To do so you need to use subprocess.
from subprocess import check_output
out = check_output("your command goes here", shell=true)
The above will work in Python 2.7. For older Pythons, use:
import subprocess
p = subprocess.Popen("your command goes here", stdout=subprocess.PIPE, shell=True)
out, err = p.communicate()
os module documentation
os.system(command)
Execute the command (a string) in a subshell. This is implemented by calling the Standard C function system(), and has the same limitations. Changes to sys.stdin, etc. are not reflected in the environment of the executed command.
On Unix, the return value is the exit status of the process encoded in the format specified for wait(). Note that POSIX does not specify the meaning of the return value of the C system() function, so the return value of the Python function is system-dependent.
If you want access to the output of the command, use the subprocess module instead, e.g. check_output:
subprocess.check_output(args, *, stdin=None, stderr=None, shell=False, universal_newlines=False)
Run command with arguments and return its output as a byte string.
ip = subprocess.Popen(["/sbin/ifconfig $(/sbin/route | awk '/default/ {print $8}') | grep \"inet addr\" | awk -F: '{print $2}' | awk \'{print $1}\'"], stdout=subprocess.PIPE)
I am not sure where to put the commas to separate them to use this command using subprocess.Popen. Does anyone know?
You are using shell features (the pipe) so instead of splitting the command, you should pass it as a single string (not a list) with shell=True
ip = subprocess.Popen("/sbin/ifconfig $(/sbin/route | awk '/default/ {print $8}') | grep \"inet addr\" | awk -F: '{print $2}' | awk \'{print $1}\'",
shell=True,
stdout=subprocess.PIPE)
Here's what I would recommend.
Create a file with this contents - call it 'route-info' and make it executable:
#!/bin/sh
/sbin/ifconfig $(/sbin/route | awk '/default/ {print $8}') |
grep "inet addr" |
awk -F: '{print $2}' |
awk '{print $1}'
In your python program, use:
ip = subprocess.Popen(["/path/to/route-info"], stdout=subprocess.PIPE)
Then you don't have to worry about quoting characters and you can independently test the route-info script to make sure it is working correctly.
The script route-info doesn't take any command line arguments, but if it did this is how you would pass them:
ip = subprocess.Popen(["/path/to/route-info", arg1, arg2, ...], stdout=subprocess.PIPE)
Quoting the official documentation of subprocess.Popen here
It may not be obvious how to break a shell command into a sequence of
arguments, especially in complex cases. shlex.split() can illustrate
how to determine the correct tokenization for args:
import shlex, subprocess
command_line = input()
args = shlex.split(command_line)
print(args)
p = subprocess.Popen(args) # Success!
shlex is included in standard library so you need not to install it.
Writing it in a single line like str.split() should look like:
import shlex
import subprocess
command = "ls -l"
proc = subprocess.Popen(shlex.split(command) , stdout = subprocess.PIPE , stderr = subprocess.PIPE)
output , errors = proc.communicate()
print(output , errors)
I am trying to adjust the "post to dropbox" services for Snow loepard (http://dl.dropbox.com/u/1144075/Post%20to%20Dropbox.zip).
I dont want the public URL, but a shortened one from goo.gl
Therefor I am using these shell commands:
curl -s --data-urlencode "url=http://link.com" http://googl/action/shorten | grep "googl" | awk -F\" '{print $(NF-1)}' | awk 'BEGIN { FS = "=" } ; { print $2}' | pbcopy
Now the python script does this to copy a dropbox URL for all the files it just copies in the public folder to the clipboard:
pasteURLs = []
for file in copied_files: # for all elements in our list
components = file.split(os.sep) # seperate the path
local_dir = os.sep.join(components[5:]) # cut off the beginning
local_dir = urllib.quote(local_dir) # convert it to a URL (' ' -> '%20', etc.)
#construct the URL
finalURL = 'http://dl.dropbox.com/u/%s/%s' % ( dropbox_id, local_dir )
pasteURLs.append(finalURL) # add the current URL to the string
copy_string = "\n".join(pasteURLs)
os.system( "echo '%s' | pbcopy" % (copy_string) ) # put the string into clipboard
I have to admit I dont know anything about python, but from what it looks like, I need to change the last two lines with this:
shortURL = []
for thisURL in pasteURLs:
shortURL = os.system( curl -s --data-urlencode "url=http://link.com" http://googl/action/shorten | grep "goo.gl" | awk -F\" '{print $(NF-1)}' | awk 'BEGIN { FS = "=" } ; { print $2}' | pbcopy )
shortURLs.append(shortURL)
copy_string = "\n".join(shortURLs)
os.system( "echo '%s' | pbcopy" % (copy_string) ) # put the string into clipboard
But my problem is, how to put the correct URL in the command? As u can see it says http://link.com But it should use thisURL instead.
Any ideas? Thanks in advance!
I think your os.system call should look something like this:
os.system("curl -s --data-urlencode \"url=%s\" http://goo.gl/action/shorten | grep \"goo.gl\" | awk -F\\\" '{print $(NF-1)}' | awk 'BEGIN { FS = \"=\" } ; { print $2}' | pbcopy " % thisURL)
UPDATE I wrote the script for you and used a much simpler command pipeline. Not that the entire thing could be done in python without curl, but here it is.
import subprocess
thisURL = 'http://whatever.com'
pipeline = []
pipeline.append('curl -s -i --data-urlencode "url=%s" ' % thisURL +
'http://goo.gl/action/shorten')
pipeline.append('grep Location:')
pipeline.append('cut -d = -f 2-')
#pipeline.append('pbcopy')
command = "|".join(pipeline)
link, ignore = subprocess.Popen(command, stdout=subprocess.PIPE,
shell=True).communicate()
print link
Other answers have already provided the core of this: use quotation marks around your command, use a format string to insert the value and consider using subprocess in order to actually get the output from the command.
However, if you, like me, think this is getting a bit too convoluted, go have a look at this example on how to do the actual shortening in python. If you're new to python, this might mean you'll need to read up on your exception handling to understand it. (It also looks like you might need a custom module, but then again it appears to only be used if you get an exception...)
I use ifconfig command with awk to catch system's ip addresses
$ ifconfig | grep -E 'inet.[0-9]' | awk '{ print $2}'
127.0.0.1
192.168.8.2
How to convert o/p into a list using python ?
import sys
list_of_lines = [line.strip() for line in sys.stdin]
You might just skip shelling out to call a pipeline of commands. You can get the IP addresses without leaving Python. If you just need the non-loopback IP:
>>> socket.gethostbyname_ex(socket.gethostname())
('furby.home', [], ['192.168.1.5'])
>>> socket.gethostbyname_ex(socket.gethostname())[2][0]
'192.168.1.5'
And to get the loopback,
>>> socket.gethostbyname_ex('localhost')
('localhost', [], ['127.0.0.1'])
There's also a module called netifaces that'll do this in one fell swoop.
import subprocess
lines = subprocess.check_output(["ifconfig | grep -E 'inet.[0-9]' | awk '{ print $2
}'"]).split('\n')
Thanks all . I could do this way.
ipa=[]
f=os.popen("ifconfig | grep -E 'inet.[0-9]' | awk '{ print $2}'")
for i in f.readlines():
ipa.append(i.rstrip('\n'))
return ipa
I just modified the code posted by #Pujan to make it work for linux. (tested in Ubuntu 12.04):
import os
ipa=[]
f=os.popen("/sbin/ifconfig | grep -i \"inet\" | grep -iv \"inet6\" | " + "awk {'print $2'} | sed -ne 's/addr\:/ /p'")
for i in f.readlines():
ipa.append(i.rstrip('\n'))
print ipa
Use sys.stdin to read this output.
Then redirect the output as follows:
$ ifconfig | grep -E 'inet.[0-9]' | awk '{ print $2}' | myProg.py