python subprocess deadlock in demonized server - python

I'm trying to set up a remote backup server for dar, along these lines. I'd really like to do all the piping with python if possible, but I've asked a separate question about that.
Using netcat in subprocess.Popen(cmd, shell=True), I succeeded in making a differential backup, as in the examples on the dar site. The only two problems are:
I don't know how to assign port numbers dynamically this way
If I execute the server in the background, it fails. Why?
Update: This doesn't seem to be related to netcat; it hangs even without netcat in the mix.
Here's my code:
from socket import socket, AF_INET, SOCK_STREAM
import os, sys
import SocketServer
import subprocess
class DarHandler(SocketServer.BaseRequestHandler):
def handle(self):
print('entering handler')
data = self.request.recv(1024).strip()
print('got: ' + data)
if data == 'xform':
cmd1 = 'nc -dl 41201 | dar_slave archives/remotehost | nc -l 41202'
print(cmd1)
cmd2 = 'nc -dl 41200 | dar_xform -s 10k - archives/diffbackup'
print(cmd2)
proc1 = subprocess.Popen(cmd1, shell=True)
proc2 = subprocess.Popen(cmd2, shell=True)
print('sending port number')
self.request.send('41200')
print('waiting')
result = str(proc1.wait())
print('nc-dar_slave-nc returned ' + result)
result = str(proc2.wait())
print('nc-dar_xform returned ' + result)
else:
result = 'bad request'
self.request.send(result)
print('send result, exiting handler')
myaddress = ('localhost', 18010)
def server():
server = SocketServer.TCPServer(myaddress, DarHandler)
print('listening')
server.serve_forever()
def client():
sock = socket(AF_INET, SOCK_STREAM)
print('connecting')
sock.connect(('localhost', 18010))
print('connected, sending request')
sock.send('xform')
print('waiting for response')
port = sock.recv(1024)
print('got: ' + port)
try:
os.unlink('toslave')
except:
pass
os.mkfifo('toslave')
cmd1 = 'nc -w3 localhost 41201 < toslave'
cmd2 = 'nc -w3 localhost 41202 | dar -B config/test.dcf -A - -o toslave -c - | nc -w3 localhost ' + port
print(cmd2)
proc1 = subprocess.Popen(cmd1, shell=True)
proc2 = subprocess.Popen(cmd2, shell=True)
print('waiting')
result2 = proc2.wait()
result1 = proc1.wait()
print('nc<fifo returned: ' + str(result1))
print('nc-dar-nc returned: ' + str(result2))
result = sock.recv(1024)
print('received: ' + result)
sock.close()
print('socket closed, exiting')
if __name__ == "__main__":
if sys.argv[1].startswith('serv'):
server()
else:
client()
Here's what happens on the server:
$ python clientserver.py serve &
[1] 4651
$ listening
entering handler
got: xform
nc -dl 41201 | dar_slave archives/remotehost | nc -l 41202
nc -dl 41200 | dar_xform -s 10k - archives/diffbackup
sending port number
waiting
[1]+ Stopped python clientserver.py serve
Here's what happens on the client:
$ python clientserver.py client
connecting
connected, sending request
waiting for response
got: 41200
nc -w3 localhost 41202 | dar -B config/test.dcf -A - -o toslave -c - | nc -w3 localhost 41200
waiting
FATAL error, aborting operation
Corrupted data read on pipe
nc<fifo returned: 1
nc-dar-nc returned: 1
The client also hangs, and I have to kill it with a keyboard interrupt.

I'd cut my losses and start over. This solution attempt is very complicated and kludgy. There are many ready-made solutions in the area:
10 outstanding Linux backup utilities
Fwbackups sounds good if you want to take the easy route, rsync+ssh for the hard core.

Use Popen.communicate() instead of Popen.wait().
The python documentation for wait() states:
Warning: This will deadlock if the child process generates enough output to a stdout or stderr pipe such that it blocks waiting for the OS pipe buffer to accept more data. Use communicate() to avoid that.
Dar and its related executables should get a -Q if they aren't running interactively.
When syncronizing multiple processes, make sure to call communicate() on the 'weakest link' first: dar_slave before dar_xform and dar before cat. This was done correctly in the question, but it's worth noting.
Clean up shared resources. The client process is holding open a socket from which dar_xform is still reading. Attempting to send/recv data on the initial socket after dar and friends are finished without closing that socket will therefore cause a deadlock.
Here is a working example which doesn't use shell=True or netcat. An advantage of this is I can have the secondary ports assigned dynamically and therefore could conceivably serve multiple backup clients simultaneously.

Related

Python true interactive remote reverse shell

I'm trying to create a true interactive remote shell using Python. When I say true, I mean I don't want to just execute a single command and send the results- I have that working already. I also don't want to abstract executing single commands by having the server interpret directory changes or what not.
I am trying to have a client start an interactive /bin/bash and have the server send commands which are then executed by the same persistent shell. For instance, so if I run cd /foo/bar then pwd would return /foo/bar because I would be interacting with the same bash shell.
Here's some slimmed down example code that currently only will do single command execution...
# client.py
import socket
import subprocess
s = socket.socket()
s.connect(('localhost', 1337))
while True:
cmd = s.recv(1024)
# single command execution currently (not interactive shell)
results = subprocess.Popen(cmd, shell=True,
stdout=subprocess.PIPE, stderr=subprocess.PIPE,
stdin=subprocess.PIPE)
results = results.stdout.read() + results.stderr.read()
s.sendall(results)
# server.py
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('localhost', 1337))
s.listen(5)
conn, _ = s.accept()
while True:
cmd = raw_input('> ').rstrip()
conn.send(cmd)
results = conn.recv(4096)
print results
I've tried many solutions none of which have worked. The subprocess module had a communication method, but it kills the shell after a single command. I'd really like to be able to accomplish this with stdlib, but I've looked at the pexpect module after reading this thread. However, I can't get that to work either? It also doesn't look like it's primary use case is for creating an interactive shell, but rather catching specific command line output for interaction. I can't even get single command execution working with pexpect...
import pexpect, sys
proc = pexpect.spawn('/bin/bash')
proc.logfile = sys.stdout
proc.expect('$ ')
proc.sendline('pwd\n')
If anyone can help it would be appreciated, I feel like there could be a way to multi-thread and spawn off a /bin/bash -i with subprocess and then some how write to stdin and read from stdout? Thanks in advance, and sorry for the length.
Try this code:
# client.py
import socket
import subprocess
s = socket.socket()
s.connect(('localhost', 1337))
process = subprocess.Popen(['/bin/bash', '-i'],
stdout=s.makefile('wb'), stderr=subprocess.STDOUT,
stdin=s.makefile('rb'))
process.wait()
# server.py
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('localhost', 1337))
s.listen(5)
conn, _ = s.accept()
fp = conn.makefile('wb')
proc1 = subprocess.Popen('cat', stdin=conn.makefile('rb'))
while True:
fp.write(sys.stdin.read(4096))
proc1.wait()

How can I do ping in pure python library? [duplicate]

How do I ping a website or IP address with Python?
See this pure Python ping by Matthew Dixon Cowles and Jens Diemer. Also, remember that Python requires root to spawn ICMP (i.e. ping) sockets in linux.
import ping, socket
try:
ping.verbose_ping('www.google.com', count=3)
delay = ping.Ping('www.wikipedia.org', timeout=2000).do()
except socket.error, e:
print "Ping Error:", e
The source code itself is easy to read, see the implementations of verbose_ping and of Ping.do for inspiration.
Depending on what you want to achive, you are probably easiest calling the system ping command..
Using the subprocess module is the best way of doing this, although you have to remember the ping command is different on different operating systems!
import subprocess
host = "www.google.com"
ping = subprocess.Popen(
["ping", "-c", "4", host],
stdout = subprocess.PIPE,
stderr = subprocess.PIPE
)
out, error = ping.communicate()
print out
You don't need to worry about shell-escape characters. For example..
host = "google.com; `echo test`
..will not execute the echo command.
Now, to actually get the ping results, you could parse the out variable. Example output:
round-trip min/avg/max/stddev = 248.139/249.474/250.530/0.896 ms
Example regex:
import re
matcher = re.compile("round-trip min/avg/max/stddev = (\d+.\d+)/(\d+.\d+)/(\d+.\d+)/(\d+.\d+)")
print matcher.search(out).groups()
# ('248.139', '249.474', '250.530', '0.896')
Again, remember the output will vary depending on operating system (and even the version of ping). This isn't ideal, but it will work fine in many situations (where you know the machines the script will be running on)
You may find Noah Gift's presentation Creating Agile Commandline Tools With Python. In it he combines subprocess, Queue and threading to develop solution that is capable of pinging hosts concurrently and speeding up the process. Below is a basic version before he adds command line parsing and some other features. The code to this version and others can be found here
#!/usr/bin/env python2.5
from threading import Thread
import subprocess
from Queue import Queue
num_threads = 4
queue = Queue()
ips = ["10.0.1.1", "10.0.1.3", "10.0.1.11", "10.0.1.51"]
#wraps system ping command
def pinger(i, q):
"""Pings subnet"""
while True:
ip = q.get()
print "Thread %s: Pinging %s" % (i, ip)
ret = subprocess.call("ping -c 1 %s" % ip,
shell=True,
stdout=open('/dev/null', 'w'),
stderr=subprocess.STDOUT)
if ret == 0:
print "%s: is alive" % ip
else:
print "%s: did not respond" % ip
q.task_done()
#Spawn thread pool
for i in range(num_threads):
worker = Thread(target=pinger, args=(i, queue))
worker.setDaemon(True)
worker.start()
#Place work in queue
for ip in ips:
queue.put(ip)
#Wait until worker threads are done to exit
queue.join()
He is also author of: Python for Unix and Linux System Administration
http://ecx.images-amazon.com/images/I/515qmR%2B4sjL._SL500_AA240_.jpg
It's hard to say what your question is, but there are some alternatives.
If you mean to literally execute a request using the ICMP ping protocol, you can get an ICMP library and execute the ping request directly. Google "Python ICMP" to find things like this icmplib. You might want to look at scapy, also.
This will be much faster than using os.system("ping " + ip ).
If you mean to generically "ping" a box to see if it's up, you can use the echo protocol on port 7.
For echo, you use the socket library to open the IP address and port 7. You write something on that port, send a carriage return ("\r\n") and then read the reply.
If you mean to "ping" a web site to see if the site is running, you have to use the http protocol on port 80.
For or properly checking a web server, you use urllib2 to open a specific URL. (/index.html is always popular) and read the response.
There are still more potential meaning of "ping" including "traceroute" and "finger".
I did something similar this way, as an inspiration:
import urllib
import threading
import time
def pinger_urllib(host):
"""
helper function timing the retrival of index.html
TODO: should there be a 1MB bogus file?
"""
t1 = time.time()
urllib.urlopen(host + '/index.html').read()
return (time.time() - t1) * 1000.0
def task(m):
"""
the actual task
"""
delay = float(pinger_urllib(m))
print '%-30s %5.0f [ms]' % (m, delay)
# parallelization
tasks = []
URLs = ['google.com', 'wikipedia.org']
for m in URLs:
t = threading.Thread(target=task, args=(m,))
t.start()
tasks.append(t)
# synchronization point
for t in tasks:
t.join()
Here's a short snippet using subprocess. The check_call method either returns 0 for success, or raises an exception. This way, I don't have to parse the output of ping. I'm using shlex to split the command line arguments.
import subprocess
import shlex
command_line = "ping -c 1 www.google.comsldjkflksj"
args = shlex.split(command_line)
try:
subprocess.check_call(args,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
print "Website is there."
except subprocess.CalledProcessError:
print "Couldn't get a ping."
Most simple answer is:
import os
os.system("ping google.com")
I develop a library that I think could help you. It is called icmplib (unrelated to any other code of the same name that can be found on the Internet) and is a pure implementation of the ICMP protocol in Python.
It is completely object oriented and has simple functions such as the classic ping, multiping and traceroute, as well as low level classes and sockets for those who want to develop applications based on the ICMP protocol.
Here are some other highlights:
Can be run without root privileges.
You can customize many parameters such as the payload of ICMP packets and the traffic class (QoS).
Cross-platform: tested on Linux, macOS and Windows.
Fast and requires few CPU / RAM resources unlike calls made with subprocess.
Lightweight and does not rely on any additional dependencies.
To install it (Python 3.6+ required):
pip3 install icmplib
Here is a simple example of the ping function:
host = ping('1.1.1.1', count=4, interval=1, timeout=2, privileged=True)
if host.is_alive:
print(f'{host.address} is alive! avg_rtt={host.avg_rtt} ms')
else:
print(f'{host.address} is dead')
Set the "privileged" parameter to False if you want to use the library without root privileges.
You can find the complete documentation on the project page:
https://github.com/ValentinBELYN/icmplib
Hope you will find this library useful.
read a file name, the file contain the one url per line, like this:
http://www.poolsaboveground.com/apache/hadoop/core/
http://mirrors.sonic.net/apache/hadoop/core/
use command:
python url.py urls.txt
get the result:
Round Trip Time: 253 ms - mirrors.sonic.net
Round Trip Time: 245 ms - www.globalish.com
Round Trip Time: 327 ms - www.poolsaboveground.com
source code(url.py):
import re
import sys
import urlparse
from subprocess import Popen, PIPE
from threading import Thread
class Pinger(object):
def __init__(self, hosts):
for host in hosts:
hostname = urlparse.urlparse(host).hostname
if hostname:
pa = PingAgent(hostname)
pa.start()
else:
continue
class PingAgent(Thread):
def __init__(self, host):
Thread.__init__(self)
self.host = host
def run(self):
p = Popen('ping -n 1 ' + self.host, stdout=PIPE)
m = re.search('Average = (.*)ms', p.stdout.read())
if m: print 'Round Trip Time: %s ms -' % m.group(1), self.host
else: print 'Error: Invalid Response -', self.host
if __name__ == '__main__':
with open(sys.argv[1]) as f:
content = f.readlines()
Pinger(content)
import subprocess as s
ip=raw_input("Enter the IP/Domain name:")
if(s.call(["ping",ip])==0):
print "your IP is alive"
else:
print "Check ur IP"
If you want something actually in Python, that you can play with, have a look at Scapy:
from scapy.all import *
request = IP(dst="www.google.com")/ICMP()
answer = sr1(request)
That's in my opinion much better (and fully cross-platform), than some funky subprocess calls. Also you can have as much information about the answer (sequence ID.....) as you want, as you have the packet itself.
using system ping command to ping a list of hosts:
import re
from subprocess import Popen, PIPE
from threading import Thread
class Pinger(object):
def __init__(self, hosts):
for host in hosts:
pa = PingAgent(host)
pa.start()
class PingAgent(Thread):
def __init__(self, host):
Thread.__init__(self)
self.host = host
def run(self):
p = Popen('ping -n 1 ' + self.host, stdout=PIPE)
m = re.search('Average = (.*)ms', p.stdout.read())
if m: print 'Round Trip Time: %s ms -' % m.group(1), self.host
else: print 'Error: Invalid Response -', self.host
if __name__ == '__main__':
hosts = [
'www.pylot.org',
'www.goldb.org',
'www.google.com',
'www.yahoo.com',
'www.techcrunch.com',
'www.this_one_wont_work.com'
]
Pinger(hosts)
You can find an updated version of the mentioned script that works on both Windows and Linux here
using subprocess ping command to ping decode it because the response is binary:
import subprocess
ping_response = subprocess.Popen(["ping", "-a", "google.com"], stdout=subprocess.PIPE).stdout.read()
result = ping_response.decode('utf-8')
print(result)
you might try socket to get ip of the site and use scrapy to excute icmp ping to the ip.
import gevent
from gevent import monkey
# monkey.patch_all() should be executed before any library that will
# standard library
monkey.patch_all()
import socket
from scapy.all import IP, ICMP, sr1
def ping_site(fqdn):
ip = socket.gethostbyaddr(fqdn)[-1][0]
print(fqdn, ip, '\n')
icmp = IP(dst=ip)/ICMP()
resp = sr1(icmp, timeout=10)
if resp:
return (fqdn, False)
else:
return (fqdn, True)
sites = ['www.google.com', 'www.baidu.com', 'www.bing.com']
jobs = [gevent.spawn(ping_site, fqdn) for fqdn in sites]
gevent.joinall(jobs)
print([job.value for job in jobs])
On python 3 you can use ping3.
from ping3 import ping, verbose_ping
ip-host = '8.8.8.8'
if not ping(ip-host):
raise ValueError('{} is not available.'.format(ip-host))
If you only want to check whether a machine on an IP is active or not, you can just use python sockets.
import socket
s = socket.socket()
try:
s.connect(("192.168.1.123", 1234)) # You can use any port number here
except Exception as e:
print(e.errno, e)
Now, according to the error message displayed (or the error number), you can determine whether the machine is active or not.
Use this it's tested on python 2.7 and works fine it returns ping time in milliseconds if success and return False on fail.
import platform,subproccess,re
def Ping(hostname,timeout):
if platform.system() == "Windows":
command="ping "+hostname+" -n 1 -w "+str(timeout*1000)
else:
command="ping -i "+str(timeout)+" -c 1 " + hostname
proccess = subprocess.Popen(command, stdout=subprocess.PIPE)
matches=re.match('.*time=([0-9]+)ms.*', proccess.stdout.read(),re.DOTALL)
if matches:
return matches.group(1)
else:
return False

Python sending command over a socket

I'm having a bit of trouble. I want to create a simple program that connects to the server and executes a command using subprocess then returns the result to the client. It's simple but I can't get it to work. Right now this is what I have:
client:
import sys, socket, subprocess
conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host = sys.argv[1]
port = int(sys.argv[2])
socksize = 1024
conn.connect((host, port))
while True:
shell = raw_input("$ ")
conn.send(shell)
data = conn.recv(socksize)
#msglen = len(data)
output = data
iotype = subprocess.PIPE
cmd = ['/bin/sh', '-c', shell]
proc = subprocess.Popen(cmd, stdout=iotype).wait()
stdout,stderr = proc.communicate()
conn.send(stdout)
print(output)
if proc.returncode != 0:
print("Error")
server:
import sys, socket, subprocess
host = ''
port = 50106
socksize = 1024
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host, port))
print("Server started on port: %s" %port)
s.listen(1)
print("Now listening...\n")
conn, addr = s.accept()
while True:
print 'New connection from %s:%d' % (addr[0], addr[1])
data = conn.recv(socksize)
cmd = ['/bin/sh', '-c', data]
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE).wait()
stdout,stderr = cmd.communicate()
if not data:
break
elif data == 'killsrv':
sys.exit()
Danger, Will Robinson!!!
Do you really want to send commands in clear text without authentication over the network? It is very, very dangerous.
Do it over SSH with paramiko.
Alright I've heard this answer too many times. I don't want to use SSH I'm just building it to learn more about sockets. I'm not going to actually use this if I want to send commands to a system. – AustinM
There is no way I could infer this noble quest from your question. :-)
The sockets module is a thin layer over the posix library; plain sockets is tedious and hard to get right. As of today (2014), asynchronous I/O and concurrency are not among Python's strongest traits - 3.4 is starting to change that but libraries will lag behind for a while. My advice is to spent your time learning some higher level API like Twisted (twistedmatrix.com/trac). If you are really interested in the low level stuff, dive in the project source.
Alright. Any idea on how I could use twisted for this type of thing? – AustinM
Look at twistedmatrix.com/documents/current/core/examples/#auto2
Well I can understand your frustration Austin; I was in the same boat. However trial and error at last worked out. Hopefully you were looking for this:
print "Command is:",command
op = subprocess.Popen(command, shell=True, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
if op:
output=str(op.stdout.read())
print "Output:",output
conn.sendall(output)
else:
error=str(op.stderr.read())
print "Error:",error
conn.sendall(error)
It's unclear why you are using subprocess.Popen() for the same command in both the client and the server. Here's an outline of what I would try to do (pseudocode):
client
while True:
read command from user
send command to server
wait for and then read response from server
print response to user
server
while True:
wait for and then read command from client
if command is "killsrv", exit
execute command and capture output
send output to client
The problem with your code is this line (in both client and server):
proc = subprocess.Popen(cmd, stdout=iotype).wait()
stdout,stderr = proc.communicate()
You are calling wait on the Popen object, which means that the variable proc is getting an int (returned by wait) instead of a Popen object. You can just get rid of the wait -- since communicate waits for the process to end before returning, and you aren't checking the exit code anyway, you don't need to call it.
Then, in your client, I don't think you even need the subprocess calls, unless you're running some command that the server is sending back.

Send output from a python server back to client

I now have a small java script server working correctly, called by:
<?php
$handle = fsockopen("udp://78.129.148.16",12345);
fwrite($handle,"vzctlrestart110");
fclose($handle);
?>
On a remote server the following python server is running and executing the comand's
#!/usr/bin/python
import os
import socket
print " Loading Bindings..."
settings = {}
line = 0
for each in open('/root/actions.txt', 'r'):
line = line + 1
each = each.rstrip()
if each != "":
if each[0] != '#':
a = each.partition(':')
if a[2]:
settings[a[0]] = a[2]
else:
print " Err # line",line,":",each
print " Starting Server...",
port = 12345
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
s.bind(("", port))
print "OK."
print " Listening on port:", port
while True:
datagram = s.recv(1024)
if not datagram:
break
print "Rx Cmd:", datagram
if settings.has_key(datagram):
print "Launch:", settings[datagram]
os.system(settings[datagram]+" &")
s.close()
Is it possible to easily send the output of the command back, when the server is started and is running a command the output is shown in the ssh window, however I want this output to be sent back to the browser of the original client, maybe setting the browser to wait for 15 seconds and then check for any data received via the socket.
I know I am asking quite a lot, however I am creating a PHP script which I have a large knowledge about, however my python knowledge lacks greatly.
Thanks,
Ashley
Yes, you can read the output of the command. For this I would recommend the Python subprocess module. Then you can just s.write() it back.
Naturally this has some implications, you would probably have to let your PHP script run for a while longer since the process may be running slow.
# The pipe behaves like a file object in Python.
process = Popen(cmd, shell=True, stdout=PIPE)
process_output = ""
while process.poll():
process_output += process.stdout.read(256)
s.write(process_output)
# Better yet.
process = Popen(cmd, shell=true, stdout=PIPE)
stdout, stderr = process.communicate() # will read and wait for process to end.
s.write(stdout)
Integrated into your code:
# ... snip ...
import subprocess
con, addr = s.accept()
while True:
datagram = con.recv(1024)
if not datagram:
break
print "Rx Cmd:", datagram
if settings.has_key(datagram):
print "Launch:", settings[datagram]
process = subprocess.Popen(settings[datagram]+" &", shell=True, stdout=subprocess.PIPE)
stdout, stderr = process.communicate()
con.send(stdout)
con.close()
s.close()
Here's an example of how to get the output of a command:
>>> import commands
>>> s = commands.getoutput("ls *")
>>> s
'client.py'

Ping a site in Python?

How do I ping a website or IP address with Python?
See this pure Python ping by Matthew Dixon Cowles and Jens Diemer. Also, remember that Python requires root to spawn ICMP (i.e. ping) sockets in linux.
import ping, socket
try:
ping.verbose_ping('www.google.com', count=3)
delay = ping.Ping('www.wikipedia.org', timeout=2000).do()
except socket.error, e:
print "Ping Error:", e
The source code itself is easy to read, see the implementations of verbose_ping and of Ping.do for inspiration.
Depending on what you want to achive, you are probably easiest calling the system ping command..
Using the subprocess module is the best way of doing this, although you have to remember the ping command is different on different operating systems!
import subprocess
host = "www.google.com"
ping = subprocess.Popen(
["ping", "-c", "4", host],
stdout = subprocess.PIPE,
stderr = subprocess.PIPE
)
out, error = ping.communicate()
print out
You don't need to worry about shell-escape characters. For example..
host = "google.com; `echo test`
..will not execute the echo command.
Now, to actually get the ping results, you could parse the out variable. Example output:
round-trip min/avg/max/stddev = 248.139/249.474/250.530/0.896 ms
Example regex:
import re
matcher = re.compile("round-trip min/avg/max/stddev = (\d+.\d+)/(\d+.\d+)/(\d+.\d+)/(\d+.\d+)")
print matcher.search(out).groups()
# ('248.139', '249.474', '250.530', '0.896')
Again, remember the output will vary depending on operating system (and even the version of ping). This isn't ideal, but it will work fine in many situations (where you know the machines the script will be running on)
You may find Noah Gift's presentation Creating Agile Commandline Tools With Python. In it he combines subprocess, Queue and threading to develop solution that is capable of pinging hosts concurrently and speeding up the process. Below is a basic version before he adds command line parsing and some other features. The code to this version and others can be found here
#!/usr/bin/env python2.5
from threading import Thread
import subprocess
from Queue import Queue
num_threads = 4
queue = Queue()
ips = ["10.0.1.1", "10.0.1.3", "10.0.1.11", "10.0.1.51"]
#wraps system ping command
def pinger(i, q):
"""Pings subnet"""
while True:
ip = q.get()
print "Thread %s: Pinging %s" % (i, ip)
ret = subprocess.call("ping -c 1 %s" % ip,
shell=True,
stdout=open('/dev/null', 'w'),
stderr=subprocess.STDOUT)
if ret == 0:
print "%s: is alive" % ip
else:
print "%s: did not respond" % ip
q.task_done()
#Spawn thread pool
for i in range(num_threads):
worker = Thread(target=pinger, args=(i, queue))
worker.setDaemon(True)
worker.start()
#Place work in queue
for ip in ips:
queue.put(ip)
#Wait until worker threads are done to exit
queue.join()
He is also author of: Python for Unix and Linux System Administration
http://ecx.images-amazon.com/images/I/515qmR%2B4sjL._SL500_AA240_.jpg
It's hard to say what your question is, but there are some alternatives.
If you mean to literally execute a request using the ICMP ping protocol, you can get an ICMP library and execute the ping request directly. Google "Python ICMP" to find things like this icmplib. You might want to look at scapy, also.
This will be much faster than using os.system("ping " + ip ).
If you mean to generically "ping" a box to see if it's up, you can use the echo protocol on port 7.
For echo, you use the socket library to open the IP address and port 7. You write something on that port, send a carriage return ("\r\n") and then read the reply.
If you mean to "ping" a web site to see if the site is running, you have to use the http protocol on port 80.
For or properly checking a web server, you use urllib2 to open a specific URL. (/index.html is always popular) and read the response.
There are still more potential meaning of "ping" including "traceroute" and "finger".
I did something similar this way, as an inspiration:
import urllib
import threading
import time
def pinger_urllib(host):
"""
helper function timing the retrival of index.html
TODO: should there be a 1MB bogus file?
"""
t1 = time.time()
urllib.urlopen(host + '/index.html').read()
return (time.time() - t1) * 1000.0
def task(m):
"""
the actual task
"""
delay = float(pinger_urllib(m))
print '%-30s %5.0f [ms]' % (m, delay)
# parallelization
tasks = []
URLs = ['google.com', 'wikipedia.org']
for m in URLs:
t = threading.Thread(target=task, args=(m,))
t.start()
tasks.append(t)
# synchronization point
for t in tasks:
t.join()
Here's a short snippet using subprocess. The check_call method either returns 0 for success, or raises an exception. This way, I don't have to parse the output of ping. I'm using shlex to split the command line arguments.
import subprocess
import shlex
command_line = "ping -c 1 www.google.comsldjkflksj"
args = shlex.split(command_line)
try:
subprocess.check_call(args,stdout=subprocess.PIPE,stderr=subprocess.PIPE)
print "Website is there."
except subprocess.CalledProcessError:
print "Couldn't get a ping."
Most simple answer is:
import os
os.system("ping google.com")
I develop a library that I think could help you. It is called icmplib (unrelated to any other code of the same name that can be found on the Internet) and is a pure implementation of the ICMP protocol in Python.
It is completely object oriented and has simple functions such as the classic ping, multiping and traceroute, as well as low level classes and sockets for those who want to develop applications based on the ICMP protocol.
Here are some other highlights:
Can be run without root privileges.
You can customize many parameters such as the payload of ICMP packets and the traffic class (QoS).
Cross-platform: tested on Linux, macOS and Windows.
Fast and requires few CPU / RAM resources unlike calls made with subprocess.
Lightweight and does not rely on any additional dependencies.
To install it (Python 3.6+ required):
pip3 install icmplib
Here is a simple example of the ping function:
host = ping('1.1.1.1', count=4, interval=1, timeout=2, privileged=True)
if host.is_alive:
print(f'{host.address} is alive! avg_rtt={host.avg_rtt} ms')
else:
print(f'{host.address} is dead')
Set the "privileged" parameter to False if you want to use the library without root privileges.
You can find the complete documentation on the project page:
https://github.com/ValentinBELYN/icmplib
Hope you will find this library useful.
read a file name, the file contain the one url per line, like this:
http://www.poolsaboveground.com/apache/hadoop/core/
http://mirrors.sonic.net/apache/hadoop/core/
use command:
python url.py urls.txt
get the result:
Round Trip Time: 253 ms - mirrors.sonic.net
Round Trip Time: 245 ms - www.globalish.com
Round Trip Time: 327 ms - www.poolsaboveground.com
source code(url.py):
import re
import sys
import urlparse
from subprocess import Popen, PIPE
from threading import Thread
class Pinger(object):
def __init__(self, hosts):
for host in hosts:
hostname = urlparse.urlparse(host).hostname
if hostname:
pa = PingAgent(hostname)
pa.start()
else:
continue
class PingAgent(Thread):
def __init__(self, host):
Thread.__init__(self)
self.host = host
def run(self):
p = Popen('ping -n 1 ' + self.host, stdout=PIPE)
m = re.search('Average = (.*)ms', p.stdout.read())
if m: print 'Round Trip Time: %s ms -' % m.group(1), self.host
else: print 'Error: Invalid Response -', self.host
if __name__ == '__main__':
with open(sys.argv[1]) as f:
content = f.readlines()
Pinger(content)
import subprocess as s
ip=raw_input("Enter the IP/Domain name:")
if(s.call(["ping",ip])==0):
print "your IP is alive"
else:
print "Check ur IP"
If you want something actually in Python, that you can play with, have a look at Scapy:
from scapy.all import *
request = IP(dst="www.google.com")/ICMP()
answer = sr1(request)
That's in my opinion much better (and fully cross-platform), than some funky subprocess calls. Also you can have as much information about the answer (sequence ID.....) as you want, as you have the packet itself.
using system ping command to ping a list of hosts:
import re
from subprocess import Popen, PIPE
from threading import Thread
class Pinger(object):
def __init__(self, hosts):
for host in hosts:
pa = PingAgent(host)
pa.start()
class PingAgent(Thread):
def __init__(self, host):
Thread.__init__(self)
self.host = host
def run(self):
p = Popen('ping -n 1 ' + self.host, stdout=PIPE)
m = re.search('Average = (.*)ms', p.stdout.read())
if m: print 'Round Trip Time: %s ms -' % m.group(1), self.host
else: print 'Error: Invalid Response -', self.host
if __name__ == '__main__':
hosts = [
'www.pylot.org',
'www.goldb.org',
'www.google.com',
'www.yahoo.com',
'www.techcrunch.com',
'www.this_one_wont_work.com'
]
Pinger(hosts)
You can find an updated version of the mentioned script that works on both Windows and Linux here
using subprocess ping command to ping decode it because the response is binary:
import subprocess
ping_response = subprocess.Popen(["ping", "-a", "google.com"], stdout=subprocess.PIPE).stdout.read()
result = ping_response.decode('utf-8')
print(result)
you might try socket to get ip of the site and use scrapy to excute icmp ping to the ip.
import gevent
from gevent import monkey
# monkey.patch_all() should be executed before any library that will
# standard library
monkey.patch_all()
import socket
from scapy.all import IP, ICMP, sr1
def ping_site(fqdn):
ip = socket.gethostbyaddr(fqdn)[-1][0]
print(fqdn, ip, '\n')
icmp = IP(dst=ip)/ICMP()
resp = sr1(icmp, timeout=10)
if resp:
return (fqdn, False)
else:
return (fqdn, True)
sites = ['www.google.com', 'www.baidu.com', 'www.bing.com']
jobs = [gevent.spawn(ping_site, fqdn) for fqdn in sites]
gevent.joinall(jobs)
print([job.value for job in jobs])
On python 3 you can use ping3.
from ping3 import ping, verbose_ping
ip-host = '8.8.8.8'
if not ping(ip-host):
raise ValueError('{} is not available.'.format(ip-host))
If you only want to check whether a machine on an IP is active or not, you can just use python sockets.
import socket
s = socket.socket()
try:
s.connect(("192.168.1.123", 1234)) # You can use any port number here
except Exception as e:
print(e.errno, e)
Now, according to the error message displayed (or the error number), you can determine whether the machine is active or not.
Use this it's tested on python 2.7 and works fine it returns ping time in milliseconds if success and return False on fail.
import platform,subproccess,re
def Ping(hostname,timeout):
if platform.system() == "Windows":
command="ping "+hostname+" -n 1 -w "+str(timeout*1000)
else:
command="ping -i "+str(timeout)+" -c 1 " + hostname
proccess = subprocess.Popen(command, stdout=subprocess.PIPE)
matches=re.match('.*time=([0-9]+)ms.*', proccess.stdout.read(),re.DOTALL)
if matches:
return matches.group(1)
else:
return False

Categories