i got this code from http://www.evolt.org/node/60276 and modified it to listen for a single "1" coming from the other side
but whenever i run this program it stops and python IDLE goes to non-responding on "data1,addr = UDPSock.recvfrom(1024)"
def get1():
# Server program, receives 1 if ball found
# ff1 is file w/ received data
import socket
import time
# Set the socket parameters
host = "mysystem"
port = 21567
#buf = 1024
addr = (host,port)
# Create socket (UDP) and bind to address
UDPSock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
UDPSock.bind(addr)
# Receive messages
while 1:
print "waiting..............."
data1,addr = UDPSock.recvfrom(1024)
print "got 1"
if not data1:
print "Client has exited!"
break
else:
print "\nReceived message '", data1,"'"
UDPSock.close() # Close socket
print "socket closed\n"
#call some other function that uses 1
and client side
def send1():
# Client program, sends 1 if ball found
# mf1 is file with data to be sent
import socket
# Set the socket parameters
host = "mysystem"
port = 21567
buf = 1024
addr = (host,port)
# Create socket (UDP)
UDPSock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
mf1=1
print mf1
# Send messages
if(UDPSock.sendto(str(mf1),addr)):
print "Sending message '",str(mf1),"'....."
# Close socket
UDPSock.close()
does anyone know what might be the cause of this? (sorry for long post)
As a second guess (I replaced my first guess with this) I suspect that you are running the receiver in IDLE and then IDLE is hanging so you can't run the client. I don't know exactly how IDLE works as I never use it, but the line containing recvfrom will stop the Python thread its running in until data is sent. So you need to start the client in a separate instance of IDLE or from the command line or something.
At any rate, I have tested the program in question on my Python with 127.0.0.1 as the host, and it worked fine, for some values of fine. The recvfrom does hang, but only until some data is sent, then it comes back with the data and prints it out and everything. You do have a bug that happens after that though. :-)
Related
I'm attempting to send data via sockets to the same IP but via different ports. These are the test scripts I've developed so far:
server:
# test_server.py
import socket
import select
# module-level variables ##############################################################################################
HOST1 = '127.0.0.1'
PORT1 = 65432
HOST2 = '127.0.0.1'
PORT2 = 65433
#######################################################################################################################
def main():
sock1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock1.bind((HOST1, PORT1))
sock1.listen()
conn1, addr1 = sock1.accept()
sock2 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock2.bind((HOST2, PORT2))
sock2.listen()
conn2, addr2 = sock2.accept()
conns = [ conn1, conn2 ]
while True:
readyConns, _, _ = select.select(conns, [], [])
for conn in readyConns:
data = conn.recv(1024)
if not data:
print('no data received')
else:
print('received: ' + data.decode("utf-8"))
# end if
conn.sendall(bytes('acknowledgement from server', 'utf-8'))
# end for
# end while
# end main
#######################################################################################################################
if __name__ == '__main__':
main()
client:
# test_client.py
import socket
import time
# module-level variables ##############################################################################################
HOST1 = '127.0.0.1'
PORT1 = 65432
HOST2 = '127.0.0.1'
PORT2 = 65433
#######################################################################################################################
def main():
myCounter = 1
sock1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock1.connect((HOST1, PORT1))
sock2 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock2.connect((HOST2, PORT2))
while True:
# sock1 ############################################
# send the original message
messageAsStr1 = 'message 1-' + myCounter
sock1.sendall(bytes(messageAsStr1, 'utf-8'))
# receive the acknowledgement
ack1 = sock1.recv(1024)
if ack1 is None:
print('error receiving acknowledgement on port 1')
else:
print('received: ' + ack1.decode('utf-8'))
# end if
time.sleep(2)
# sock2 ############################################
# send the original message
messageAsStr2 = 'message 2-' + myCounter
sock2.sendall(bytes(messageAsStr2, 'utf-8'))
# receive the acknowledgement
ack2 = sock2.recv(1024)
if ack2 is None:
print('error receiving acknowledgement on port 2')
else:
print('received: ' + ack2.decode('utf-8'))
# end if
time.sleep(2)
myCounter += 1
# end while
# end main
#######################################################################################################################
if __name__ == '__main__':
main()
If I start test_server.py then test_client.py, test_server.py will start successfully, but upon starting test_client.py I get:
$ python3 test_client.py
Traceback (most recent call last):
File "test_client.py", line 66, in <module>
main()
File "test_client.py", line 23, in main
sock2.connect((HOST2, PORT2))
ConnectionRefusedError: [Errno 111] Connection refused
I don't understand why the 2nd connection won't go through, b/c if I divide test_client.py into 2 separate programs as follows:
# test_client1.py
import socket
import time
# module-level variables ##############################################################################################
HOST1 = '127.0.0.1'
PORT1 = 65432
#######################################################################################################################
def main():
myCounter = 1
sock1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock1.connect((HOST1, PORT1))
while True:
# sock1 ############################################
# send the original message
messageAsStr1 = 'message 1-' + str(myCounter)
sock1.sendall(bytes(messageAsStr1, 'utf-8'))
# receive the acknowledgement
ack1 = sock1.recv(1024)
if ack1 is None:
print('error receiving acknowledgement on port 1')
else:
print('received: ' + ack1.decode('utf-8'))
# end if
myCounter += 1
time.sleep(2)
# end while
# end main
#######################################################################################################################
if __name__ == '__main__':
main()
and:
# test_client2.py
import socket
import time
# module-level variables ##############################################################################################
HOST2 = '127.0.0.1'
PORT2 = 65433
#######################################################################################################################
def main():
myCounter = 1
sock2 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock2.connect((HOST2, PORT2))
while True:
# sock2 ############################################
# send the original message
messageAsStr2 = 'message 2-' + str(myCounter)
sock2.sendall(bytes(messageAsStr2, 'utf-8'))
# receive the acknowledgement
ack2 = sock2.recv(1024)
if ack2 is None:
print('error receiving acknowledgement on port 2')
else:
print('received: ' + ack2.decode('utf-8'))
# end if
myCounter += 1
time.sleep(2)
# end while
# end main
#######################################################################################################################
if __name__ == '__main__':
main()
Then start them in the order test_server.py, test_client1.py, test_client2.py, I get the expected results:
(first command prompt):
$ python3 test_server.py
received: message 1-1
received: message 2-1
received: message 1-2
received: message 2-2
received: message 1-3
received: message 2-3
received: message 1-4
received: message 2-4
(second command prompt):
$ python3 test_client1.py
received: acknowledgement from server
received: acknowledgement from server
received: acknowledgement from server
received: acknowledgement from server
(third command prompt):
python3 test_client2.py
received: acknowledgement from server
received: acknowledgement from server
received: acknowledgement from server
received: acknowledgement from server
At this time my questions are:
1) Why does the 1st way (with the different port numbers in test_client.py) not work but it does if I split it into two scripts?
2) Is there a more elegant/robust way to achieve the same objective? I should mention that in my final program I absolutely need the acknowledgement content and the flexibility to use different port numbers as well as IPs.
3) Another limitation of the above is in the 2nd case where it worked, I have to start the programs in the specific order test_server.py, test_client1.py, test_client2.py. In the production version I'll eventually be making the order will vary. Is there a recommended change to elegantly handle this?
I can see why this would be confusing. Fortunately, the explanations are simple.
Question 1 - blocking calls and race conditions
In your server script, when you tell it to accept(), your script "waits" (we say this function call is "blocking"). The rest of your server script has not even been looked at yet. Now you go run your client script, it connects on the first port and this causes the call in your server script to unblock and proceed. Now the "race" is on, will your server script execute the call to listen() and then accept() before your client makes its next call to connect()?? Maybe! But not likely, unless you put a sleep in your client. Then you will see it work every time. But that is not the right solution.
Question 2 - the "elegant" way
You really want to be waiting for connections on X number of ports simultaneously, not in series. The simple way to do this would be to use python multiprocessing module to launch parallel server threads of execution. But, that is a heavy solution.
The right way really is to use select() which is the official means of saying "I'm just one process, but I want to listen for an wait for activity on multiple things (ports)". So, you'll setup your multiple sockets and then give select the list of things you want to wait for. It will block until there's activity on any one of those. You do the work to process that and then loop back to select() again to block and wait for more activity (maybe next time on a different port, or maybe a new connection on the same port).
There is an important caveat, which may make select an 'advanced level' tool. As you process requests, be careful you don't spend too much time in processing, because all those other connections are waiting in the meantime. There are techniques to do this properly, such as managing all your socket and file io with libevent.
Using multiprocessing is more straightforward, but it will not scale as well as a well architected event-based design. Here's a discussion about an example of this, with Apache versus Nginx. Another example of the power of solutions based on event handling architecture is NodeJS, which famously runs everything in an event loop.
All this detail is to highlight, since you said you are going to step up to a production implementation next that there is a lot to think about.
Best solution: focus on solving your real problem, and let someone else design the server. Learn to use gunicorn or wsgi (good servers already) and put your request processing in there.
Question 3 - the right way to client
Naturally, you'll want to spiff up your client too. The right way to do a client is of course to not expect everything to be perfect. You are trying to connect to a faraway machine. The network might be down, the server might be offline, etc. So, choose your strategy from among the popular choices for clients in general:
fail (out of the script) with clear error to the user, they will retry if and when they want to by re-running your client script
inform the user that you are pausing and will try again in 10 seconds. Make X attempts (e.g. 10 attempts) then fail out of the (client) script.
be a client with multiple options, automatically try other servers. If your problem domain warrants it, you can plan on architecting a grand solution with contingency plans. A great client automatically tries other servers if the first one is not answering (and a great service provides multiple available servers).
So, general answer: catch that exception and then decide what you want to do. Wait and retry in a loop, bail, or "other" ?
I am trying to learn Python sockets and have hit a snare with the socket.accept() method. As I understand the method, once I call accept, the thread will sit and wait for an incoming connection (blocking all following code). However, in the code below, which I got from https://docs.python.org/2/library/socket.html and am using localhost. I added a print('hello') to the first line of the server. Yet the print doesn't appear until after I disconnect the client. Why is this? Why does accept seem to run before my print yet after I bind the socket?
# Echo server program
import socket
print('hello') # This doesn't print until I disconnect the client
HOST = 'localhost'
PORT = 50007
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
s.listen(1)
conn, addr = s.accept()
print 'Connected by', addr
while 1:
data = conn.recv(1024)
if not data: break
conn.sendall(data)
conn.close()
# Echo client program
import socket
HOST = 'localhost' # The remote host
PORT = 50007 # The same port as used by the server
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
s.sendall('Hello, world')
data = s.recv(1024)
s.close()
print 'Received', repr(data)
You are likely using an output device on a system that Python's IO doesn't recognize as interactive. As a workaround, you can add sys.stdout.flush() after the print.
The standard output is a buffered stream, meaning that when you print something, the output sticks around in an internal buffer until you print enough data to fill the whole buffer (unlikely in a small program, the buffet is several kilobytes in size), or until the program exits, when all such buffers are automatically flushed. Normally when the output is a terminal service, the IO layer automatically switches to line buffering, where the buffer is also flushed whenever a newline character is printed (and which the print statement inserts automatically).
For some reason, that doesn't work on your system, and you have to flush explicitly. Another option is to run python -u, which should force unbuffered standard streams.
In trying to familiarize myself with the socket library, I have a simple server and client setup. Basically I've stumbled through and am able to set up connection and get the server and client to talk to each other. To make it more interactive, I have client.py able to send text through the command line. Everything appears to be working properly (with the exception of the server side tearing down connection properly if client input is blank), if I type a message from the client side, it spits it right back out to me. In this example, I have it set up for the server side to print the text as well. What I noticed was, that the server side doesn't alway 'register' what it being sent from the client. I am trying to figure out why this is the case. For being a test, it doesn't really affect anything, I just can't figure out what is taking place behind the scenes.
EDIT:
Actually, after playing around with it for a bit, it appears every other message is being printed out to the server console. I've still yet to figure out why this is the case
Server side:
#server.py
import socket
ss = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ss.bind(('127.0.0.1',5000))
ss.listen(5)
while True:
conn, addr = ss.accept()
with conn:
print ('Connected by', addr)
while True:
data = conn.recv(4096)
print (data)
if not data:
print ("nothing received from client")
ss.close()
break
Client side:
#client.py
import socket
server = 'localhost'
port = 5000
s = socket. socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('127.0.0.1', 5000))
s.send(bytes(b'Client has connected'))
while True:
msg = input()
s.send(bytes(input(msg),'utf-8'))
if not msg:
print ("Server should tear down connection...")
# s.close()
break
In sockets you there are no methods __exit__ implemented, so you can't use the with conn:
you need to remove this part of code.
I have been self-learning python since few months now , and finally learning Socket programming. As an text book exercise, I am supposed to design a half-duplex chat system . Below is the code. The first request and response are just fine , but everytime I try sending a second message from client, the server seems to be hanging. The program is TCP based.
I am suspecting that since ss.accept() is being called everytime a new message has to be sent, a new thread is being created but since I have made only 1 call to sc.connect() from client , may be my new connection at the server end is hanging there for infinite time.
As a trail : I called ss.accept() outside the while loop, ie making only 1 connection and listening to data over and over on while loop, the conversations works just fine
Can someone please have a look a the code and help me understand where exactly is the issue.
Since, I am learning, I have not moved to twisted yet. I want to learn all the basics first before I move to frameworks.
!bin/usr/env python
import socket, sys
HOST =''
PORT = 1060
ADDR =(HOST,PORT)
def userinput(sock):
usermessage = input('>')
sock.sendall(str(len(usermessage)))
return usermessage
def server():
s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
s.bind(ADDR)
s.listen(1)
print 'the server is listening at',s.getsockname()
while True:
ss,sockname = s.accept()
#listen to determine the bytes sent by client
msglen = ss.recv(4096)
#accept the complete message
msg = ss.recv(int(msglen))
print 'client:', repr(msg)
servermsg = userinput(ss)
ss.sendall(servermsg)
print " ---------------"
ss.close()
def client():
sc = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
sc.connect(ADDR)
while True:
message = userinput(sc)
sc.sendall(message)
replylen = sc.recv(4096)
reply = sc.recv(int(replylen))
print "reply:",reply
print "----------------"
sc.close()
if sys.argv[1:] == ['server']:
server()
elif sys.argv[1:] == ['client']:
client()
else:
print >> sys.stderr,'usage:tcp_2_7.py server|client[host]'
Your trial - accepting once and then receiving multiple messages - is how you should do this. Calling accept is waiting for a new connection - you don't need to do this every time you want to send or receive a message, just as you don't want to call connect every time you want to send or receive.
Think of it this way:
When you connect to a chat server, do you connect, send a message, then disconnect immediately? No - you have a constant open connection which messages are sent through, and the connection is only closed at the end of a chat session.
From the docs on accept:
socket.accept()
Accept a connection. The socket must be bound to an
address and listening for connections. The return value is a pair
(conn, address) where conn is a new socket object usable to send and
receive data on the connection, and address is the address bound to
the socket on the other end of the connection.
I have a basic client-server script in Python using sockets. The server binds to a specific port and waits for a client connection. When a client connects, they are presented with a raw_input prompt that sends the entered commands to a subproc on the server and pipes the output back to the client.
Sometimes when I execute commands from the client, the output will hang and not present me with the raw_input prompt until I press the [enter] key.
At first I thought this might have been a buffer problem but it happens when I use commands with a small output, like 'clear' or 'ls', etc.
The client code:
import os, sys
import socket
from base64 import *
import time
try:
HOST = sys.argv[1]
PORT = int(sys.argv[2])
except IndexError:
print("You must specify a host IP address and port number!")
print("usage: ./handler_client.py 192.168.1.4 4444")
sys.exit()
socksize = 4096
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
server.connect((HOST, PORT))
print("[+] Connection established!")
print("[+] Type ':help' to view commands.")
except:
print("[!] Connection error!")
sys.exit(2)
while True:
data = server.recv(socksize)
cmd = raw_input(data)
server.sendall(str(cmd))
server.close()
Server code:
import os,sys
import socket
import time
from subprocess import Popen,PIPE,STDOUT,call
HOST = ''
PORT = 4444
socksize = 4096
activePID = []
conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
conn.bind((HOST, PORT))
conn.listen(5)
print("Listening on TCP port %s" % PORT)
def reaper():
while activePID:
pid,stat = os.waitpid(0, os.WNOHANG)
if not pid: break
activePID.remove(pid)
def handler(connection):
time.sleep(3)
while True:
cmd = connection.recv(socksize)
proc = Popen(cmd,
shell=True,
stdout=PIPE,
stderr=PIPE,
stdin=PIPE,
)
stdout, stderr = proc.communicate()
if cmd == ":killme":
connection.close()
sys.exit(0)
elif proc:
connection.send( stdout )
connection.send("\nshell => ")
connection.close()
os._exit(0)
def accept():
while 1:
global connection
connection, address = conn.accept()
print "[!] New connection!"
connection.send("\nshell => ")
reaper()
childPid = os.fork() # forks the incoming connection and sends to conn handler
if childPid == 0:
handler(connection)
else:
activePID.append(childPid)
accept()
The problem I see is that the final loop in the client only does one server.recv(socksize), and then it calls raw_input(). If that recv() call does not obtain all of the data sent by the server in that single call, then it also won't collect the prompt that follows the command output and therefore won't show that next prompt. The uncollected input will sit in the socket until you enter the next command, and then it will be collected and shown. (In principle it could take many recv() calls to drain the socket and get to the appended prompt, not just two calls.)
If this is what's happening then you would hit the problem if the command sent back more than one buffer's worth (4KB) of data, or if it generated output in small chunks spaced out in time so that the server side could spread that data over multiple sends that are not coalesced quickly enough for the client to collect them all in a single recv().
To fix this, you need have the client do as many recv() calls as it takes to completely drain the socket. So you need to come up with a way for the client to know that the socket has been drained of everything that the server is going to send in this interaction.
The easiest way to do this is to have the server add boundary markers into the data stream and then have the client inspect those markers to discover when the final data from the current interaction has been collected. There are various ways to do this, but I'd probably have the server insert a "this is the length of the following chunk of data" marker ahead of every chunk it sends, and send a marker with a length of zero after the final chunk.
The client-side main loop then becomes:
forever:
read a marker;
if the length carried in the marker is zero then
break;
else
read exactly that many bytes;.
Note that the client must be sure to recv() the complete marker before it acts on it; stuff can come out of a stream socket in lumps of any size, completely unrelated to the size of the writes that sent that stuff into the socket at the sender's side.
You get to decide whether to send the marker as variable-length text (with a distinctive delimiter) or as fixed-length binary (in which case you have to worry about endian issues if the client and server can be on different systems). You also get to decide whether the client should show each chunk as it arrives (obviously you can't use raw_input() to do that) or whether it should collect all of the chunks and show the whole thing in one blast after the final chunk has been collected.