I am trying to have a client connect to my server, and have a stream of communication between them. The only reason the connection should break is due to network errors, or unless the client wants to stop talking.
The issue I am running into is keeping the handler in a tight loop, and parsing the JSON.
My server code is :
#!/usr/bin/env python
import SocketServer
import socket
import json
import time
class MyTCPServer(SocketServer.ThreadingTCPServer):
allow_reuse_address = True
class MyTCPServerHandler(SocketServer.BaseRequestHandler):
def handle(self):
while 1:
try:
networkData = (self.request.recv(1024).strip())
try:
jsonInputData = json.loads(networkData)
print jsonInputData
try:
if jsonInputData['type'] == 'SAY_HI':
print "HI"
except Exception, e:
print "no hi"
pass
try:
if jsonInputData['type'] == 'GO_AWAY':
print "Going away!"
except Exception, e:
print "no go away"
pass
except Exception, e:
pass
#time.sleep(0.001)
#print "JSON Error", e
except Exception, e:
#time.sleep(0.001)
pass
#print "No message", e
server = MyTCPServer(('192.168.1.115', 13373), MyTCPServerHandler)
server.serve_forever()
My client code is simple :
#!/usr/bin/env python
import socket
import json
import time
import sys
hostname = '192.168.1.103'
port = 13373
try:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((hostname,port))
except Exception, e:
print "Error, could not open socket: ", e
data = {'type':'SAY_HI'}
sock.send(json.dumps(data))
data = {'type':'SAY_BYE'}
sock.send(json.dumps(data))
Sometimes I'll see the messages being sent, "SAY_HI" and "SAY_BYE", but most of the times, no data is being displayed on the server side.
This question is really not clear, but calling self.request.recv(1024) is very likely not what you want to do. You're eliminating all of the nice application-level handling that TCP will happily do for you. If you change that to self.request.recv(8) or a similarly very small number (such that recv() returns whenever it receives data, and doesn't try to fill your buffer), you may get better results.
Ultimately this is super-simplistic change, even if it works, that will not work in a larger context. You will need to be handling exceptions from your json parser on the server side and waiting for more data until an entire well-formed message is received.
This is a hopelessly more complex subject than will be handled generally in any SO answer. If you're going to be doing any amount of raw sockets programming, you absolutely must own a copy of Unix Network Programming, Volume 1.
Related
Using pyBluez, I use the following code to advertise and listen for a bluetooth connection:
def connect_socket():
global client_sock
try:
server_sock = BluetoothSocket(RFCOMM)
server_sock.bind(("", PORT_ANY))
server_sock.listen(1)
port = server_sock.getsockname()[1]
uuid = "00001101-0000-1000-8000-00805F9B34FB"
advertise_service(server_sock, "GSA",
service_id=uuid,
service_classes=[uuid, SERIAL_PORT_CLASS],
profiles=[SERIAL_PORT_PROFILE])
print("Waiting for connection on RFCOMM channel %d" % port)
client_sock, client_info = server_sock.accept()
print("Accepted connection from ", client_info)
except Exception as e: (yes, I know I'm catching all exceptions)
print(e)
I use the following to call the above and send data out from the socket. (I wind up waiting for a connection on every possible channel, which is not desirable, but that's not my only problem or the one that's prompting this question, though I'd like to fix it, too.)
def write_bt(message):
global client_sock
if client_sock is None:
threading.Thread(target=connect_socket).start()
if client_sock is not None:
try:
client_sock.send(message)
except Exception as e:
gsa_msg.message(e)
client_sock = None
I also need to receive data from the socket and write it to a usb connection. For this, I use the following:
def forward_bt_to_usb():
global client_sock
global serUSB
if (client_sock is not None) and (serUSB is not None):
try:
data = client_sock.recv(1024)
serUSB.write(data)
except Exception as e:
gsa_msg.error(e)
client_sock = None
Both write_bt() and forward_bt_to_usb() get called continuously from a loop and are communicating with the same client, but there isn't always data being received over the socket, and forward_bt_to_usb() seems to block everything in that case.
I believe that I probably have all of this structured improperly for what I'm trying to do, or perhaps I just need to have separate threads for sending and receiving data, but it's not obvious to me how to do that (Initially I just put some of the code from forward_bt_to_usb() in a separate thread, without realizing that that would just keep creating new threads as forward_bt_to_usb() kept getting called.)
It seems that what I'm trying to do should be pretty straightforward and certainly not novel, but I haven't been able to find examples or an explanation that I've been able to implement.
I am writing a small python script in order to use it for checking haproxy. What the script does is to connect on haproxy socket and "poll" for stats.
#!/usr/bin/env python
import socket
import sys
my_socket = socket.socket( socket.AF_UNIX, socket.SOCK_STREAM )
try:
my_socket.connect( "/var/run/haproxy/haproxy.sock" )
except socket.error:
print "cant connect to socket"
sys.exit(1)
my_socket.send("show stat\n")
response = my_socket.recv(1024)
print response
What i wish to do is if there is no response from the socket, meaning if haproxy will not output the stats, to exit the script with exit code (1).Is it possible to somehow evaluate if an answer is received?
By default the socket will be in blocking mode and recv() will block until data is received or the connection is closed.
If you can assume that the proxy will respond within a certain amount of time you can set a timeout on the client socket. The timeout is the number of seconds to wait for a socket operation to complete. If the operation is not complete an exception is raised:
my_socket.settimeout(5.0) # 5 seconds. Set this after connecting.
try:
response = my_socket.recv(1024)
print response
except socket.timeout as exc:
print 'timed out waiting for response from proxy'
my_socket.close()
sys.exit(1)
That's one way and it's probably the easiest way. You could also look at the select() module which provides functions that will let your client wait for the socket to become readable, which indicates that there is data to be read, or that the socket has been closed. It really depends on what behaviour you want. Example using select():
import select
r, _, _ = select.select([my_socket], [], [], 5.0)
if r:
response = my_socket.recv(1024)
print response
else:
print 'Nothing received from proxy in 5 seconds'
my_socket.close()
sys.exit(1)
I have a python program where I use a server socket to send data. There is a class which has some Threading methods. Each method checks a queue and if the queue is not empty, it sends the data over the sever socket. Queues are being filled with what clients send to server(server is listening for input requests). Sending is accomplished with a method call:
def send(self, data):
self.sqn += 1
try:
self.clisock.send(data)
except Exception, e:
print 'Send packet failed with error: ' + e.message
When the program starts, sending rate is around 500, but after a while it decreases instantly to 30 with this exception:
Send packet failed with error: <class 'socket.error'>>>[Errno 32] Broken pipe
I don't know what causes the rate to increase! Any idea?
That error is from your send function trying to write to a socket closed on the other side. If that is intended then catch the exception using
import errno, socket
try:
self.clisock.send(data)
except socket.error, err:
if err[0] == errno.EPIPE:
# do something
else:
pass # do something else
If this isn't intended behavior on the part of the client then you'll have to update your post with the corresponding client code.
I'm getting started with Socket programming in Python.
As part of my requirement I want to send some requests (call send_post_requests) and then receive all of them.
Can anyone tell me what I'm doing wrong here.
Right now I'm able to receive only first response.
Here's my code snippet-
def send_post_request(s,i="1",question='{"a": {"b":"dummy}}'):
s.send('POST /request HTTP/1.1\r\nHost: localhost\r\nAccept-Encoding: identity\r\nContent-Length: '+str(len(question))+'\r\nrequest-id: '+i+'\r\nConnection: Keep-Alive\r\nContent-Type: application/json\r\n\r\n')
s.send(question)
def pipe(noOfSockets=1,noOfRequest = 5,question='{"a": {"b":"dummy}}',sizeFactor = 1):
sockets =[]
for i in range(0,noOfSockets ):
sockets.append(socket( AF_INET, SOCK_STREAM))
for s in sockets:
try:
s.connect( (<IP>, <port>))
for i in range(0,noOfRequest):
send_post_request(s,str(i),question)
except(error,timeout), e:
print(e)
except Exception, e:
print("unknown except(%s) :(" % e)
passes =0
allResp =[]
for s in sockets:
sleep(2)
size = noOfRequest * 1000 *sizeFactor
responses = s.recv(size)
allResp.append(responses)
return allResp
Webservers normally close the connection after one request. Use a library for http-requests like urllib2 or requests.
I'm trying to write a simple socket-based client in Python that will connect to a telnet server. I can test the server by telnetting to its port (5007), and entering text. It responds with a NAK (error) or an AK (success), sometimes accompanied by other text. Seems very simple.
I wrote a client to connect and communicate with the server, but it hangs on the first attempt to read the response. The connection is successful. Queries like getsockname and getpeername are successful. The send command returns a value that equals the number of characters I'm sending, so it seems to be sending correctly. But in the end, it always hangs when I try to read the response.
I've tried using both file-based objects like readline and write (via socket.makefile), as well as using send and recv. With the file object I tried making it with "rw" and reading and writing via that object, and later tried one object for "r" and another for "w" to separate them. None of these worked.
I used a packet sniffer to watch what's going on. I'm not versed in all that I'm seeing, but during a telnet session I can see my typed text and the server's text coming back. During my Python socket connection, I can see my text going to the server, but packets back don't seem to have any text in them.
Any ideas on what I'm doing wrong, or any strategies to try?
Here's the code I'm using (in this case, it's with send and recv):
#!/usr/bin/python
host = "localhost"
port = 5007
msg = "HELLO EMC 1 1"
msg2 = "HELLO"
import socket
import sys
try:
skt = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error, e:
print("Error creating socket: %s" % e)
sys.exit(1)
try:
skt.connect((host,port))
except socket.gaierror, e:
print("Address-related error connecting to server: %s" % e)
sys.exit(1)
except socket.error, e:
print("Error connecting to socket: %s" % e)
sys.exit(1)
try:
print(skt.send(msg))
print("SEND: %s" % msg)
except socket.error, e:
print("Error sending data: %s" % e)
sys.exit(1)
while 1:
try:
buf = skt.recv(1024)
print("RECV: %s" % buf)
except socket.error, e:
print("Error receiving data: %s" % e)
sys.exit(1)
if not len(buf):
break
sys.stdout.write(buf)
Oh, in case it's useful, here's an example telnet session done manually:
ubuntu:~/mac/python$ telnet localhost 5007
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
HELLO EMC 1 1
HELLO ACK EMCNETSVR 1.1
The first 'HELLO' line is what I typed, the second one is the response.
You probably need to terminate your msg with some kind of "line-ending characters" -- perhaps \r\n, perhaps just one of the two. When you're in telnet, didn't you terminate your typed text by hitting a Return or Enter key? In the Python code, you're not doing the equivalent of that.
You need to flush the socket right after the send, to force the TCP stack to actually send the data. Otherwise it will wait for more data to send, in order to fill a packet effectively. While you are waiting for a response from the server nothing has actually been sent yet.