I need to simulate a UDP server, which sends content of a text file line by line in an endless loop. I have written the code below but at the other end I do not receive anything (The other side is Qt code and I am sure it works):
import socket
import time
# setup UDP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sa = ('localhost', 9763)
sock.bind(sa)
filename = '/home/saeid/Documents/data.txt' # file to read
numlines = sum(1 for line in open(filename)) # get number of lines in file
# get line of a file
def get_line(line):
with open(filename) as fp:
for i, l in enumerate(fp):
if i == line:
return l
# main loop
while True:
currline = 0
while currline < numlines:
sock.sendto(get_line(currline), sa)
currline += 1
time.sleep(1)
I am no python pro and cant figure out the problem :(
For starters this is messed up:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sa = ('localhost', 9763)
sock.bind(sa)
...
sock.sendto(get_line(currline), sa)
By binding you basically say "I want to listen on that host/port". But then you send the data to the same host/port. I assume that there's some other destination address, e.g. sock.sendto(get_line(currline), ('my_host', 1234)). By the way, why are you binding to the address anyway? sock.bind(sa) line is unnecessary, remove it.
The other thing is that your file reading code is extremely inefficient and difficult to read (it took me a while to understand what's going on). Try something like this:
with open(filename, 'r') as fo:
while True:
for line in fo:
sock.sendto(line, DEST_ADDRESS)
time.sleep(1)
fo.seek(0) # go back to the begining of the file and repeat
and get rid of get_line function.
That's at least what I've came up with after reading your description. If you don't want to inifinitely send the same file then you can get rid of while True: loop and fo.seek(0) call.
Uncle google came up with User Datagram Client and Server
User Datagram Client and Server
The user datagram protocol (UDP) works differently from TCP/IP. Where TCP is a stream oriented protocol, ensuring that all of the data is transmitted in the right order, UDP is a message oriented protocol. UDP does not require a long-lived connection, so setting up a UDP socket is a little simpler. On the other hand, UDP messages must fit within a single packet (for IPv4, that means they can only hold 65,507 bytes because the 65,535 byte packet also includes header information) and delivery is not guaranteed as it is with TCP.
Echo Server
Since there is no connection, per se, the server does not need to listen for and accept connections. It only needs to use bind() to associate its socket with a port, and then wait for individual messages.
import socket
import sys
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
# Bind the socket to the port
server_address = ('localhost', 10000)
print >>sys.stderr, 'starting up on %s port %s' % server_address
sock.bind(server_address)
Messages are read from the socket using recvfrom(), which returns the data as well as the address of the client from which it was sent.
while True:
print >>sys.stderr, '\nwaiting to receive message'
data, address = sock.recvfrom(4096)
print >>sys.stderr, 'received %s bytes from %s' % (len(data), address)
print >>sys.stderr, data
if data:
sent = sock.sendto(data, address)
print >>sys.stderr, 'sent %s bytes back to %s' % (sent, address)
Echo Client
The UDP echo client is similar the server, but does not use bind() to attach its socket to an address. It uses sendto() to deliver its message directly to the server, and recvfrom() to receive the response.
import socket
import sys
# Create a UDP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
server_address = ('localhost', 10000)
message = 'This is the message. It will be repeated.'
try:
# Send data
print >>sys.stderr, 'sending "%s"' % message
sent = sock.sendto(message, server_address)
# Receive response
print >>sys.stderr, 'waiting to receive'
data, server = sock.recvfrom(4096)
print >>sys.stderr, 'received "%s"' % data
finally:
print >>sys.stderr, 'closing socket'
sock.close()
Client and Server Together
Running the server produces:
$ python ./socket_echo_server_dgram.py
starting up on localhost port 10000
waiting to receive message
received 42 bytes from ('127.0.0.1', 50139)
This is the message. It will be repeated.
sent 42 bytes back to ('127.0.0.1', 50139)
waiting to receive message
and the client output is:
$ python ./socket_echo_client_dgram.py
sending "This is the message. It will be repeated."
waiting to receive
received "This is the message. It will be repeated."
closing socket
$
You need to change the destination address or port in your call to sendto:
peer_address = ('localhost', peer_port)
sock.sendto(get_line(currline), peer_address)
To which address and port does the Qt application bind?
Related
I am trying to establish a client-server communication. The client is written in Ruby whereas the server is written in Python.
client.rb
require 'socket'
hostname = 'localhost'
port = 7778
s = TCPSocket.open(hostname, port)
s.write("2020-06-25T11:11:00+00:00 5 127.0.0.1 printer: event")
while line = s.gets
puts line.chop
end
s.close()
The ruby client sends a log to the Python server and tries to receive it back.
server.py
import socket
#Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#bind the socket to the port - tuple
server_address = ('localhost', 7778)
print('starting up on %s port %s' %server_address)
sock.bind(server_address)
#Listen for incoming connections
sock.listen(1)
while True:
print('waiting for a connection')
connection, client_address = sock.accept()
while True:
data = connection.recv(1024)
print('received "%s"' % data)
if data:
print('sending data back to the client')
connection.send(data)
else:
print('no more data from', client_address)
break
connection.close()
The log is sent to the python server and when the python server sends it back. When the ruby client receives it, it doesn't receive the full log.
example:
2020-06-25T11:11:00+00:00 5 127.0.0.1 printer: eve
I think this comes from the fact that TCP is a streaming protocol and we never know if we can get the full message each time.
Could you propose me a solution for both the client and the server so I can be sure they always receive the full message between each other? I would really appreciate it if anyone would help.
So the issue is that you're assuming the data received has a new line character - however the data you're sending is not terminated by a new line.
s.write("2020-06-25T11:11:00+00:00 5 127.0.0.1 printer: event") will not write the string with a new line character - you should use puts IO#puts
s.gets will return the data because the socket is closed by the python server after it has sent the data. So even getssays it will read the next line from the socket, in reality its just reading what remained in the buffer after the socket was closed.
line.chop will remove the last character, and you're using it here to strip a newline character (assuming that it has one from gets). However since there is no newline character it will remove the last character instead.
So the fix would be to replace in the ruby client s.write with s.puts.
From WireShark I can see that a UDP packet is being sent to my server containing data that I need.
Raw packet data from wireshark:
55aa001e03840000c864da6ea1a613422f7b18835780f2ac1f657fb39264405cc8a9f6f09291a7532b69ca128d549112002674027daf6fedf73ce5bd73e6e1e31a108970b8881da0c4a19149eed46a3a40870ed89e10a5e197283afa554a91d5ee6a24c96d375862da4a909832890049ce01778d63dddf3ae17d800421032f971d2d671232d1b64d45589b44663a39763575236160ca6bc26e66523cef74ffa1d248ed35329232132341442a0095ce012f7cdfacf71be1bd534d639e2ea78f6a293d46f7066886518cd6777d9b33d0b0d1f2c55701842c88740d0251ff6964487b5b26d949021229124900b26f012b84e2abdffac3bff9c4433d76079d03cdabc303a22045464f80cd25dd7f0bd2f59bf139c61484847621533df77092b06bb59a69a40a4448934800558d012774e29cf7dbd5bd657c87997a53be922e82a42d0736664caf04b66bbcdf6d2cbf3746eb041116768081a4b6f99092a96ed312dad9129110952400928f01757ce79bf71bd4bd2a41ef49a8bc12c940d96e44045032ea5ed74f6b5f76640fa9741ae80df8a4726d909082fb18da3259c4313365921425c9848019ca00e773635ad63791f95340d098253a6474c0cf05c8590402240140bd8286f7a7be0cab23e67dbb1d5feadbbe5225bbc9fe93b23532a49192c95044a59401006b093ae7f85679d779aa4124980000004190822c6441a38230783f0d7fb72d796fb284d74a93f6eda6ad09a741decafecce4a852499a484d8cd4424600805c685a6119d7f9de37b557ee34e978d6d288fe3d06bca9ff8e2a265eca4328c141b000440dd86a1f8499be010a18ff58d9112691c9646529582624c500980d02ef84ef8cdfbdd6bb3e84abdee3c36adfa9ac736e5b656d1de1eb7573e42cb2d30c2f28400cb8164080892a06ff28b591ad724924242551a9310a0019ae01ab6363abf73de3bd8ed73857beeab76ccd9a671cd761680070585e3e2297e3bc10d425094a1634308d2a8bd4ff6de9a2d32508538889244993a400338e012563ef69f73bd379a9cf47c9cb2a169c64cc6d81232c78125f4978265e31d3758f1b5f11dadabc72ffb3254ffecae5a93251a9d84809c998a844004dad01a56aeb6af75cd7bd2fd6b383467e23b5a3bac5c2d800288dd7e661451a4e887a906563a894d5a6b445c96d26df40d5a6a63125e94d06102504990069ad012754f32bf75ce7bfb18b8cffd72cc18eed938d4e02761a260fa466a129b06af27602a422fab2b6a705da8ca2deaad4ab5e63cc
Using python, how do I remove the first 20 characters:
55aa001e03840000c864
and save the rest?
Current code:
import socket
import sys
# Create a UDP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
# Bind the socket to the port
server_address = ('0.0.0.0', 8484)
sock.bind(server_address)
print >>sys.stderr, 'starting up on %s port %s' % server_address
while True:
print('\nwaiting to receive message')
data, address = sock.recvfrom(10240)
Since the within the data variable is of type bytes, which I think can be translated to something like b'<string_with_your_packet_data>', I think you can use slicing just as you would do on a standard string and it should work just fine.
So something among the lines of
while True:
print('\nwaiting to receive message')
data, address = sock.recvfrom(10240)
sliced_packet = data[20:] # trimming your first 20 characters from your data
I'm trying to send a stream of data via socket in Python. So far I manage to create a dummy_data_gen.py which sends a line containing 4 floats to the server.py. However, I'm still having issues in the stability of the all setup.
server.py:
import sys
import time
import socket
import numpy as np
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Bind the socket to the port
server_address = ('localhost', 5002)
print >>sys.stderr, 'starting up on %s port %s' % server_address
sock.bind(server_address)
# Listen for incoming connections
sock.listen(1)
# Create a list to store the incoming data
data = []
while True:
# Wait for a connection
print >>sys.stderr, 'waiting for a connection'
connection, client_address = sock.accept()
try:
print >>sys.stderr, 'connection from', client_address
while True:
incoming_data = connection.recv(48).split(',')
print incoming_data
event = float(incoming_data[0]), float(incoming_data[1]), float(incoming_data[2]), float(incoming_data[3])
data += [event]
time.sleep(0.01)
finally:
# Clean up the connection
connection.close()
dummy_data_gen.py
import sys
import time
import socket
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Connect the socket to the port where the server is listening
server_address = ('localhost', 5002)
sock.connect(server_address)
file = '../data/myfile.txt'
# Simulating a real-time data stream at 100 Hz
try:
with open(file) as f:
for line in f:
sock.sendall(line)
time.sleep(0.01)
finally:
print >>sys.stderr, 'closing socket'
sock.close()
My problem is that sometimes the communication is working properly, however, I have situations where I receive more data per line than I should. In the following output example the first 7 lines are correct, however the following lines are incorrect and therefore problematic:
['391910745379', '24.134277', '-1.9487305', '-117.373535', '\n']
['391920745379', '24.434082', '-1.3491211', '-117.373535', '\n']
['391930745379', '23.68457', '-0.5996094', '-116.62402', '\n']
['391940745379', '24.434082', '-1.0493164', '-115.57471', '\n']
['391950745379', '24.134277', '-1.0493164', '-116.47412', '\n']
['391960745379', '23.234863', '-1.0493164', '-116.47412', '\n']
['391970745379', '24.583984', '-0.89941406', '-116.92383', '\n']
['391980745379', '23.384766', '-0.2998047', '-116.62402', '\n39']
['1990745379', '23.68457', '-0.5996094', '-115.72461', '\n39200']
['0745379', '23.834473', '-0.44970703', '-115.87451', '\n392010']
['745379', '23.534668', '-1.0493164', '-114.9751', '\n392020745']
['379', '23.384766', '-1.7988281', '-115.72461', '\n39203074537']
['9', '22.935059', '-0.44970703', '-114.9751', '\n392040745379', '']
I tried to play with the connection.recv bytes but I'm still facing this issue.
EDIT1: Following some suggestions I modified the server.py as follows:
del_message = '\n'
del_stream = ','
while True:
_buffer += connection.recv(1)
if del_message in _buffer:
incoming_data = _buffer.split(del_stream)
event = float(incoming_data[0]), \
float(incoming_data[1]), \
float(incoming_data[2]), \
float(incoming_data[3])
This approach seems to solve my issue, however the performance is extremely slow. My files contains approximately 6300 lines that were actually sent in 70 seconds (time interval at which the socket was closed on my dummy data generator). However, I took almost 10 minutes to receive all of the 6300 lines. It seems also that I receive more samples per second on the beginning rather that on the end of the stream.
If you have a message protocol that terminates messages with a newline, you need to write some code to implement that protocol. It won't work by magic.
You need a "receive a message" function, where "message" is defined as "a sequence of bytes delimited by a newline". You've never written any such function, so you're not receiving messages but just the chunks of bytes you're sending.
I am using TCP sockets to communicate between my server and clients. The server code and socket code are as below:
server:
from socket import *
HOST = 'xx.xx.xx.xx'
PORT = 1999
serversocket = socket(AF_INET,SOCK_STREAM)
serversocket.bind((HOST,PORT))
print 'bind success'
serversocket.listen(5)
print 'listening'
while True:
(clientsocket, address) = serversocket.accept()
print ("Got client request from",address)
#clientsocket.send('True')
data = clientsocket.recv(1024)
print data
clientsocket.send('True')
clientsocket.close()
client:
import socket
import sys
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Connect the socket to the port on the server given by the caller
server_address = ('xx.xx.xx.xx', 1999)
print >>sys.stderr, 'connecting to %s port %s' % server_address
sock.connect(server_address)
try:
message = 'This is the message. It will be repeated.'
print >>sys.stderr, 'sending'
for x in range (0,1):
name=raw_input ('what is ur name')
print type(name)
sock.send(name)
print sock.recv(1024)
finally:
sock.close()
I am able to communicate with the server from client and able to send and receive data. But the problem I am facing is that I am not able to send and receive data continuously from the server. I have to restart my client code on my laptop to send and receive data again from the server. The way the above client code is working is that when I give a keyboard input, then the socket sends data to server and server responds back. But in the client code, in the for loop if I do two iterations, for the second iteration the data I enter from keyboard is not reaching server. I need to restart my client code to send data again. How do I fix this ?
Also, when once client is connected to the server, the other cannot connect to the server. Any ideas on how to do this ?
You need to design and implement a protocol that specifies what each side is supposed to do and then implement that protocol. You're expecting it to work by magic.
For example:
data = clientsocket.recv(1024)
I suspect you are expecting this to receive a "message". But TCP has no notion of messages. If you need to send and receive messages, you need to define precisely what a "message" is for your protocol and write code to send and receive them.
It may be helpful to look at the specifications for other protocols that use TCP such as HTTP, FTP, or IRC. It really is worth the time to write out a specification of your protocol before you write any code. It will save a lot of pain.
I am learning socket programming using python. my first assignment is to a write a client.py and a server.py. The client sends a message to server. The server receives the message of 16 bytes each time. After it has received the entire message, it will send the same message back to client.
so it is very simple. The server has backlog of 1. After the server sends the message to client, the connection to client close and the server should be open to receive new connection.
my current code fails in the last step. It is not open to receive new connections. It is throwing error. I even figured out the error. but I do not know how to fix this.
The error comes from server.py because I call for sock.accept() but I have closed the sock.
Let me explain my server.py code: I have two while loops. The outer loop looks for new connection, and the inner loop looks process request from connections i.e it simply receives data, wait till everything is received and send it back to client and finally close the connection.
I am asked not to change the structure of two while loops but just implement them.
Any thoughts or ideas on this:
client.py
import socket
import sys
def client(msg, log_buffer=sys.stderr):
server_address = ('localhost', 10000)
sock=socket.socket(socket.AF_INET,socket.SOCK_STREAM,socket.IPPROTO_IP)
sock.connect(server_address)
print >>log_buffer, 'connecting to {0} port {1}'.format(*server_address)
try:
print >>log_buffer, 'sending "{0}"'.format(msg)
sock.sendall(msg)
chunk = ''
done=False;
while not done:
chunk+=sock.recv(16)
if chunk==msg:
done=True
print >>log_buffer, 'received "{0}"'.format(chunk)
finally:
print >>log_buffer, 'closing socket'
sock.close()
if __name__ == '__main__':
if len(sys.argv) != 2:
usg = '\nusage: python echo_client.py "this is my message"\n'
print >>sys.stderr, usg
sys.exit(1)
msg = sys.argv[1]
client(msg)
Server.py
import socket
import sys
def server(log_buffer=sys.stderr):
# set an address for our server
address = ('127.0.0.1', 10000)
sock=socket.socket(socket.AF_INET,socket.SOCK_STREAM,socket.IPPROTO_IP)
sock.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
# log that we are building a server
print >>log_buffer, "making a server on {0}:{1}".format(*address)
sock.bind(address)
sock.listen(1)
try:
# the outer loop controls the creation of new connection sockets. The
# server will handle each incoming connection one at a time.
while True:
print >>log_buffer, 'waiting for a connection'
conn,add=sock.accept()
addr=(conn,add)
try:
print >>log_buffer, 'connection - {0}:{1}'.format(*addr)
# the inner loop will receive messages sent by the client in
# buffers. When a complete message has been received, the
# loop will exit
data = ''
while True:
recvdata=conn.recv(16)
print recvdata
data+=recvdata
print >>log_buffer, 'received "{0}"'.format(data)
print >>log_buffer, "len of received data: {0}".format(len(recvdata))
if len(recvdata)<16:
print >>log_buffer,"sending data"
conn.sendall(data)
break
conn.close()
finally:
sock.close()
except KeyboardInterrupt:
sock.close()
if __name__ == '__main__':
server()
sys.exit(0)
I runpython server.py in one terminal andpython client.py "This is the first message. send me back"` in a different terminal. The client connection is lost normally as expected. But I get the following error at server side (towards the end):
making a server on 127.0.0.1:10000
waiting for a connection
connection - <socket._socketobject object at 0x100849c20>:('127.0.0.1', 50626)
sairam hopefully
received "sairam hopefully"
len of received data: 16
this works lets
received "sairam hopefully this works lets"
len of received data: 16
c
received "sairam hopefully this works lets c"
len of received data: 2
sending data
waiting for a connection
Traceback (most recent call last):
File "echo_server.py", line 89, in <module>
server()
File "echo_server.py", line 39, in server
conn,add=sock.accept()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 202, in accept
sock, addr = self._sock.accept()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 170, in _dummy
raise error(EBADF, 'Bad file descriptor')
socket.error: [Errno 9] Bad file descriptor
You are closing sock inside your while loop. Don't do that. sock is your long-lasting server socket, which needs to remain open to listen for new connections. conn is your ephemeral socket, which needs to remain open only the length of a single connection.
Close conn after each connection, close sock when the server needs to terminate.
More simply, replace these lines:
finally:
sock.close()
with
finally:
conn.close()
What you are trying to do is a simple echo server, which I believe you can implement much more simply.
Server:
import socket
host = ''
port = 50000
backlog = 5
size = 1024
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host,port))
s.listen(backlog)
while 1:
client, address = s.accept()
data = client.recv(size)
if data:
client.send(data)
client.close()
Client:
import socket
host = 'localhost'
port = 50000
size = 1024
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host,port))
s.send('Hello, world')
data = s.recv(size)
s.close()
print 'Received:', data