We Keep Coding

Select call only working on tcp inputs twice. How to properly use select with tcp and udp server?

I am creating a server which using select() for non-blocking i/o in Python.
I have a server which has select with inputs being a TCP connection and a UDP connection. Within the TCP connection, there may be calls to other UDP servers.
The problem is that my server only checks for TCP input twice when I am running the server and some data transfer with other UDP servers (not the server's own UDP sever) takes place. If there is no need for UDP transfer then there it is working as many times as it is invoked by the web browser.
My server seems to be working fine other than that weird situation where I can only send the TCP request twice.
I also suspect that I am able to send it twice because of some favicon request and actual request by the browser which is making 's' twice as inputs and isn't working after it. I also think that there might be an issue on how I am using select for server's own UDP connection.
This is my first time writing a server program so there might be some straightforward issue with how I am dealing with select function and sockets in general.
inputs = [ server, udp ]
# Sockets to which we expect to write
outputs = [ ]
# Outgoing message queues (socket:Queue)
message_queues = {}
while inputs:
# Wait for at least one of the sockets to be ready for processing
print('\nwaiting for the next event')
readable, writable, exceptional = select.select(inputs, outputs, inputs)
This is somewhat how my loop for readable inputs look:
if s is server:
connection, client_address = s.accept()
connection.setblocking(0)
inputs.append(connection)
message_queues[connection] = queue.Queue()
elif s != udp:
print(readable)
try:
data = s.recv(4096)
if data:
# do something
message_queues[s].put(data)
if s not in outputs:
outputs.append(s)
else:
# Interpret empty result as closed connection
print('closing', client_address, 'after reading no data')
if s in outputs:
outputs.remove(s)
inputs.remove(s)
s.close()
del message_queues[s]
except Exception:
if s in outputs:
outputs.remove(s)
inputs.remove(s)
s.close()
if s is udp:
print(s)
udpdata,addr = s.recvfrom(UDP_PORT_NO)
This is somewhat how my loop for writeable looks:
for s in writable:
try:
next_msg = message_queues[s].get_nowait()
if (something):
udp_others = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
try:
# Send data
sent = udp_others.sendto(packet.encode(), ('localhost', destination_port))
# Receive response
clientdata, server = udp_test.recvfrom(4096)
# do something with data
finally:
print ('closing socket')
udp_others.close()
next_msg = "Hello"
except queue.Empty:
# No messages waiting so stop checking for writability.
print('output queue for', s.getpeername(), 'is empty')
outputs.remove(s)
else:
print(response)
# Handle "exceptional conditions"
for s in exceptional:
print >>sys.stderr, 'handling exceptional condition for', s.getpeername()
# Stop listening for input on the connection
inputs.remove(s)
if s in outputs:
outputs.remove(s)
s.close()
# Remove message queue
del message_queues[s]

Send and receive messages on same port? Peer to peer message app python

I'm trying to create a peer to peer message app, I understand I need each instance of the app to be both a server and a client as I've got for the below code but I'm wondering how to set up the ports, can I send and receive messages on the same port?
The below code is one instance of the app, I can communicate with another version but I have to set the other version to send messages on port 9000 and receive messages on 6190. This won't work going forward as how would a third user connect?
Current situation:
User 1: Receives on 9000, sends on 6190
User 2: Receives on 6190, sends on 9000
import socket
import time
import threading
global incoming
def server_socket(): #call server_socket() in build method?
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('', 9000))
s.listen(1)
host_name = socket.gethostname()
ip_address = socket.gethostbyname(host_name)
print("IP address is: ", ip_address)
except socket.error as e:
print("Socket Error !!", "Unable To Setup Local Socket. Port In Use")
while True:
conn, addr = s.accept()
incoming_ip = str(addr[0])
data = conn.recv(4096)
data = data.decode('utf-8')
print("message recieved is: ", data)
conn.close()
s.close()
def client_send_message():
message = "Hello World"
message = message.encode('utf-8')
c = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
c.connect(("127.0.0.1", 6190))
except Exception as e:
print("Connection Refused", "The Address You Are Trying To Reach Is Currently Unavailable")
try:
c.send(message)
except Exception as e:
print(e)
c.close()
t = threading.Thread(target=server_socket)
t.start()
for i in range(5):
time.sleep(30)
client_send_message()

You currently use TCP and with this design you need a separat socket for each client. You can exchange data on this socket in both directions though. More common for peer to peer networks is UDP: here you can use a single socket to recvfrom data from arbitrary clients and sendto data to arbitrary clients.

python3 - broken pipe error when using socket.send()

I am programming a client-server instant message program. I created a similar program in Python 2, and am trying to program it in Python 3. The problem is when the server takes the message and tries to send it to the other client, it gives me "[Errno 32] Broken Pipe" and exits.
I have done some research, and found that this occurs when the client disconnects, so I did some more testing but could not find when the client disconnects. (I am using Ubuntu 14.04 and Python 3.4)
Here is the server code:
import socket, select, sys
def broadcast(sock, messaged):
for socket in connection_list:
if socket != s and socket != sock:
# Here is where it gives me the broken pipe error
try:
s.send(messaged.encode("utf-8"))
except BrokenPipeError as e:
print(e)
sys.exit()
connection_list = []
host = ''
port = 5558
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((host,port))
s.listen(5)
connection_list.append(s)
read_sockets,write_sockets,error_sockets = select.select(connection_list,[],[])
while True:
for sock in read_sockets:
if sock == s:
conn, addr = s.accept()
connection_list.append(conn)
client = "Client (%s,%s) connected" % addr
print(client)
broadcast(sock,client)
else:
try:
data = sock.recv(2048)
decodeddata = data.decode("utf-8")
if data:
broadcast(sock, decodeddata)
except:
offline = "Client " + addr + "is offline"
broadcast(sock, offline)
print(offline)
connection_list.remove(sock)
sock.close()
continue
And the client code:
import socket, select, string, sys, time
def prompt(data) :
print("<You> " + data)
def Person(data) :
print("<Receiver> " + data)
if __name__ == "__main__":
host = "localhost"
port = 5558
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(2)
try:
s.connect((host,port))
except:
print('Unable to connect')
sys.exit()
print('Connected.')
socket_list = [s]
read_sockets,write_sockets,error_sockets = select.select(socket_list,[],[])
while 1:
for sock in read_sockets:
if sock == s:
try:
time.sleep(1)
data = sock.recv(1024)
Person(data.decode("utf-8"))
except:
msg = input("Send a message: ")
try:
s.send(str.encode(msg))
except:
print("Server is offline")
sys.exit()
else:
print("Server is offline")
sys.exit()

There are two problems that you have to fix to make this work.
First, on both the client side and the server side, you have to put the select inside the loop, not outside. Otherwise, if there was something to read before you got to the loop, you'll recv over and over, and if there wasn't, you'll never recv. Once you fix this, you can get rid of the time.sleep(1). (You should never need a sleep to solve a problem like this; at best it masks the problem, and usually introduces new ones.)
Meanwhile, on the server side, inside broadcast, you're doing s.send. But s is your listener socket, not a connected client socket. You want socket.send here, because socket is each socket in connection_list.
There are a number of unrelated problems in your code as well. For example:
I'm not sure what the except: in the client is supposed to be catching. What it mainly seems to catch is that, about 50% of the time, hitting ^C to end the program triggers the send prompt. But of course, like any bare except:, it also masks any other problems with your code.
There's no way to send any data back and forth other than the "connected" message except for that except: clause.
addr is a tuple of host and port, so when someone goes offline, the server raises a TypeError from trying to format the offline message.
addr is always the last client who connected, not the one who's disconnecting.
You're not setting your sockets to nonblocking mode.
You're not checking for EOF on the recv. This means that you don't actually detect that a client has gone offline until you get an error. Which normally happens only after you try to send them a message (e.g., because someone else has connected or disconnected).

Send/receive Packets with TCP sockets

Recently, I managed to create sockets on my PC and my Raspberry Pi to enable communication between both devices. Currently, the client is able to automatically send messages to the server. I was wondering, if it is possible to modify the scripts to send tcp data packets instead of purely text messages, as I would very much like to control the raspberry pi using my PC in the future without having the need to ssh/etc.
I've looked at some examples, but as I don't have much experience in writing my own scripts/codes, I'm not very sure how to go about doing this. I would appreciate if someone could guide me in the right direction with explanation and some examples if possible.
Anyway here is the server/client script I'm running at the moment:
Client:
import socket
import sys
import struct
import time
#main function
if __name__ == "__main__":
if(len(sys.argv) < 2) :
print 'Usage : python client.py hostname'
sys.exit()
host = sys.argv[1]
port = 8888
#create an INET, STREAMing socket
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error:
print 'Failed to create socket'
sys.exit()
print 'Socket Created'
try:
remote_ip = socket.gethostbyname( host )
s.connect((host, port))
except socket.gaierror:
print 'Hostname could not be resolved. Exiting'
sys.exit()
print 'Socket Connected to ' + host + ' on ip ' + remote_ip
#Send some data to remote server
message = "Test"
try :
#Set the whole string
while True:
s.send(message)
print 'Message sent successfully'
time.sleep(1)
print 'Sending...'
except socket.error:
#Send failed
print 'Send failed'
sys.exit()
def recv_timeout(the_socket,timeout=2):
#make socket non blocking
the_socket.setblocking(0)
#total data partwise in an array
total_data=[];
data='';
#beginning time
begin=time.time()
while 1:
#if you got some data, then break after timeout
if total_data and time.time()-begin > timeout:
break
#if you got no data at all, wait a little longer, twice the timeout
elif time.time()-begin > timeout*2:
break
#recv something
try:
data = the_socket.recv(8192)
if data:
total_data.append(data)
#change the beginning time for measurement
begin=time.time()
else:
#sleep for sometime to indicate a gap
time.sleep(0.1)
except:
pass
#join all parts to make final string
return ''.join(total_data)
#get reply and print
print recv_timeout(s)
s.close()
Server:
import socket
import sys
from thread import *
HOST = '' # Symbolic name meaning all available interfaces
PORT = 8888
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print 'Socket created'
try:
s.bind((HOST, PORT))
except socket.error , msg:
print 'Bind failed. Error Code : ' + str(msg[0]) + ' Message ' + msg[1]
sys.exit()
print 'Socket bind complete'
s.listen(10)
print 'Socket now listening'
#Function for handling connections
def clientthread(conn):
#Sending message to connected client
conn.send('Welcome to the server. Receving Data...\n') #send only takes string
#infinite loop so that function do not terminate and thread do not end.
while True:
#Receiving from client
data = conn.recv(1024)
reply = 'Message Received at the server!\n'
print data
if not data:
break
conn.sendall(reply)
conn.close()
#now keep talking with the client
while 1:
#wait to accept a connection
conn, addr = s.accept()
print 'Connected with ' + addr[0] + ':' + str(addr[1])
#start new thread
start_new_thread(clientthread ,(conn,))
s.close()

socket.socket(socket.AF_INET, socket.SOCK_STREAM) already creates a connection that provides a reliable stream of bytes between two machines. This uses TCP, which is on top of IP and Ethernet. The latter two are package-based, while TCP creates a stream of continuous bytes on top of it. It also adds some error checking and error correction, so it is pretty reliable.
I honestly don't understand what you want to achieve with what you call "send packets". What you don't want to do is to create an implementation of TCP yourself, as that's a non-trivial task, so sending RAW packets is out. In general, even using TCP is already relatively low-level and should be avoided unless really necessary.
Using e.g. ZeroMQ you get a message-based interface that does all the transmission for you. It does so on top of TCP (or other transports) and adds more error correction for e.g. disconnects. There, you also have something like "packets", but those are independent of how many TCP or IP packets were required to send it underneath. If you don't want to implement a specific protocol, I'd suggest you use this framework instead of lowlevel TCP sockets.
Another simple alternative is to use HTTP, for which there is also existing code in Python. The downside is that it is always one side that initiates some communication and the other side only replies. If you want some kind of active notification, you either have to poll or use hacks like delaying an answer.

You are already sending data packets - those packets juts happen to contain text data at the moment. Try looking into pickle in the standard libraries and into pyro.

Server send data in python

I'm writing a simple client server app in python, where the client is listening every type of data entering in the specific port, and I want to when receiving a data flow, send back to the connected client (which have a dinamic ip) a string, in this case "001". But when I try to send the message, it fails!
#!/usr/bin/env python
import socket
TCP_IP = '192.168.1.115'
TCP_PORT = 55001
BUFFER_SIZE = 1024
MESSAGE = '01'
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((TCP_IP, TCP_PORT))
s.listen(1)
conn, addr = s.accept()
print ('Connection address:', addr)
while 1:
data = conn.recv(BUFFER_SIZE)
if not data: break
print ('received data:', data)
conn.send(data) # echo
print ('Sending data to client...')
addr change every connection .. i cannot manage this!
s.connect((addr, TCP_PORT))
s.send(MESSAGE)
data = s.recv(BUFFER_SIZE)
s.close()

(Connected stream) sockets are bidirectional, so there's no need to call connect to get a connection to the client—you already have one.
But you want to know why your code fails. And there are at least three problems with it.
First, after you call listen or connect on a socket, you can't call connect again; you will get an exception (EISCONN on POSIX, something equivalent on Windows). You will have to create a new socket.
Second, is client actually binded and listening for a connection on the same port as the server? If not, your connect can't work. If so, the bind will fail if the client and server are on the same machine.
Third, the addr you get back from accept is a (host, port) pair, not just a host. So, as written, you're trying to connect((('192.168.1.115', 12345), 55001)), which obviously isn't going to work.

You are trying to reply to the client using the server listening socket (s). This is only possible in UDP Servers. Since this is a TCP Server you have to use the conn which is crated using s.accept() to communication with remote client.