We Keep Coding

How to break the while loop if data is not receiving and data length is unknow in python [TCP]

I am new in python and socket programming. I am trying to send data to the client, and the length of data is unknown. After receiving all the data on client side, the program is not terminating because of a while loop (See below code). I also use the command (if not message: break) but it is also not working.
The second one is the packet loss issue. When I do not give time. Sleep () at the sender side then receiver miss some packet(does not receive all packets at receiver side where the sender sends all packets).
How I can come out from while loop without sys.exit command?.
and ....How I can handle the second issue without using time.sleep function.
It is appreciable if anybody can help me.
Thanks
[Receiver Node]
import socket
import os,sys
def frame_reception_function ():
while True:
PORT = 123
s=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
s.bind(('',PORT))
s.listen()
conn,address=s.accept()
message=conn.recv(4096).decode()
print (message)
conn.close()
frame_reception_function()
[Sender node]
import os,sys
import socket
import time
MyNeighborSet_ip= ['192.168.1.2']
Data_transfer_listt = [['192.168.1.1', '192.168.1.2'], ['192.168.1.2', '192.168.1.3'], ['192.168.1.2', '192.168.1.4'], ['192.168.1.4', '192.168.1.5'], ['192.168.1.4', '192.168.1.6']]
def sending_Neighobr_ip_list():
#nn1=n1
message='Neighbor_list_sending'
#print (len(Data_transfer_listt))
for i in range(len(Data_transfer_listt)):
receiver_ip=Data_transfer_listt[i][0]
receiver_node_list=Data_transfer_listt[i][1]
T_message= message + ";" + receiver_ip + ";" + receiver_node_list
T_message_bytes= bytes(T_message,'utf-8')
PORT = 123
print ("just after socket")
for k in range (len(MyNeighborSet_ip)):
s=socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
s.connect((MyNeighborSet_ip[k],PORT))
s.sendall (T_message_bytes)
s.close()
time.sleep(0.01)
sending_Neighobr_ip_list()

It is very resource intensive to keep opening and closing a socket, I am going to take those out of the while loop. Also, I can't remember if recv is a blocking function or not. But this should help you break out of the while loop properly and solve your first problem.
Receiver:
def frame_reception_function ():
data = []
PORT = 123
s=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
s.bind(('0.0.0.0',PORT))
s.listen(1)
conn,address=s.accept() # accept an incoming connection using accept() method which will block until a new client connects
while True:
datachunk=conn.recv(1024) # reads data chunk from the socket in batches of 4086 bytes using method recv() until it returns an empty string
if not datachunk:
break # no more data coming in, so break out of the while loop
data.append(datachunk) # add chunk to your already collected data
conn.close()
print(data)
return
frame_reception_function()
Sender:
import os,sys
import socket
import time
MyNeighborSet_ip= [&ltTHE IP ADDRESS OF YOUR RECEIVER&gt]
Data_transfer_listt = [['192.168.1.1', '192.168.1.2'], ['192.168.1.2', '192.168.1.3'], ['192.168.1.2', '192.168.1.4'], ['192.168.1.4', '192.168.1.5'], ['192.168.1.4', '192.168.1.6']]
def sending_Neighobr_ip_list():
#nn1=n1
message='Neighbor_list_sending'
#print (len(Data_transfer_listt))
PORT = 123
for k in range (len(MyNeighborSet_ip)):
s=socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((MyNeighborSet_ip[k],PORT))
s.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
for i in range(len(Data_transfer_listt)):
receiver_ip=Data_transfer_listt[i][0]
receiver_node_list=Data_transfer_listt[i][1]
T_message= message + ";" + receiver_ip + ";" + receiver_node_list
T_message_bytes= bytes(T_message)
print("sending message")
s.sendall (T_message_bytes)
s.close()
sending_Neighobr_ip_list()
For THE IP ADDRESS OF YOUR RECEIVER, I used my local IP address (192.168.x.x).
This is the output on the receiver side:
['Neighbor_list_sending;192.168.1.1;192.168.1.2', 'Neighbor_list_sending;192.168.1.2;192.168.1.3', 'Neighbor_list_sending;192.168.1.2;192.168.1.4', 'Neighbor_list_sending;192.168.1.4;192.168.1.5', 'Neighbor_list_sending;192.168.1.4;192.168.1.6']
The problem is on the sender side, you are closing the socket every iteration. That makes the receiver close its connection too. So sender successfully sends the first message, then errors out on the second connection cause receiver has been closed and is not looking for a connection. Instead, switch the for loops order. make the connection first, then send your messages, then close the connection.

sending images from client to server using socket python

I'm trying to send multiple images from client to server .
from my client I send one image at a time then for each image I get the size in the server and then send the size back to client and then try and store all the sizes of all images in a table .
I wrote this code and it doesn't seem to work:
client.py:
from PIL import Image
import glob
import sys
import pickle
import socket
import os
import numpy
reload(sys)
def readFileImages(strFolderName):
st = os.path.join(strFolderName, "*.png")
print st
return glob.glob(st)
client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client_socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
client_socket.bind(("127.0.0.1", 4000))
list1=readFileImages("test")
myoutput =[]
while (list1):
for im in list1:
f=open(im,"rb")
while True:
veri = f.read()
if not veri:
break
client_socket.send(veri)
f.close()
data = client_socket.recv(4096)
data_arr=pickle.loads(data)
newrow=numpy.asarray(data_arr)
myoutput=numpy.vstack([myoutput,newrow])
client_socket.close()
numpy.savetxt("testTable.csv",myoutput,delimiter=",")
server.py:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1);
s.bind(("127.0.0.1",4000))
s.listen(5)
client_socket, address = s.accept()
print "Connected to - ",address,"\n"
fname="test.png"
fp = open(fname,'wb')
# image
while True:
strng = client_socket.recv(1024)
if not strng:
break
fp.write(strng)
fp.close()
#T[0]=detect_carte_grise(fp)
im = Image.open(fp)
T= im.size #width,height
data=pickle.dumps(T)
client_socket.send(data)
and why do i get this error ?:[errno98] address already in use
I cannot even connect to server

First, in server code you bind to port, but in client code, you need to CONNECT to that server. You are binding in both of your scripts and address is already used by the first running script. so in client drop bind and change to client_socket.connect(("127.0.0.1", 4000)). That will resolve current issues, if you have any more, please, ask another question.

I got the same error, I Changed the "port number". It's worked fine

Python efficient socket communication

i recently started making a pure skype resolver and after doing everything fine i stuck on the socket communication.
Let me explain
I'm using python to get the user's IP and then the script opens a socket server and it sends the username to an other program written in .NET
Why is that? Well, the python skype API is not that powerfull so i'm using the axSkype library in order to gather more info.
The problem
The python socket sends the username as it should but i dont know the most efficient way to get the info back. I was thinking opening a socket server in the same script and wait for what the .NET program sends back.
I dont really kwon how to make this as fast as possible so i'm asking for your help.
The code
class api:
def GET(self, username):
skypeapi.activateSkype(username)
time.sleep(1) # because skype is ew
buf = []
print("==========================")
print("Resolving user " + username)
#This is where i'm starting the socket and sending data
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("127.0.0.1", 5756))
s.sendall(username)
s.close()
#at this poaint i want to get data back from the .NET app
for logfile in glob.glob('*.log'):
buf += logparse.search(logfile, username)
print("Done!")
print("==========================")
return json.dumps(buf)
class index:
def GET(self):
return render.index()
if __name__ == "__main__":
app.run()

You can bind your socket to the connection. This way, your socket stream will remain open and you will be able to send and receive information easily. Integrate this with the _thread module and you will be able to handle multiple streams. Here is some example code that binds a socket to a stream and just sends back whatever the clients sends it(Although in your case you could send whatever data is necessary)
import socket
from _thread import *
#clientHandle function will just receive and send stuff back to a specific client.
def clientHandle(stream):
stream.send(str.encode("Enter some stuff: "))
while True:
#Here is where the program waits for a response. The 4000 is a buffer limit.
data = stream.recv(4000)
if not data:
#If there is not data, exit the loop.
break
stream.senddall(str.encode(data + "\n"))
#Creating socket.
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host = "" #In this case the host is the localhost but you can put your host
port = 80
try:
#Here the program tries to bind the socket to the stream.
s.bind((host, port))
except socket.error as e:
print("There was an error: " + str(e))
#Main program loop. Uses multithreading to handle multiple clients.
while True:
conn, addr = s.accept()
print("Connected to: " + addr[0] + ": " + str(addr[1]))
start_new_thread(clientHandle,(conn,))
Now in your case, you can integrate this into your api class(Is that where you want to integrate it? Correct me if I'm wrong.). So now when you define and bind your socket, use this code:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host, port))
Where, in your case, host is 127.0.0.1, in other words, your localhost, which can also be accessed by socket.gethostbyname(socket.gethostname())(but that's a bit verbose), and then port, which for you is 5756. Once you have bounded your socket, you have to accept connections through the following syntax:
conn, addr = s.accept()
Which then you can pass conn and addr to whatever function or just use in any other code.
Regardless of what you use it in, to receive data you can use socket.recv() and pass it a buffer limit. (Remember to decode whatever you receive.) And of course, you send data by using socket.sendall().
If you combine this with the _thread module, as shown above, you can handle multiple api requests, which could come handy in the future.
Hope this helps.

Send/receive Packets with TCP sockets

Recently, I managed to create sockets on my PC and my Raspberry Pi to enable communication between both devices. Currently, the client is able to automatically send messages to the server. I was wondering, if it is possible to modify the scripts to send tcp data packets instead of purely text messages, as I would very much like to control the raspberry pi using my PC in the future without having the need to ssh/etc.
I've looked at some examples, but as I don't have much experience in writing my own scripts/codes, I'm not very sure how to go about doing this. I would appreciate if someone could guide me in the right direction with explanation and some examples if possible.
Anyway here is the server/client script I'm running at the moment:
Client:
import socket
import sys
import struct
import time
#main function
if __name__ == "__main__":
if(len(sys.argv) < 2) :
print 'Usage : python client.py hostname'
sys.exit()
host = sys.argv[1]
port = 8888
#create an INET, STREAMing socket
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error:
print 'Failed to create socket'
sys.exit()
print 'Socket Created'
try:
remote_ip = socket.gethostbyname( host )
s.connect((host, port))
except socket.gaierror:
print 'Hostname could not be resolved. Exiting'
sys.exit()
print 'Socket Connected to ' + host + ' on ip ' + remote_ip
#Send some data to remote server
message = "Test"
try :
#Set the whole string
while True:
s.send(message)
print 'Message sent successfully'
time.sleep(1)
print 'Sending...'
except socket.error:
#Send failed
print 'Send failed'
sys.exit()
def recv_timeout(the_socket,timeout=2):
#make socket non blocking
the_socket.setblocking(0)
#total data partwise in an array
total_data=[];
data='';
#beginning time
begin=time.time()
while 1:
#if you got some data, then break after timeout
if total_data and time.time()-begin > timeout:
break
#if you got no data at all, wait a little longer, twice the timeout
elif time.time()-begin > timeout*2:
break
#recv something
try:
data = the_socket.recv(8192)
if data:
total_data.append(data)
#change the beginning time for measurement
begin=time.time()
else:
#sleep for sometime to indicate a gap
time.sleep(0.1)
except:
pass
#join all parts to make final string
return ''.join(total_data)
#get reply and print
print recv_timeout(s)
s.close()
Server:
import socket
import sys
from thread import *
HOST = '' # Symbolic name meaning all available interfaces
PORT = 8888
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print 'Socket created'
try:
s.bind((HOST, PORT))
except socket.error , msg:
print 'Bind failed. Error Code : ' + str(msg[0]) + ' Message ' + msg[1]
sys.exit()
print 'Socket bind complete'
s.listen(10)
print 'Socket now listening'
#Function for handling connections
def clientthread(conn):
#Sending message to connected client
conn.send('Welcome to the server. Receving Data...\n') #send only takes string
#infinite loop so that function do not terminate and thread do not end.
while True:
#Receiving from client
data = conn.recv(1024)
reply = 'Message Received at the server!\n'
print data
if not data:
break
conn.sendall(reply)
conn.close()
#now keep talking with the client
while 1:
#wait to accept a connection
conn, addr = s.accept()
print 'Connected with ' + addr[0] + ':' + str(addr[1])
#start new thread
start_new_thread(clientthread ,(conn,))
s.close()

socket.socket(socket.AF_INET, socket.SOCK_STREAM) already creates a connection that provides a reliable stream of bytes between two machines. This uses TCP, which is on top of IP and Ethernet. The latter two are package-based, while TCP creates a stream of continuous bytes on top of it. It also adds some error checking and error correction, so it is pretty reliable.
I honestly don't understand what you want to achieve with what you call "send packets". What you don't want to do is to create an implementation of TCP yourself, as that's a non-trivial task, so sending RAW packets is out. In general, even using TCP is already relatively low-level and should be avoided unless really necessary.
Using e.g. ZeroMQ you get a message-based interface that does all the transmission for you. It does so on top of TCP (or other transports) and adds more error correction for e.g. disconnects. There, you also have something like "packets", but those are independent of how many TCP or IP packets were required to send it underneath. If you don't want to implement a specific protocol, I'd suggest you use this framework instead of lowlevel TCP sockets.
Another simple alternative is to use HTTP, for which there is also existing code in Python. The downside is that it is always one side that initiates some communication and the other side only replies. If you want some kind of active notification, you either have to poll or use hacks like delaying an answer.

You are already sending data packets - those packets juts happen to contain text data at the moment. Try looking into pickle in the standard libraries and into pyro.

Can't send correct value over a socket using Python

I'm new to using Python and sockets in general (only started yesterday) so I've been having a lot of issues trying to set up a TCP client and server. The issue I'm having is that I want to send a key from the server to the client. I know that the server grabs the key correctly as it prints out the correct key, however it has a 0 appended to it in a new line and when the key is sent to the client the only thing that is displayed is " b'0' ".
I've made very little progress due to my lack of experience and after searching for hours I still haven't found a solution to my problem.
Here is the server code:
import os
from socket import * #import the socket library
HOST = '' #We are the host
PORT = 29876
ADDR = (HOST, PORT)
BUFFSIZE = 4096
message = 'Hello, World!'
serv = socket( AF_INET,SOCK_STREAM)
serv.bind(ADDR,)
serv.listen(5)
print ('listening...')
conn,addr = serv.accept()
print (conn,addr)
print ('...connected')
key = os.system("cat ~/.ssh/id_rsa.pub")
conn.send(str(key))
print (key)
conn.close()
Here is the client code
from socket import *
import os
HOST = 'xxx.xxx.xxx.xxx'
PORT = 29876
ADDR = (HOST,PORT)
BUFFSIZE = 4096
message = "Hello, World!"
cli = socket( AF_INET, SOCK_STREAM)
cli.connect(ADDR,)
data = cli.recv(BUFFSIZE)
print (data)
cli.close()
As you can tell from my code I'm using Python 3.3
Any help with this issue is greatly appreciated.

os.system() does not return the process's output, but the return value (ie. integer 0).
If you only want to read a file, do it manually:
with open(os.path.expanduser("~/.ssh/id_rsa.pub")) as f:
key = f.read()
conn.sendall(key)
If you need process output, read the documentation for the subprocess module.