We Keep Coding

Send sensor data over socket communication (TCP/IP) between Pi's with python

Hello everyone,
I have a problemen with some socket communication coding. I have two pi's communicating over TCP/IP with python scripts. The idea is that one pi (client)reads temperature/humidity sensor data and sends it over to the other pi (server), where it will be stored in a SQLite database. In the future I want multiple pi's (clients) sends over sensor data to one (web)server with database to display data on a local website.
I have written some python code for server and client side and it works pretty well. If I start the server side it will listen for other connections. When I start the client side it will generate sensor data and send it to the server. The server receives the data. But when I want to close to connection to make room for (a future other pi) it terminates completely the script and don't listens for new "calls". See code below for both sides. Code is written in Python2.7.
Goal of this project: Within a zoo setting monitor and log multiple exhibits and aquariums on temperature and humidity, display the information on LCD in every exhibit, use LED as warning, and have a central-server to store/log data, that is displayed with a central computer.
RaspiServer - server-side
import socket
from LED import callLED
host = ''
port = 5560
def setupServer():
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print("Socket created.")
try:
s.bind((host, port))
except socket.error as msg:
print(msg)
print("Socket bind complete.")
return s
def setupConnection():
s.listen(1)
conn, address = s.accept()
print("Connected to: " + address[0] + ":" + str(address[1]))
return conn
def dataTransfer(conn):
while True:
data = conn.recv(1024)
data = data.decode('utf-8')
dataMessage = data.split(":", 2)
command = dataMessage[0]
humidity = dataMessage[1]
temperature = dataMessage[2]
if command == 'DATA':
print("Received: " + humidity + " : " + temperature)
callLED()
elif command == 'EXIT':
print("Disconnected with Client")
break
else:
reply = 'Unknow Command'
conn.sendall(str.encode(reply))
Print("Reply has been send.")
conn.close()
s = setupServer()
while True:
try:
conn = setupConnection()
dataTransfer(conn)
except:
break
On the client-side there are three python scripts, where the main python scripts communicate with other scripts. I have decides to split it in multiple script to use them also as stand-alone (blinking LED, display data on LCD etc.).
RaspiMonitor - client-side
Run this script on client-side
from time import sleep
from client_sensordata import GetTemp
from monitor_client import transmit
sleepTime = 20
def tempMonitorServer():
humidity, temperature = GetTemp()
temp = str(temperature)
hum = str(humidity)
message = "DATA:" + hum + ":" + temp
print("Transmitting Data.")
response = transmit(message)
print(response)
while True:
tempMonitorServer()
sleep(sleepTime)
Use this script to send and receive data over TCP/IP
import socket
host = '192.168.2.3'
port = 5560
def setupSocket():
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host, port))
return s
def sendReceive(s, message):
s.send(str.encode(message))
print("Message transmitted")
reply = s.recv(1024)
print("We have received a reply.")
print("Send closing message.")
s.send(str.encode("EXIT"))
s.close()
reply = reply.decode('utf-8')
return reply
def transmit(message):
s = setupSocket()
response = sendReceive(s, message)
return response
Use this script to retrieve sensor data
def GetReading():
import sys
import Adafruit_DHT
humidity, temperature = Adafruit_DHT.read_retry(Adafruit_DHT.DHT22, 17)
if humidity is not None and temperature is not None:
print('Reading sensor data')
return humidity, temperature
else:
print('no sensor data')
def GetTemp():
humidity, temperature = GetReading()
name = "DHT22Client1"
print('Read sensor: {0} humidity: {1:0.2f}% temperature: {2:0.2f}C'.format(name, humidity, temperature))
return humidity, temperature
Terminal output
Terminal output, left server and right the client
Could anyone give some tips or help, or tell me why how to fix this problemen? I searched already multiple threads and other posts, but can't find the solution.
Thanks in advance for every help you give.

You have may few error in your server socket setup. The way a server socket works is it should be always open listening for new connection. When you accept a connection from the client(s.accept), it create a connection to the client. If you close that connection , it should not impact the server listening to incomming socket, The number of client or conn to client is limited only by the number you specify in the server socket.listen(NUMBER). When that number is reached, incoming connection will be rejected.
def setupServer():
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print("Socket created.")
try:
s.bind((host, port))
s.listen(5)
except socket.error as msg:
print(msg)
print("Socket bind complete.")
return s
Then remove s.listen from setup connection()
I also suggests that you handle data transfer() in a new thread to be able to process incoming connection concurrently insted of consecutive.
Here is a reference: https://docs.python.org/2/howto/sockets.html

Rather than closing and opening new sockets repeatedly, I would suggest maintaining multiple open socket connections server side and handling each socket using the select() method.
https://docs.python.org/2/library/select.html
https://pymotw.com/2/select/
Can a server handle multiple sockets in a single thread?

The code that works for now (still under construction for adding more functionalities) is:
Server-side
import socket
import sys
from LED import callLED
from monitor_log3 import ExtractStoreData
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Bind the socket to the port
server_address = ('', 5560)
print >>sys.stderr, 'Starting up on %s port %s' % server_address
sock.bind(server_address)
# Listen for incoming connections
sock.listen(1)
while True:
# Wait for a connection
print >>sys.stderr, 'Waiting for a connection'
connection, client_address = sock.accept()
try:
print >>sys.stderr, 'Connection from', client_address
# Receive the data in small chunks and retransmit it
while True:
data = connection.recv(1024)
data = data.decode('utf-8')
message = data
if data:
print >>sys.stderr, 'Send data receive confirmation'
connection.sendall(data)
callLED()
ExtractStoreData(message)
else:
print >>sys.stderr, 'No more data from', client_address
break
finally:
# Clean up the connection
print >>sys.stderr, 'Closing connection'
print >>sys.stderr, '------------------'
connection.close()
And the client-side looks like this:
Client-side
import socket
import sys
import datetime
from time import sleep
from client_sensordata import GetTemp
timeSleep = 10
def ClientSocket():
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Connect the socket to the port where the server is listening
server_address = ('192.168.2.3', 5560)
print >>sys.stderr, 'Connecting to: %s port: %s' % server_address
sock.connect(server_address)
try:
# Send data
name, humidity, temperature = GetTemp()
Sensorname = str(name)
temp = str(temperature)
hum = str(humidity)
print(Sensorname + ":" + temp + ":" + hum)
message = (Sensorname + ":" + temp + ":" + hum)
print("Transmitting data...")
print >>sys.stderr, 'Sending data...'
sock.send(str.encode(message))
# Look for the response
amount_received = 0
amount_expected = len(message)
while amount_received < amount_expected:
data = sock.recv(1024)
amount_received += len(data)
print >>sys.stderr, 'Send data'
finally:
print >>sys.stderr, 'Closing connection'
print >>sys.stderr, '------------------'
sock.close()
while True:
print("Start external server measurement at " + datetime.datetime.now().strftime("%H:%M:%S"))
ClientSocket()
sleep(timeSleep)
for t2 in range(5):
print("Start internal display measurement at " + datetime.datetime.now().strftime("%H:%M:%S"))
GetTemp()
print("------------------")
sleep(timeSleep)
It is looping 5 cyclus for internal measurement (for on a LCD display) and then send the data once to the server, that will store it in the database (SQlite3). Still want to add more functionalities (LED when reading, warning LED when under or above limits, alarm sound, LCD connected. When there is progress I'll update the code.
Offcourse any tips and sugestions are more then welcome!

Problems with a chat program written in Python and PyGTK

I am new to python and I am currently working on a chat room program in Python (still in progress...). I have also made a GUI for my program. Initially, I made two py files, one for the GUI and one for the chatting function. They both worked perfectly when separated. After, I combined the two files. I faced the following two problems:
One of my threads (target = loadMsg) is used to wait for the host's msg and print it out on the screen. The problem is that it delays for one msg every time. For example, I sent a "1" to the host and the host should return a "1" immediately. But, the "1" I received didn't appear on my screen. Then I send a "2" to the host and the host should reply a "2" immediately. Then, my screen shows a "1" but the "2" is still missing until the host reply a "3" to me, after I send a "3" to the host. Where is the problem?
This is a technical problem. I was testing the stability of the chat room and I found that about 10% of my msg disappeared during the transmission and this situation occurs randomly. How can I fix such a problem?
Sorry for my poor English. I hope someone can help me with it.T_T
Here is my code for your reference:
---Client
import pygtk,gtk
import logging
from threading import *
import socket
DEBUG = 1
HOST = ''
PORT = 8018
TIMEOUT = 5
BUF_SIZE = 1024
class Base():
def reload(self):
try:
buf = self.sock.recv(BUF_SIZE)
print buf
self.addMsg(buf)
except:
pass
def reload_butt(self,widget):
try:
self.thread = Thread(target=self.reload)
self.thread.start()
except:
pass
def loadMsg(self):
try:
while True :
buf = self.sock.recv(BUF_SIZE)
print buf
self.addMsg(buf)
except:
self.sock.close()
def sendMsg(self,widget):
if DEBUG : print "Send Msg"
if self.entry.get_text() : self.sock.send(self.entry.get_text())
self.entry.set_text("")
def addMsg(self,string):
if DEBUG : print "Try to add Msg"
if self.entry.get_text() :
iter = self.buffer1.get_iter_at_offset(-1)
self.buffer1.insert(iter,("\n Username: "+string))
self.entry.set_text("")
self.adj = self.scrolled_window.get_vadjustment()
self.adj.set_value( self.adj.upper - self.adj.page_size )
if DEBUG : print "Add msg ok"
def destroy(self,widget):
if DEBUG : print "Destroy function called"
self.sock.close()
gtk.main_quit()
def __init__(self,sock):
if DEBUG : print "Initializing..."
self.sock = sock
self.win=gtk.Window()
self.win.connect("destroy",self.destroy)
self.vbox=gtk.VBox()
self.win.add(self.vbox)
self.view=gtk.TextView()
self.view.set_editable(False)
self.buffer1=self.view.get_buffer()
self.scrolled_window=gtk.ScrolledWindow()
self.scrolled_window.set_policy(gtk.POLICY_AUTOMATIC,gtk.POLICY_AUTOMATIC)
self.scrolled_window.add(self.view)
self.vbox.add(self.scrolled_window)
self.entry=gtk.Entry()
self.entry.connect("activate",self.sendMsg)
self.enter=gtk.Button("Enter")
self.enter.connect("clicked",self.sendMsg)
self.reload=gtk.Button("Reload")
self.reload.connect("clicked",self.reload_butt)
self.hbox=gtk.HBox()
self.hbox.add(self.entry)
self.hbox.pack_start(self.reload,False,False)
self.hbox.pack_start(self.enter,False,False)
self.vbox.pack_start(self.hbox,False,False)
self.win.show_all()
if DEBUG : print "Finish initializing"
def main(self):
try :
gtk.main()
except :
print "Error!!!"
def main() :
try :
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((HOST, PORT))
print ('Connecting to '+ str(HOST) +' ' + str(PORT))
base=Base(sock)
thread1=Thread(target=base.loadMsg)
thread2=Thread(target=base.main)
thread2.start()
thread1.start()
except :
print "Err0r!!!"
sock.close()
main()
---host (an echo host)
import socket
HOST = ''
PORT = 8018
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
s.listen(5)
conn, addr = s.accept()
print 'Connected by', addr
try :
print "Start!"
while True:
data = conn.recv(1024)
print data
reply = data # echo
if not reply : break
if reply== "!q" :
conn.close()
break
conn.send(reply)
conn.close()
except :
print "Error!!!!!"
conn.close()

I would seriously recommend to use the gio library (part of glib). Using that library, you connect functions to the socket operations such as when data is available, or when data can be written to the socket. The library will call these function when necessary, and you don't need a wait loop. Which is more CPU-friendly.
http://jcoppens.com/soft/howto/gtk/chat_socket.php contains an example of communications between a C program and Python, using gio, which might be useful to you.
This way, you can start monitoring the sockets after the GUI has started, and you do not need threads to attend the communications.

Python Simple Sever and Client script (Socket programming)

My objective is every time the client send the data to the server, the server will display it. But Currently when the client send the data, the server capture the data and display it only for the first time. What is causing the problem? Btw, this is my first time doing network programming, so pls keep your answer simple.
Server Script
import socket
server_socket = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
server_socket.bine(("",5001))
server_socket.listen(5)
print "TCP Server Waiting for incoming client connection on port 5001..."
while True:
client_socket, address =server_socket.accept()
print "Connection from ", address
data = client_socket.recv(512)
print "RECIEVED:" , data
Client Script
import socket
client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client_socket.connect(('169.254.0.1', 5001))
data=''
while data!='quit':
data = raw_input ( "SEND :" )
client_socket.send(data)

Your server code receive only once, then accept another client.
You should loop until client disconnect. (when disconnected, recv() return empty string)
while True:
client_socket, address = server_socket.accept()
print "Connection from ", address
while 1:
data = client_socket.recv(512)
if not data:
break
print "RECIEVED:" , data
BTW, your server code (and my code) does handle only one client at a time.

while True:
client_socket, address =server_socket.accept()
print "Connection from ", address
data = client_socket.recv(512)
print "RECIEVED:" , data
This should be:
client_socket, address =server_socket.accept()
print "Connection from ", address
while True:
data = client_socket.recv(512)
print "RECIEVED:" , data
server_socket.accept() will wait indefinitely until a new client is connected. Actually your loop is like : "accept a client and receive one time the data he sent".

Send/receive Packets with TCP sockets

Recently, I managed to create sockets on my PC and my Raspberry Pi to enable communication between both devices. Currently, the client is able to automatically send messages to the server. I was wondering, if it is possible to modify the scripts to send tcp data packets instead of purely text messages, as I would very much like to control the raspberry pi using my PC in the future without having the need to ssh/etc.
I've looked at some examples, but as I don't have much experience in writing my own scripts/codes, I'm not very sure how to go about doing this. I would appreciate if someone could guide me in the right direction with explanation and some examples if possible.
Anyway here is the server/client script I'm running at the moment:
Client:
import socket
import sys
import struct
import time
#main function
if __name__ == "__main__":
if(len(sys.argv) < 2) :
print 'Usage : python client.py hostname'
sys.exit()
host = sys.argv[1]
port = 8888
#create an INET, STREAMing socket
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error:
print 'Failed to create socket'
sys.exit()
print 'Socket Created'
try:
remote_ip = socket.gethostbyname( host )
s.connect((host, port))
except socket.gaierror:
print 'Hostname could not be resolved. Exiting'
sys.exit()
print 'Socket Connected to ' + host + ' on ip ' + remote_ip
#Send some data to remote server
message = "Test"
try :
#Set the whole string
while True:
s.send(message)
print 'Message sent successfully'
time.sleep(1)
print 'Sending...'
except socket.error:
#Send failed
print 'Send failed'
sys.exit()
def recv_timeout(the_socket,timeout=2):
#make socket non blocking
the_socket.setblocking(0)
#total data partwise in an array
total_data=[];
data='';
#beginning time
begin=time.time()
while 1:
#if you got some data, then break after timeout
if total_data and time.time()-begin > timeout:
break
#if you got no data at all, wait a little longer, twice the timeout
elif time.time()-begin > timeout*2:
break
#recv something
try:
data = the_socket.recv(8192)
if data:
total_data.append(data)
#change the beginning time for measurement
begin=time.time()
else:
#sleep for sometime to indicate a gap
time.sleep(0.1)
except:
pass
#join all parts to make final string
return ''.join(total_data)
#get reply and print
print recv_timeout(s)
s.close()
Server:
import socket
import sys
from thread import *
HOST = '' # Symbolic name meaning all available interfaces
PORT = 8888
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print 'Socket created'
try:
s.bind((HOST, PORT))
except socket.error , msg:
print 'Bind failed. Error Code : ' + str(msg[0]) + ' Message ' + msg[1]
sys.exit()
print 'Socket bind complete'
s.listen(10)
print 'Socket now listening'
#Function for handling connections
def clientthread(conn):
#Sending message to connected client
conn.send('Welcome to the server. Receving Data...\n') #send only takes string
#infinite loop so that function do not terminate and thread do not end.
while True:
#Receiving from client
data = conn.recv(1024)
reply = 'Message Received at the server!\n'
print data
if not data:
break
conn.sendall(reply)
conn.close()
#now keep talking with the client
while 1:
#wait to accept a connection
conn, addr = s.accept()
print 'Connected with ' + addr[0] + ':' + str(addr[1])
#start new thread
start_new_thread(clientthread ,(conn,))
s.close()

socket.socket(socket.AF_INET, socket.SOCK_STREAM) already creates a connection that provides a reliable stream of bytes between two machines. This uses TCP, which is on top of IP and Ethernet. The latter two are package-based, while TCP creates a stream of continuous bytes on top of it. It also adds some error checking and error correction, so it is pretty reliable.
I honestly don't understand what you want to achieve with what you call "send packets". What you don't want to do is to create an implementation of TCP yourself, as that's a non-trivial task, so sending RAW packets is out. In general, even using TCP is already relatively low-level and should be avoided unless really necessary.
Using e.g. ZeroMQ you get a message-based interface that does all the transmission for you. It does so on top of TCP (or other transports) and adds more error correction for e.g. disconnects. There, you also have something like "packets", but those are independent of how many TCP or IP packets were required to send it underneath. If you don't want to implement a specific protocol, I'd suggest you use this framework instead of lowlevel TCP sockets.
Another simple alternative is to use HTTP, for which there is also existing code in Python. The downside is that it is always one side that initiates some communication and the other side only replies. If you want some kind of active notification, you either have to poll or use hacks like delaying an answer.

You are already sending data packets - those packets juts happen to contain text data at the moment. Try looking into pickle in the standard libraries and into pyro.

IOS smallSocket and python

I'm working on an IOS app.
I'm starting with a python server on mac that should connect to an iphone and print data sent from iphone.
the connection seems to be established but python print infinite " b " " as data... I don't know why.
the strange thing is that it happens also with cocoaAsynchronousSocket
this is the server
#!/usr/bin/python
import time
import socket
import sys
addr = sys.argv[1]
port = 4444
if not addr :
print ("No host address specified, plese specify an address", files=sys.stderr)
sock = socket.socket (socket.AF_INET, socket.SOCK_STREAM)
print ("connecting...")
try:
sock.connect ((addr, port))
except socket.error:
print ("unable to connect to", addr)
sys.exit(0)
print ("Connected to", addr)
while 1:
data = sock.recv(0)
data2 = sock.recv(1)
# if not data: break
print (data)
print (data2)
and this is some code that i use to create the connection
- (IBAction)openPressed:(id)sender {
socket = [Socket socket];
[socket listenOnPort:4444];
[socket acceptConnection];
[socket writeString:#"connection accepted"];
}

Why did u add this line:
data = sock.recv(0)
Besides that, your sever, while client might be a better name, seems good.
If it doesn't print what you expect, I suggest that you use some sniffer tools, like wireshark, to check what it really receives.