We Keep Coding

Python Server Client Remote command execution : error while executing more than 2 commands

Server Code:
#!/usr/bin/python
import socket
import threading
import sys
import os
import time
bind_ip = raw_input("\nset lhost : ")
print "lhost set %s" % bind_ip
bind_port =int(raw_input("\nset lport : "))
print "lport set %s" % bind_port
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind((bind_ip, bind_port))
server.listen(5)
time.sleep(0.8)
print "\n[*]Listening on %s:%d" % (bind_ip, bind_port)
client, addr = server.accept()
time.sleep(0.8)
print "[*] Binding connection on %s:%d" % (bind_ip, bind_port)
time.sleep(0.8)
print "[*] Accepted connection from %s:%d" % (bind_ip, bind_port)
time.sleep(0.5)
print "[*] Spwaning command shell"
time.sleep(0.5)
print "[*] Spwaned!!"
while True:
try:
print "\ncommand$control:~$"
# take commands
# command = raw_input("\ncommand$control:~ ")
command = sys.stdin.readline()
# if command == exit then exit
if command == "exit\n":
print "[!] Exiting..!"
client.send(command)
client.close()
os._exit(1)
else: # send 1st command
client.send(command)
recvd = None
# if recvd == # break loop and ask next command
while recvd != "#":
recvd = None
recvd = client.recv(4096)
if recvd == "#":
break
elif len(recvd):
recvd = recvd.replace('\r', '').replace('\n', '')
#recvd = recvd.rstrip('\t')
print recvd
except Exception, e:
print "Error: %s" % str(e)
os._exit(1)
Client Code:
#!/usr/bin/python
import socket
import threading
import subprocess
import sys
import os
target_host = raw_input("set target host: ")
target_port = int(raw_input("set target port: "))
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect((target_host, target_port))
def run_command(command):
output = ''
command = command.rstrip()
output = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, s**strong text**tderr=subprocess.STDOUT)
for line in iter(output.stdout.readline, ''):
line = line.replace('\n', '').replace('\r', '').replace('\t', '')
print line
client.send(line)
sys.stdout.flush()
while True:
try:
cmd_buffer = ""
while "\n" not in cmd_buffer:
cmd_buffer+=client.recv(1024)
if cmd_buffer == "exit\n":
client.close()
os._exit()
run_command(cmd_buffer)
# After run_command for loop ends, # is send to break
# from the server's while loop
client.send("#")
except Exception:
client.close()
os._exit(0)
The code works,the client sends '#' to server to indicate that is has finished sending realtime command output, so the server on receiving '#' breaks from the loop,and ask for next command. But after entering 2/3 commands the # is printed on servers stdout which should'nt and it doesn't break from loop. Also the output from client isn't received as i have formatted it using replace(). Please help.it will be appreciated.

How to send stream trough socket when using select()?

After long hours of research and testing I finally ask here.
My script has to handle multiple client connections and in the same time has to get and send a stream from another socket.
Finally I've been able to make it work but only for one user. That user connects to the socket, the script connects to the other socket, then return the stream to the client.
The script works pretty well but has a some hard limitations :
- it send the stream to the client but,
- even if the socket is in non-blocking mode I think that calling a socket inside another one is the main reason why it reacts like it was in blocking mode (because one ot these is continuously sending datas ?)
By the way I think that the select() method could allow me to do what I want, but I don't clearly understand how.
Here is the server code taht works for one client, but is blocking
#!/usr/bin/env python
# coding: utf-8
from __future__ import print_function
import sys, time, base64, socket
server_ip = 'XX.XX.XX.XX'
def caster_connect(connected_client, address):
username = 'XXXXXXX'
password = 'XXXXXXXXX'
host = 'XX.XX.XX.XX'
port = 2102
pwd = base64.b64encode("{}:{}".format(username, password).encode('ascii'))
pwd = pwd.decode('ascii')
u_message = ''
stream_type = 'CMRp'
header = \
"GET /" + str(stream_type) + " HTTP/1.1\r\n" +\
"Host " + str(host) + "\r\n" +\
"Ntrip-Version: Ntrip/1.0\r\n" +\
"User-Agent: my_script.py/0.1\r\n" +\
"Accept: */*\r\n" +\
"Authorization: Basic {}\r\n\r\n".format(pwd) +\
"Connection: close\r\n"
print("Connecting to caster...\n")
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host,int(port)))
s.send(header.encode('ascii'))
print("Waiting answer from caster...\n")
while True:
try:
data = s.recv(2048)
connected_client.send(data)
print("Sending data from caster at %s" % time.time())
sys.stdout.flush()
# On any error, close sockets
except socket.error, e:
print("No data received from caster : %s" % e)
print("Close client connection at %s" % format(address))
s.close()
break
return
#----------------
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind((server_ip, 5680))
sock.settimeout(3)
try:
while True:
try:
sock.listen(5)
client, address = sock.accept()
print ("%s connected" % format(address) )
msg = client.recv(4096)
except socket.timeout, e:
err = e.args[0]
if err == 'timed out':
print("Timed out, retry later")
continue
else:
print(socket.error)
sock.close()
except socket.error:
print(socket.error)
sock.close()
else:
if len(msg) == 0:
print("Shutdown on client end")
sock.close()
else:
print(msg)
caster_response = caster_connect(client, address)
sys.stdout.flush()
print("Close")
client.close()
sock.close()`enter code here`
except KeyboardInterrupt:
print("W: Keyboard interrupt, closing socket")
finally:
sock.close()
And this is the code I found to handle select()
#!/usr/bin/env python
# coding: utf-8
import select, socket, sys, Queue
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setblocking(0)
server.bind(('XX.XX.XX.XX', 64000))
server.listen(5)
inputs = [server]
outputs = []
message_queues = {}
while inputs:
readable, writable, exceptional = select.select(
inputs, outputs, inputs)
for s in readable:
if s is server:
connection, client_address = s.accept()
print("New connection from %s" % client_address)
connection.setblocking(0)
inputs.append(connection)
message_queues[connection] = Queue.Queue()
else:
data = s.recv(1024)
print("Data received : %s" % data)
if data:
message_queues[s].put(data)
if s not in outputs:
outputs.append(s)
else:
if s in outputs:
outputs.remove(s)
inputs.remove(s)
s.close()
del message_queues[s]
for s in writable:
try:
next_msg = message_queues[s].get_nowait()
print("Next msg : %s" % next_msg)
except Queue.Empty:
outputs.remove(s)
else:
s.send(next_msg)
for s in exceptional:
inputs.remove(s)
if s in outputs:
outputs.remove(s)
s.close()
del message_queues[s]
In this code (found at this page) I didn't make changes as I don't know how to handle this.
Maybe by creating another server script that would only handle the stream part, so the main script would act as a server for clients, but as client for the stream part ?

Socket.close() doesn't close the Socket

I'm trying to develop a very simple client/server program, the server part is working properly but I've a problem with the client part but I can't understand why.
The client's work is very simple, just retrive the counter value from a external device, then I'm trying to send the retrieved data to the server part.
At the beginning the socket is working well, but some time when I should send the data I've got the server exception and after that the Client is not working.
I can't understand if the s.close() function is working properly.
UPDATE: the exception that I got is "errno 110 connection timed out"
Client:
import time, socket, struct, array, json
import Client_Axis
import sys
import serial
import os
import datetime
import re
import packet
import Config_mio
usbport = '/dev/ttyAMA0'
h = "/r/n"
if __name__=="__main__":
"""Main function that starts the server"""
curr_value = "0000000000"
prev_value = ""
line = '111111111111111'
error_counter = 0
people_in = 0
people_out = 0
txtDate = ""
no_updates_counter = 0
while True:
ser = None
try:
# print('1')
ser = serial.Serial('/dev/ttyAMA0', 9600, timeout=1)
# ser.open()
# print('2')
for line in ser.read():
line = ser.readline()
print(line[6:10])
curr_value = line
except:
print('Serial error')
# print('3')
pass
finally:
if ser:
ser.close()
try:
error_counter += 1
# print('1')
response = Client_Axis.Read_Axis_Camera_Occupancy()
content = response.split(",")
people_in_refresh = int(re.search(r'\d+', content[3]).group())
people_out_refresh = int(re.search(r'\d+', content[4]).group())
# print('2')
error_flag = 0
if people_in != people_in_refresh or people_out != people_out_refresh:
people_in = people_in_refresh
people_out = people_out_refresh
try:
# Creates TCP socket in the specified IP address and port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Connect client to the server
s.connect((Config_mio.IP_Server_Add, Config_mio.ws_port))
# Create message and send it to server
timestamp = str(time.time())
# msg = packet("c", timestamp, Config_mio.RbPi_Id, content[3], content[4], None)
msg = "c"+","+str(timestamp)+","+str(Config_mio.RbPi_Id)+","+str(people_in)+","+str(people_out)+","+"None"
# json_message = json.dumps(msg)
# s.send(json_message)
s.send(msg)
print "messaggio INVIATO"
# Close connection when data is sent
#s.close()
except:
print('Server connection error 1')
pass
finally:
s.close()
#AXIS_occup_old = AXIS_occup
#AXIS_occup = response.read()
#my_dict = json.loads(AXIS_occup)
# print(AXIS_occup)
# print(my_dict)
#del my_dict["timestamp"]
#AXIS_occup = my_dict
#error_counter = 0
# print('3')
except:
error_msg = "Error retrieving occupancy from: " + Config_mio.IP_AXIS_Add
error_flag = 1
if (error_flag == 1):
no_updates_counter = 0
print "Error detected: %s \r\n" % error_msg
print error_counter
if (error_counter > 200):
os.system("sudo reboot")
elif (line[6:10] != '1111' and prev_value != curr_value):
try:
# Creates TCP socket in the specified IP address and port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Connect client to the server
s.connect((Config_mio.IP_Server_Add, Config_mio.ws_port))
# Create message and send it to server
timestamp = str(time.time())
msg = "r"+","+str(timestamp)+","+str(Config_mio.RbPi_Id)+","+"None"+","+"None"+","+str(line[6:10])
#msg = {"Id": raspberry_id,
# "Ranging": line[6:10],
# "timestamp": timestamp}
#json_message = json.dumps(msg)
#s.send(json_message)
s.send(msg)
print "Message : %s" % msg
# Close connection when data is sent
s.close()
except:
print('Server connection error 2')
pass
else:
no_updates_counter += 1
# Send message despite there are no changes in value
# This is a heartbeat message of 10 min
if (no_updates_counter > 200):
no_updates_counter = 0
AXIS_occup = ""
prev_value = curr_value
# Reboot device every day - routine
# We have 4 cases incase we miss few seconds
txtDate = str(datetime.datetime.fromtimestamp(time.time()))
if (txtDate[11:19] == "00:00:00"):
os.system("sudo reboot")
if (txtDate[11:19] == "00:00:01"):
os.system("sudo reboot")
if (txtDate[11:19] == "00:00:02"):
os.system("sudo reboot")
if (txtDate[11:19] == "00:00:03"):
os.system("sudo reboot")
# Kill time - 1 sec: Remove it for high speed localisation
time.sleep(1)
Server:
import socket
import json
import time
import Config_mio
import packet
import sqlite3 as lite
if __name__ == "__main__":
"""Main function that starts the server"""
# Creates TCP socket in the specified IP address and port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((Config_mio.IP_Server_Add, Config_mio.ws_port))
# Starts server, up to 10 clients are queued
s.listen(Config_mio.Max_Num_Clients)
while True:
conn, addr = s.accept()
#print "sono dopo ascolto"
msg = conn.recv(1024)
print "Data value:",msg
msg = msg.split(",")
if msg[0] == "c":
print "counter msg"
elif msg[0] == "r":
print "range msg",msg[1],msg[2],msg[5]
conn.close()

How to make the server side socket code run continuously

I have a server/client socket program that is used to transfer a file from the client to the server. The issue is that the code stops running once the file is transferred. I want to change it such that the server side code is continuously running so that I can transfer a file multiple times without having to run the code again and again
Server code:
import socket
host = ''
port = 5560
def setupServer():
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print("Socket created.")
try:
s.bind((host, port))
except socket.error as msg:
print(msg)
print("Socket bind comlete.")
return s
def setupConnection():
s.listen(1) # Allows one connection at a time.
conn, address = s.accept()
print("Connected to: " + address[0] + ":" + str(address[1]))
return conn
def storeFile(filePath):
picFile = open(filePath, 'wb')
print(filePath)
print("Opened the file.")
pic = conn.recv(1024)
#print(pic)
while pic:
print("Receiving picture still.")
picFile.write(pic)
pic = conn.recv(1024)
picFile.close()
def dataTransfer(conn):
# A big loop that sends/receives data until told not to.
while True:
# Receive the data
data = conn.recv(1024) # receive the data
data = data.decode('utf-8')
# Split the data such that you separate the command
# from the rest of the data.
dataMessage = data.split(' ', 1)
command = dataMessage[0]
if command == 'GET':
reply = GET()
elif command == 'REPEAT':
reply = REPEAT(dataMessage)
elif command == 'STORE':
print("Store command received. Time to save a picture")
storeFile(dataMessage[1])
reply = "File stored."
elif command == 'LED_ON':
callLED()
reply = 'LED was on'
else:
reply = 'Unknown Command'
# Send the reply back to the client
conn.sendall(str.encode(reply))
#print("Data has been sent!")
conn.close()
s = setupServer()
while True:
try:
conn = setupConnection()
dataTransfer(conn)
except:
break
The client side code is below:
import socket
from time import sleep
from time import time
host = '192.168.0.17'
port = 5560
data = "hi"
filepath = "/var/www/html/unknown.txt"
def setupSocket():
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((host, port))
return s
def sendPic(s, filePath):
print(filePath)
pic = open(filePath, 'rb')
chunk = pic.read(1024)
s.send(str.encode("STORE " + filePath))
t = time()
while chunk:
print("Sending Picture")
s.send(chunk)
#print(chunk)
chunk = pic.read(1024)
pic.close()
print("Done sending")
print("Elapsed time = " + str(time() - t) + 's')
#s.close()
return "Done sending"
def sendReceive(s, message):
s.send(str.encode(message))
reply = s.recv(1024)
print("We have received a reply")
print("Send closing message.")
s.send(str.encode("EXIT"))
#s.close()
reply = reply.decode('utf-8')
return reply
def transmit(message):
s = setupSocket()
response = sendReceive(s, message)
return response
def backup(filePath):
s = setupSocket()
response = sendPic(s, filePath)
return response
while True:
backup(filepath)
print("Backup Complete!")
break
I do not own the code. I have made some change to the code that I got from a YouTube video.

Have you had a look at the SocketServer module?
You could setup your dataTransfer() function as the handle() method of a RequestHandler class, then start your server with the serve_forever() method.

python socket tchat issue

I started to code in python with sockets and I have a little problem for my chat script.
Server script
import pickle, socket, struct, sys, threading
SERVERADDRESS = ("localhost", 6030)
class helloChatServer(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.__server = socket.socket()
self.users = []
try:
self.__server.bind(SERVERADDRESS)
except socket.error:
print('Bind failed {}'.format(socket.error))
self.__server.listen(10)
def exit(self):
self.__server.close()
def run(self):
print( "Listening... {}".format(SERVERADDRESS))
while True:
client, addr = self.__server.accept()
try:
threading.Thread(target=self._handle, args=(client, addr)).start()
except OSError:
print('Error during processing the message')
def _handle(self, client, addr):
print('Client connected with {}:{}'.format(addr[0], str(addr[1])))
self.users.append(addr)
while True:
data = client.recv(1024)
print(data)
client.send(data)
client.close()
if __name__ == '__main__':
helloChatServer().run()
Client script
import pickle, socket, struct, sys, threading
SERVERADDRESS = (socket.gethostname(), 6050)
class helloChatClient():
def __init__(self, host='localhost', port=5000, pseudo="Visitor"):
self.__socket = socket.socket()
self.__socket.bind((host, port))
self.__pseudo = pseudo
print('Listening on {}:{}'.format(host, port))
def run(self):
handlers = {
'/exit': self._exit,
'/quit': self._quit,
'/join': self._join,
'/send': self._send
}
self.__running = True
self.__address = None
threading.Thread(target=self._receive).start()
while self.__running:
line = sys.stdin.readline().rstrip() + ' '
# Extract the command and the param
command = line[:line.index(' ')]
param = line[line.index(' ')+1:].rstrip()
# Call the command handler
if command in handlers:
try:
handlers[command]() if param == '' else handlers[command](param)
except:
print("Error during the execution of the message")
else:
print('Command inconnue:', command)
def _exit(self):
self.__running = False
self.__address = None
self.__socket.close()
def _quit(self):
self.__address = None
def _join(self, param):
if self.__pseudo == "Visitor":
self.__pseudo = input("Choose a username: ")
tokens = param.split(' ')
if len(tokens) == 2:
try:
self.__address = (tokens[0], int(tokens[1]))
self.__socket.connect(self.__address)
print('~~~~~~~~~~~~~~~~~~~~~~~~~~')
print('Connected at {}:{}'.format(*self.__address))
print('~~~~~~~~~~~~~~~~~~~~~~~~~~')
except OSError:
print("Error during the sending of the message")
self.__socket.send(self.__pseudo.encode())
def _send(self, param):
if self.__address is not None:
try:
message = param.encode()
totalsent = 0
while totalsent < len(message):
sent = self.__socket.send(message[totalsent:])
totalsent += sent
print(self.__socket.recv(1024).decode())
except OSError:
print('Error during the reception of the message')
def _receive(self):
while self.__running:
try:
data = self.__socket.recv(1024).decode()
print(data)
except socket.timeout:
pass
except OSError:
return
if __name__ == '__main__':
if len(sys.argv) == 4:
helloChatClient(sys.argv[1], int(sys.argv[2]), sys.argv[3]).run()
else:
helloChatClient().run()
Well when I run the script on the terminal, I see this.
Server
MacBook-Pro-de-Saikou:labo2 saikouah$ python3.4 helloChatServer.py
En écoute sur... ('MacBook-Pro-de-Saikou.local', 6030)
Client connected with 127.0.0.1:5004
Il y a actuellement 1 connecté
b'bluebeel'
b'hello'
Client
MacBook-Pro-de-Saikou:labo2 saikouah$ python3.4 helloChatClient.py localhost 5004 bluebeel
Écoute sur localhost:5004
/join MacBook-Pro-de-Saikou.local 6030
~~~~~~~~~~~~~~~~~~~~~~~~~~
Connecté à MacBook-Pro-de-Saikou.local:6030
~~~~~~~~~~~~~~~~~~~~~~~~~~
/send hello
bluebeel
On the client terminal he doesn't print me hello but bluebeel. I made several test and he took me every time the previous one message. Looks like he is late.
Someone can help me? :)

PROBLEM ANALYSIS
Your code fails in _receive function:
data = self.__socket.recv(1024).decode()
This line throws OSError because you try to call .recv before connecting to the server. Thus the exception handler is fired and the function exits. So what happens is that after calling
threading.Thread(target=self._receive).start()
function _receive exits before you call /join. So watch what happens
You call /join.
bluebeel is send to the server
Server receives it and sends it back to the client
But _receive function is no longer there. So the message is "stacked" on the socket (it will wait for next .recv() call)
You call /send hello
Server receives hello and sends it back
Client calls print(self.__socket.recv(1024).decode()) in _send method
But .recv retrieves the first message that is stacked on the socket. In that case it is not hello, it is bluebeel.
Now this schema continues to work. You send message, server pings it back but there's always 1 message in front of the received one. The "late" message.
SOLUTION
One way of solving this issue is to call
threading.Thread(target=self._receive).start()
in ._join method after .connect. Remember to remove print(self.__socket.recv(1024).decode()) from _send method, otherwise it will block stdin.
Of course you will have problems when issuing multiple /join commands. To properly address that you would have to keep track of _receive thread and kill it at the begining of ._join method. This however is beyond the scope of this question IMHO.
SIDE NOTE
Don't ever handle exceptions like you did. This is wrong:
try:
data = self.__socket.recv(1024).decode()
print(data)
except socket.timeout:
pass
except OSError:
return
At least do this:
import traceback
try:
data = self.__socket.recv(1024).decode()
print(data)
except socket.timeout:
traceback.print_exc()
except OSError:
traceback.print_exc()
return