We Keep Coding

Get threading.Thread target class object when defined as a variable

I'm trying to understand the basics of networking and I believe the right way to start learning basic server handling is through sockets.
I am aware there are better libraries such as twisted or socketServer. I want to do it specifically in socket module.
I am aware using socket module to build servers is unreliable.
I also am aware I need to implement some kind of protocol for future use.
I tried building P2pClient with threading.Thread inheritance:
import socket
import threading
class P2pServer(threading.Thread):
def __init__(self, host, port):
threading.Thread.__init__(self)
print "Info: %s:%s" % (host, port)
self.server = socket.socket()
self.server.bind((host, port))
print "Created the server"
self.server.listen(1) # p2p chat
def run(self):
while True:
c, addr = self.server.accept()
print "Connected from %s (%s)" % (addr, c)
class P2pClient(threading.Thread):
def __init__(self, host, port):
threading.Thread.__init__(self, host, port)
self.client = socket.socket()
self.client.connect((host, port))
def send_msg(self, msg):
try:
self.client.send(msg)
return "Sent: %s" % msg
except socket.error as e:
return "Could not send because of %s" % e
def run(self):
while True:
recv = self.client.recv(1024)
if len(recv) > 0:
print recv
server = P2pServer("localhost", 44444) # This is our server
server.start() # Run it
client = None
print "Ready to take input"
while True:
print "Your command: ",
cmd = raw_input()
if cmd.startswith("connect "):
cmd = cmd.split()
client = P2pClient(cmd[1], cmd[2])
client.start()
else:
client.send_msg(cmd)
But ended up getting the following error when the input is "connect localhost :
Traceback (most recent call last):
File "C:/Users/edız/Desktop/Ediz/Python/Playground/servers test/p2p.py", line 49, in <module>
client = P2pClient(cmd[1], cmd[2])
File "C:/Users/edız/Desktop/Ediz/Python/Playground/servers test/p2p.py", line 22, in __init__
threading.Thread.__init__(self, host, port)
File "C:\Python27\lib\threading.py", line 670, in __init__
assert group is None, "group argument must be None for now"
AssertionError: group argument must be None for now
Solution on the site suggested making it so that I use the threading.Thread() function directly instead of doing it the former way. However, the threading documenation doesn't give me an idea about how to reach the target function. How can I build the thread that will allow me to access P2pClient in the client variable?
Purpose of the following code is so that it will be run in two different port in same host, making a P2P connection between two servers.
import socket
import threading
class P2pServer(threading.Thread):
def __init__(self, host, port):
threading.Thread.__init__(self)
print "Info: %s:%s" % (host, port)
self.server = socket.socket()
self.server.bind((host, port))
print "Created the server"
self.server.listen(1) # p2p chat
def run(self):
while True:
c, addr = self.server.accept()
print "Connected from %s (%s)" % (addr, c)
class P2pClient(object):
def __init__(self, host, port):
self.client = socket.socket()
self.client.connect((host, int(port)))
def send_msg(self, msg):
try:
self.client.send(msg)
return "Sent: %s" % msg
except socket.error as e:
return "Could not send because of %s" % e
def run(self):
while True:
recv = self.client.recv(1024)
if len(recv) > 0:
print recv
server = P2pServer("localhost", 44444) # This is our server
server.start() # Run it
client = None
print "Ready to take input"
while True:
print "Your command: ",
cmd = raw_input()
if cmd.startswith("connect "):
cmd = cmd.split()
client = threading.Thread(target=P2pClient, args=(cmd[1], cmd[2]))
client.start()
else:
pass
# Here I just want something like this:
# client.send_msg("Hello other server")

Your first example is close. The problem is that you passed the extra parameters you need for the child P2pClient to the parent Thread class. thread takes a different set of parameters. Just leave those parameters out when initializing the parent.
class P2pClient(threading.Thread):
def __init__(self, host, port):
threading.Thread.__init__(self)
self.client = socket.socket()
self.client.connect((host, port))

Multi threaded server send function

I got this multi threaded server code and it works but when I type something to send to the client its not sending it the send function only work if I send the data string
anyone knows what's the problem?
#!/usr/bin/env python
import socket, threading
class ClientThread(threading.Thread):
def __init__(self, ip, port, clientsocket):
threading.Thread.__init__(self)
self.ip = ip
self.port = port
self.csocket = clientsocket
print "[+] New thread started for "+ip+":"+str(port)
def run(self):
print "Connection from : "+ip+":"+str(port)
clientsock.send("Welcome to the server ")
data = "dummydata"
while len(data):
data = self.csocket.recv(2048)
print "Client(%s:%s) sent : %s"%(self.ip, str(self.port), data)
userInput = raw_input(">")
self.csocket.send(userInput)
print "Client at "+self.ip+" disconnected..."
host = "0.0.0.0"
port = 4444
tcpsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
tcpsock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
tcpsock.bind((host, port))
while True:
tcpsock.listen(4)
print "nListening for incoming connections..."
(clientsock, (ip, port)) = tcpsock.accept()
#pass clientsock to the ClientThread thread object being created
newthread = ClientThread(ip, port, clientsock)
newthread.start()

Well, I can see at least one thing that will prevent this from working as intended:
def run(self):
print "Connection from : "+ip+":"+str(port)
clientsock.send("Welcome to the server ")
clientsock is undefined.

My suggestion is don't try to reinvent the wheel (unless you want to understand how the wheel works). There's already the built-in SocketServer but that is synchronous, meaning each request must be completed before the next request can be started.
There are already very easy to use implementations of asynchronous (non-blocking) TCP servers out there. If you want something that doesn't require you to learn a framework and just runs out-of-the-box, I suggest simpleTCP. Here's an example of an echo server:
from simpletcp.tcpserver import TCPServer
def echo(ip, queue, data):
queue.put(data)
server = TCPServer("localhost", 5000, echo)
server.run()
And here's an example of a client connecting to it:
from simpletcp.clientsocket import ClientSocket
s1 = ClientSocket("localhost", 5000)
response = s1.send("Hello, World!")

Choosing recepient of socket.send - Python socket module

I am trying to make a group chat program, where an unlimited amount of clients may join the server using the same script, it'll work by the server receiving the clients message and sending it to all the connected clients including the sender. I have only managed to make it so that the sender only gets his own message back, but not what another client sends.
I was thinking of storing all the connected client IP's in a list, and sending it to each IP, but I do not know how to change the recpient of socket.send
Server code:
from threading import *
import socket
s = socket.socket()
host = socket.gethostbyname(socket.gethostname())
port = 1337
s.bind((host, port))
s.listen(5)
print("Server host is", host)
def getMainThread():
for thread in enumerate():
if thread.name == 'MainThread':
return thread
return None
class client(Thread):
def __init__(self, socket, address):
Thread.__init__(self)
self.socket = socket
self.address = address
self.start()
def run(self):
main = getMainThread()
while main and main.isAlive():
print(self.address, "has connected!")
message = self.socket.recv(8192).decode('utf-8')
self.socket.send(bytes(message, 'UTF-8'))
self.socket.close()
while True:
c, addr = s.accept()
client(c, addr)
clients = [] #list for all client IP's
clients.append(addr)
Also, is there a way so that the client can establish a connection with the server so it doesn't keep poping up on the server.py that client has connected each time it sends a message?
Client code:
import socket
import os
import sys
host = '25.154.84.23'
print("""
=======================================================
=Welcome to Coder77's local internet messaging service=
=======================================================
The current soon to be encrypted server is {0}
You can enter the command /help for a list of commands available
""".format(host))
#username = input("Enter username: ")
username = 'Smile'
print("Now connecting to {0}....".format(host))
def printhelp():
print("""
The following commands are in the current version of this program:
/clear to clear the screen
/username to change your username
/exit to exit
/help for a list of commands
""")
def main():
global username
global host
sock = socket.socket()
try:
sock.connect((host, 1337))
while True:
message2 = input("{0}: ".format(username))
message = ('{0}: {1}'.format(username,message2))
if '/quit' in message:
sys.exit()
if '/clear' in message:
os.system('cls')
if '/help' in message:
printhelp()
if '/username' in message:
username = input("What would you like as your new username? ")
sock.send(bytes(message, 'UTF-8'))
received = sock.recv(8192).decode('utf-8')
print(received)
except socket.error:
print("Host is unreachable")
if __name__ == '__main__':
main()
#
Corrected Server code:
import sys
print(sys.version)
from threading import *
import socket
s = socket.socket()
host = socket.gethostbyname(socket.gethostname())
port = 1337
s.bind((host, port))
s.listen(5)
print("Server host is", host)
def getMainThread():
for thread in enumerate(): # Imported from threading
if thread.name == 'MainThread':
return thread
return None
class Client(Thread):
def __init__(self, socket, address):
Thread.__init__(self)
self.socket = socket
self.address = address
self.start()
def run(self):
main = getMainThread()
print(self.address, "has connected!")
while main and main.isAlive():
message = self.socket.recv(8192).decode('utf-8')
print(message)
self.socket.send(bytes(message, 'UTF-8'))
for each_client in clients:
each_client.socket.send(bytes(message, 'UTF-8'))
while True:
c, addr = s.accept()
this_client = Client(c, addr)
clients = []
clients.append(this_client)
The new code, adapted by gravetii is causing a lot of format errors. What happens now, is the user gets back what he sent, he does not get back what other users send and the user gets back what his previous message was, its terribly confusing. Please run the code, and you'll see as it's very hard to explain.
Example

In your server code, you are doing only a self.socket.send(bytes(message, 'UTF-8')). How can you then expect the server to send the message to all the clients? To do that you would need to iterate through the list of clients and call the send() method on each of their sockets.
while True:
c, addr = s.accept()
client(c, addr)
clients = [] #list for all client IP's
clients.append(addr)
In this code, you are creating a client object but never adding it to the list, then what's the point of creating one?
I think what you want is this:
while True:
c, addr = s.accept()
this_client = client(c, addr)
clients = [] #list for all client IP's
clients.append(this_client)
Then, you can send the message to all the clients by modifying the relevant part of your server code:
def run(self):
main = getMainThread()
while main and main.isAlive():
print(self.address, "has connected!")
message = self.socket.recv(8192).decode('utf-8')
self.socket.send(bytes(message, 'UTF-8'))
for each_client in clients:
each_client.socket.send(bytes(message, 'UTF-8'))
Also, why are you closing the connection after sending just one message? I believe your intention is to send more than one message to the server, and in that case, you don't need to close the connection.
Also, it's a better idea to create a class with its name starting with an upper case letter. So you may want to use Client instead of client for the class name.
Now coming to the issue of the message popping up everytime a client says something in your server.py, look at the run() method for the client thread:
def run(self):
main = getMainThread()
while main and main.isAlive():
print(self.address, "has connected!")
message = self.socket.recv(8192).decode('utf-8')
self.socket.send(bytes(message, 'UTF-8'))
The thread starts executing as soon as you create the client object, and so the first time when it connects to the server, it is right in showing that message. But then it's incorrect to place the print(self.address, "has connected!") in the while loop. So everytime the client says something, the server sends it back to the client and then the loop runs again, thus displaying the message back again. You need to modify it like so:
def run(self):
print(self.address, "has connected!")
main = getMainThread()
while main and main.isAlive():
message = self.socket.recv(8192).decode('utf-8')
self.socket.send(bytes(message, 'UTF-8'))
Hope this helps!

Python Socket Listening

All of the below mentioned is on windows machines using python 2.7
Hello,
I am currently attempting to listen on a socket for data send by a remote program. This data is then printed to the screen and user input is requested that is then returned to remote program. In testing I have been able to have the remote program send me a menu of command line programs (cmd, ipconfig, whoami, ftp) and then my program returns with a number as a selection of the menu option.
The remote program receives my response and sends the output of the selected command. ipconfig and whoami work perfectly, but cmd and ftp only returns the output of the terminal once. (I.E. I can enter one command into the FTP program and send that too the remote program before I never hear back)
The part of my code that fails is that
if ready[0]: never becomes ready a second time after the first conversation.
I know the remote program is functioning correctly as I can use netcat to act in lieu of my code and operate the cmd terminal indefinitely.
How do I go about properly implementing a python socket listener that can account for this type of connection?
My "program" in its entirety:
import socket, sys, struct, time, select
host = ''
port = 50000
connectionSevered=0
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error:
print 'Failed to create socket'
sys.exit()
print '[+] Listening for connections on port '+str(port)+'.'
s.bind((host,port))
s.listen(5)
def recvall(the_socket,timeout=2):
global connectionSevered
data=''; # Data found by recv
total_data=[]; # Finally list of everything
s.setblocking(0) #make socket non blocking
begin=time.time() #beginning time
while 1:
ready = select.select([client], [], [], .2)
if time.time()-begin > timeout:
print 'Timeout reached'
#Leave loop, timer has reached its threshold
break
if ready[0]:
print 'In ready loop!'
try:
data = client.recv(4096) #attempt to fetch data
if data:
begin=time.time() #reset timeout timer
total_data.append(data)
data='';
except socket.error:
print '[+] Lost connection to client. Printing buffer...'
connectionSevered=1 # Let main loop know connection has errored
pass
time.sleep(1)
#join all parts to make final string
return ''.join(total_data)
client, address = s.accept()
print '[+] Client connected!'
while (connectionSevered==0): # While connection hasn't errored
print "connectionSevered="+str(connectionSevered) # DEBUG
recvall(s)
response = raw_input() #take user input
client.sendto(response) #send input
client.close(0)
Please let me know if you need more information, any help would be greatly appreciated, I am very new to this and eager to learn.

Playing around with this for a while finally got it working nice with a telnet session locally using python 2.7.
What it does is it sets up a thread that runs when the client connects listening for client stuff.
When the client sends a return ("\r\n" might have to change that if your interacting with a Linux system?) the message gets printed to the server, while this is happening if there is a raw input at the server side this will get sent to the client:
import socket
import threading
host = ''
port = 50000
connectionSevered=0
class client(threading.Thread):
def __init__(self, conn):
super(client, self).__init__()
self.conn = conn
self.data = ""
def run(self):
while True:
self.data = self.data + self.conn.recv(1024)
if self.data.endswith(u"\r\n"):
print self.data
self.data = ""
def send_msg(self,msg):
self.conn.send(msg)
def close(self):
self.conn.close()
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host,port))
s.listen(5)
except socket.error:
print 'Failed to create socket'
sys.exit()
print '[+] Listening for connections on port: {0}'.format(port)
conn, address = s.accept()
c = client(conn)
c.start()
print '[+] Client connected: {0}'.format(address[0])
c.send_msg(u"\r\n")
print "connectionSevered:{0}".format(connectionSevered)
while (connectionSevered==0):
try:
response = raw_input()
c.send_msg(response + u"\r\n")
except:
c.close()
The above answer will not work for more than a single connection. I have updated it by adding another thread for taking connections. It it now possible to have more than a single user connect.
import socket
import threading
import sys
host = ''
port = 50000
class client(threading.Thread):
def __init__(self, conn):
super(client, self).__init__()
self.conn = conn
self.data = ""
def run(self):
while True:
self.data = self.data + self.conn.recv(1024)
if self.data.endswith(u"\r\n"):
print self.data
self.data = ""
def send_msg(self,msg):
self.conn.send(msg)
def close(self):
self.conn.close()
class connectionThread(threading.Thread):
def __init__(self, host, port):
super(connectionThread, self).__init__()
try:
self.s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.s.bind((host,port))
self.s.listen(5)
except socket.error:
print 'Failed to create socket'
sys.exit()
self.clients = []
def run(self):
while True:
conn, address = self.s.accept()
c = client(conn)
c.start()
c.send_msg(u"\r\n")
self.clients.append(c)
print '[+] Client connected: {0}'.format(address[0])
def main():
get_conns = connectionThread(host, port)
get_conns.start()
while True:
try:
response = raw_input()
for c in get_conns.clients:
c.send_msg(response + u"\r\n")
except KeyboardInterrupt:
sys.exit()
if __name__ == '__main__':
main()
Clients are not able to see what other clients say, messages from the server will be sent to all clients. I will leave that as an exercise for the reader.

If you're in Python 3 by now and still wondering about sockets, here's a basic way of using them:
server.py
import time
import socket
# creating a socket object
s = socket.socket(socket.AF_INET,
socket.SOCK_STREAM)
# get local Host machine name
host = socket.gethostname() # or just use (host == '')
port = 9999
# bind to pot
s.bind((host, port))
# Que up to 5 requests
s.listen(5)
while True:
# establish connection
clientSocket, addr = s.accept()
print("got a connection from %s" % str(addr))
currentTime = time.ctime(time.time()) + "\r\n"
clientSocket.send(currentTime.encode('ascii'))
clientSocket.close()
client.py
import socket
# creates socket object
s = socket.socket(socket.AF_INET,
socket.SOCK_STREAM)
host = socket.gethostname() # or just use (host = '')
port = 9999
s.connect((host, port))
tm = s.recv(1024) # msg can only be 1024 bytes long
s.close()
print("the time we got from the server is %s" % tm.decode('ascii'))
Run server.py first, then run client.py.
This is just send and receive the currentTime.
What's new in Python 3.4 sockets?
A major difference between python 2.7 sockets and python 3.4 sockets is the sending messages. you have to .encode() (usually using 'ascii' or blank as parameters/arguments)
and then using .decode()
For example use .encode() to send, and use .decode() to receive.
Extra info: client/server socket tutorial

Python client sockets connecting on the same server socket port

I am working for the first time on sockets with python.
I need to connect more than a client socket to the same server socket.
To do this I used the the following code:
import socket
import time
import random
from threading import Thread
import thread
import subprocess, signal, os
class ServerObject:
def __init__(self, port):
self.socket = ''
self.host = ''
self.port = port
self.conn = ''
self.data = ''
print "Server port is: ", self.port
def openSocketConnectionAsServer(self):
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.bind((self.host, self.port))
self.socket.listen(5)
self.conn, addr = self.socket.accept()
print 'Server Connected by', addr
def getData(self):
return self.data
def getHost(self):
return self.host
def getPort(self):
return self.port
def sendMessage(self, message):
try :
self.conn.sendall(message)
except socket.error:
print 'Server Send failed'
sys.exit()
def receiveMessageFromClient(self):
while 1:
reply = self.conn.recv(4096)
if not self.data: break
return reply
def receiveMessageFromServer(self):
reply = self.socket.recv(4096)
return reply
def closeConnectionAsServer(self):
self.socket.shutdown(1)
self.socket.close()
del self.socket
class ClientObject:
def __init__(self):
self.data = str(random.choice('abcdefghil'))
print "Client Data Random: ", self.data
self.host = 'localhost'
self.port = ''
def openSocketConnectionAsClient(self):
self.port = 50000 + random.randint(1,3)
print "Client socket port is: ", self.port
try:
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error:
print 'Client Failed to create socket'
sys.exit()
print 'Client Socket Created'
try:
remote_ip = socket.gethostbyname( self.host )
except socket.gaierror:
print 'Client Hostname could not be resolved. Exiting'
sys.exit()
self.socket.connect((remote_ip , self.port))
def getData(self):
return self.data
def getHost(self):
return self.host
def getPort(self):
return self.port
def sendMessage(self, message):
try :
self.socket.sendall(message)
except socket.error:
print 'Client Send failed'
os.sys.exit()
def receiveMessageFromServer(self):
reply = self.socket.recv(4096)
return reply
def closeConnection(self):
self.socket.shutdown(1)
self.socket.close()
del self.socket
def clientProcess():
print "Client Start the client process"
client1 = ClientObject()
for i in range(1,10):
try:
print "Client try to open the connection socket: attempt number ",i
client1.openSocketConnectionAsClient()
print "Client connection socket established on port ", client1.getPort()
break
except socket.error:
print "Client connection error on the port ", client1.getPort()
if i == 10:
os.sys.exit()
print "Client send the message"
client1.sendMessage(client1.getData())
print "Client receive the message back"
client1.receiveMessageFromServer()
print "Client Data requested: ", client1.getData(), " Hostname: ", client1.getHost(), " Port: ", client1.getPort()
print "Client Close connection"
client1.closeConnection()
def serverProcess(port=40007):
print "Server oject create"
server = ServerObject(port)
print "Server open connection as server"
server.openSocketConnectionAsServer()
print "Server receive the client message"
receivedMessage = server.receiveMessageFromClient()
message = "Server Data requested are: "+receivedMessage
print "Server send the message back to the client"
server.sendMessage(message)
print "Server close the connection with the client"
server.closeConnectionAsServer()
if __name__ == '__main__':
threadServer = Thread(target=serverProcess, args=[50001])
print "New Server thread created with name: ", threadServer.getName()
threadServer.start()
for i in range(1,3):
threadClient = Thread(target=clientProcess)
print "New Client thread created with name: ", threadClient.getName()
threadClient.start()
threadClient.join()
When I run it, the first client is able to connect to the server, the second one never get connected, even though I close the connection (from server and client side) at the end of the first client thread.
Please, could anybody tell me, what's wrong in my code?
Thanks

I would highly recommend using the excellent socketserver module for doing stuff like this. As for your code, the problem is that your threaded socket server only runs in a single thread. At the very least, you should create a new thread to handle the connection after accepting the new connection in openSocketConnectionAsServer.

What #Ber said was correct but incomplete.
You're problem is here:
self.socket.listen(5)
self.conn, addr = self.socket.accept()
listen will open up the port and prepare to receive connections. accept will wait for the next connection.
While you only need to call listen once, you must call accept multiple times, once per connection. There are several ways you can arrange this. For starters you could call accept again when the current connection closes, but this will only allow one client at a time. Better to have one thread call accept to wait for the next connection, but have it start a worker thread to handle each one.
Or you could use non-blocking I/O, but if you go this way, check out Twisted.