So I'm working on a simple chat server in python.
The server is running fine and on the client I have one thread for receiving incoming data and one for sending messages.
I can send a message from client_1 to the server which passes it to all other clients which will then print the message.
Even though everything technically works fine, there is still one thing that is VERY annoying whenever it happens:
Say client_1 is typing text into the console.
At the same time client_2 sends a message to the server, the server sends it to client_1 and client_1 prints the message.
Now the text client_1 was originally typing into the console is no longer in the line it was supposed to be in.
This is what the consoles looked like before client_2 sent the string "test test": https://ibb.co/hFdeo7
and this is what they looked like after sending: https://ibb.co/mEWAvn
NOTE: If I were to press Enter on client_1 the message "TEST TEST TEST" would still be sent correctly. The problem only lies in the conflict between the text that is being printed and the text in the input() statement.
My code looks like This:
Server.py
connections = []
while True:
readable, writeable, exception = select.select(connections, [], [], 0)
for sock in readable:
if sock == server:
conn, addr = server.accept()
connections.append(conn)
else:
data = str(sock.recv(1024), 'utf8')
if data:
for s in connections:
if s != server_socket and s != sock:
s.send(bytes(data, 'utf8'))
else:
connections.remove(sock)
Client.py
def receive():
while True:
readable, writeable, exception = select.select([0, client], [], [])
for sock in readable:
if sock == client:
data = str(sock.recv(1024), 'utf8')
if data:
print(data)
def send():
while True:
readable, writeable, exception = select.select([0, client], [], [])
for sock in readable:
if sock == client:
pass
else:
msg = input()
client.send(bytes(msg, 'utf8'))
Thread(target=receive).start()
Thread(target=send).start()
Is there any way to solve this problem without running the send() and receive() functions in separate scripts, or using a GUI module like Tkinter?
EDIT: I would like to print the incoming message as soon as it is received but then display the input() prompt and typed text again afterwards.
Here's one approach to showing the incoming message and then redisplaying the input prompt and partial input string (as specified in a comment). It uses readline.get_line_buffer to read the currently-input string and redisplay it. A warning, though: reading and writing to the same stream from different threads without locking is going to be prone to glitches.
It just requires a small modification to the receive function:
if data:
print('\n' + data)
sys.stdout.write(readline.get_line_buffer())
sys.stdout.flush()
The '\n' in the print is so the incoming message doesn't land right on top of whatever's being typed. The flush is necessary so that you can write the current input without a newline but without it getting buffered. I might suggest adding a prompt (like '> ') to the input and the sys.stdout.write, just to make it clearer to the user what's happening.
Lastly, running this may mess up your terminal output. You might need to run reset afterwards. There's probably a way to clean up that prevents that but I don't know offhand what it is.
Related
I am trying to do multiple while loops but somehow they don't work. I already searched the internet but none of the problems I found has the same issue.
So here is the code containing only the necessary information. I am basically opening a socket, giving an in input (i\n) and receiving the output in the first step. I want to continue receiving the output until I have some specific characters xxx in the output. Then I want to go to the elif statement in the next loop.
def netcat(h, p):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((h,p))
i = 0
a = True
while a == True:
socket_list=[sys.stdin, s]
r,w,x = select.select(socket_list, [], [])
if i==0:
time.sleep(1)
message = s.recv(1024)
print(message)
s.send("i\n")
sys.stdout.flush()
while "xxx" not in message:
message = s.recv(1024)
print(message)
i+=1
elif i==1:
print("+++++++++++++++++++++++++++")
i+=1
print("hello")
server.close()
What I would expect the code to do is to print the message from the if statement, then print hello, then the message from the elif statement and then hello over and over again because the while loop is still active. So in summary this is the expected output:
message
hello
+++++++++++++++++++++++++++
hello
hello
hello
hello...
What it really prints is
message
hello
and then it finishes.
What I found out is that if I comment out the following lines:
while "xxx" not in message:
message = s.recv(1024)
print(message)
it works as expected. The hello at the end of the code gets printed to the screen over and over again. I just don't get it why this second while loop has anything to do with it. I would really appreciate help here.
Since the working code was requested, here is also the full code. The hostname and port are from a CTF which is still working so you will be interacting with the CTF-server:
#!/usr/bin/env python
import socket
import time
import select
import sys
base64="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/ ="
hostname = "18.188.70.152"
port = 36150
def netcat(h, p):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((h,p))
i = 0
a = True
b = True
while a == True:
socket_list=[sys.stdin, s]
r,w,x = select.select(socket_list, [], [])
if i==0:
time.sleep(1)
message = s.recv(1024)
print(message)
s.send("i\n")
sys.stdout.flush()
while "flag" not in message:
message = s.recv(1024)
print(message)
txtfile = message[9:38]
print(txtfile)
i+=1
elif i==1:
print("+++++++++++++++++++++++++++")
i+=1
print("hello")
server.close()
netcat(hostname, port)
You're mixing event-based code (select.select()) with blocking synchronous code (your small while loop with the s.recv()).
If you want your code not to block, every recv() needs to be paired up with a preceding select().
Not only that, but you must also check the returned values from select(). Only s.recv() if s was in the first returned list. If you s.recv() in any other case, the code will also block on the receive call.
Update:
Try something along the lines of:
not_done = True
while not_done:
read_sockets, _, _ = select.select([sys.stdin, s], [], [])
if s in read_sockets:
message = s.recv(1024)
print(message)
... more code ...
if 'flag' in message:
... react on flag ...
if 'quit' in message:
not_done = False
... processing of other sockets or file descriptors ...
The important point being that there is only this one s.recv() in the if branch that checks for whether select detected something was received.
The outer while will just come back to the same if branch later when additional data was received.
Note that processing stdin alongside socket code is tricky and will likely also block at some point. You will likely have to put the terminal into raw mode or something first and then be ready to process partial lines yourself as well as maybe also echoing the input back to the user.
Update:
If you want to do something while no message was received, you can give a timeout to select() and then do other processing if there was nothing received on the socket. Something like this:
say_hello_from_now_on = False
not_done = True
while not_done:
read_sockets, _, _ = select.select([s], [], [], 1)
if s in read_sockets:
message = s.recv(1024)
print(message)
say_hello_from_now_on = True
elif say_hello_from_now_on:
print("hello")
I'd check your indentation, try installing and running autopep8 on your code and see if that fixes any of your issues.
[edit] user has updated their question and it's clear that this isn't the answer.
I'm working on assignment where I need to connect to a server (no details of server are disclosed), capture the reply, modify it and send it back for verification.
I have created following code which does what I need, but the issue is that after 1st correct reply, server sends another.
Code:
# -*- encoding: utf-8 -*-
import socket
from decryptmsg import decryptmsg
from cleanmsg import cleanmsg
#connection
ip="<IP>"
port=4000
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((ip,port))
def recvall(sock):
BUFFER = 8192
data = b''
while True:
part = sock.recv(BUFFER)
data += part
if len(part) < BUFFER:
break
return data
while True:
print "[+] start communication"
data = recvall(sock)
print data
data = cleanmsg(data)
if data != None:
valmis = decryptmsg(str(data))
if valmis == None:
print "[-] no results"
break
else:
print "[+] sending message... "
sock.send(valmis)
continue
When I hit the second question, I get the input captured fine with this code and processed as expected, but when I try to send the 2nd reply back I get error:
Traceback (most recent call last):
File "challenge.py", line 28, in <module>
sock.send(valmis)
socket.error: [Errno 32] Broken pipe
If I do not close or shutdown the socket, no reply is ever sent to server.
How can I tell my client to send the message and wait for reply without socket.shutdown? Or if I need to open new socket for each loop, how should the loop be constructed? The reply from server changes each time so if I open new connection completely and request for data, I get new reply and the process starts from beginning again.
UPDATE:
the issue seems to be when trying to receive the second reply from server, only the first line of message is received by client.
How do you know it does not send anything? I modified your code a bit (there is something odd in the else: clause, I will come back to that later).
import socket
#connection
ip="localhost"
port=4000
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((ip,port))
while True:
data = sock.recv(8192)
if not data: break;
print data
if data != None:
valmis = data
if valmis == None:
print "[-] no results"
break
else:
print "[+] sending message... "
sock.send(valmis) #this never gets sent without
continue
Basically this is a stripped version of your code - no decrypting or external functionality. It just sends back whatever it receives from the server.
Then I ran a "server" with ncat:
ncat -l 4000
start your program and start typing in lines (1, 2, 3, 4 etc) and this happens at "server". The client promptly echoes my messages back:
test#xyzzy:/tmp$ ncat -l 4000
1
1
2
2
3
3
And this happens at the client:
test#xyzzy:/tmp$ python so.py
1
[+] sending message...
2
[+] sending message...
3
[+] sending message...
To me it looks this code works fine. If the server does not receive your reply, it might be that there is a problem on the server side. It might for example expect a terminator character in the response. Does your cleanmsg clean the message too much and for example remove a trailing newline and the server expects to receive one?
There is a problem in your original else clause as you do another sock.recv() there. Which means after receiving a reply, you block there to wait for the next message from server and when you do receive one, you will continue your loop and hit sock.recv() again. The second message was consumed already in your else clause.
This may be intentional if your server somehow acknowledges your decryption. If your protocol is this:
server -> client (request)
client -> server (decrypted message)
server -> client (some kind of acknowledgement - unclear from your code)
server -> client (request 2)
etc.
Then you have probably hit the issue in Jason's comment. TCP sockets are completely agnostic to the concept of a message. They just transmit data. When your code hits sock.recv(), one of five things can happen:
There is nothing in the socket and the call blocks
There is a full "message" and only that in the socket and you receive that
There is a partial message and you will receive that. Either because the message is over 8192 bytes, or your code just decides to read when the server has only transmitted some of the message data.
There are two or more complete "messages" waiting and you will receive them all.
As four, but the last message is partial
Always when operating with TCP sockets, you must cater for scenarios 2-5. You must parse the data, make sure everything is there, and if not, wait for more. And if there was more than you expected, process them accordingly. If they are complete messages, process them. If the last message is partial, process everything else and wait for more.
If messages seem to "disappear" in a self-made communication protocol using TCP sockets, 99% of the problems are caused by making the assumption that sockets would know or care what is the structure of your "message". A very common mistake is to read your socket empty and ignore everything you received after your first message.
Hope this is helpful. Your code without the additional recv seems to work fine from socket communication perspective - no need to shut down the socket. It may be a server side issue, protocol issue or message parsing problem.
For this reason, always have only one recv call for your socket. Even if you are expecting some kind of an acknowledgement instead of a new message, have only one place where you process socket data. Then do some kind of a conditional there to detect what kind of a message you received and then decide what to do with it.
The server is already set up. I connected to it and got it to return any data. The received data printed for the first piece of data that I sent. But then I tried to send something else, which, in theory, should return another string, but it just says that it's still expecting it.
At the moment, this is my code:
import socket
clientsocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
clientsocket.connect(('ServerHostHere', PortNumberHere))
clientsocket.send('hello')
dataOne = clientsocket.recv(2048)
print(dataOne)
This returned : "Hello to you too!"
Then I tried sending another piece of information:
clientsocket.send('How are you?')
dataTwo = clientsocket.recv(1024)
print(dataTwo)
Instead of returning a line, it just says "Error, expecting command question") meaning that it didn't receive my second command.
Is my code wrong or am I supposed to add something else?
I am trying to modify a server and client chat script implemented in python. One of the requirement is that the client exits when the user types CTRL-D. My question is how do I read that the user typed (ctrl-D) and implement it within this code. Should I just close the socket like c_socket.close() without any message back to the server that I am exiting?
Thanks!
# telnet program example
import socket, select, string, sys
import readline
def prompt() :
sys.stdout.write('<You> ')
sys.stdout.flush()
#main function
if __name__ == "__main__":
if(len(sys.argv) < 3) :
print 'Usage : python MClient hostname port'
sys.exit()
host = sys.argv[1]
port = int(sys.argv[2])
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(2)
# connect to remote host
try :
s.connect((host, port))
except :
print 'Unable to connect'
sys.exit()
print 'Connected to remote host. Start sending messages'
prompt()
while 1:
socket_list = [sys.stdin, s]
# Get the list sockets which are readable
read_sockets, write_sockets, error_sockets = select.select(socket_list , [], [])
for sock in read_sockets:
#incoming message from remote server
if sock == s:
data = sock.recv(4096)
if not data :
print '\nDisconnected from chat server'
sys.exit()
else :
#print data
sys.stdout.write(data)
prompt()
#user entered a message
else :
msg = sys.stdin.readline()
s.send(msg)
prompt()
Last thing first - whether you need to send something to the server before closing the socket or not depends purely on your protocol. Given the simplistic nature of the presented client code, I'd guess that closing of the socket should be enough (and the server should treat unannounced disconnects anyway as one shouldn't consider network I/O as persistent).
Second, CTRL+D will cause the returned message from sys.stdin.readline() to return as empty so you can test against that, e.g.:
msg = sys.stdin.readline()
if msg:
s.send(msg)
prompt()
else:
s.close()
sys.exit()
If you really need to send something to the server (again, depends on the protocol) you can use socket's shutdown() method before calling close().
However, keep in mind that the code presented here doesn't account for pressing the CTRL+D while reading from the socket (i.e. receiving data) so if there is a particularly long data stream coming in your CTRL+D won't be registered. You can solve this by checking the user input (and writing it to buffer for later sending) during the retrieval procedure, or you can just place your receiving code in a separate thread and leave the main thread just for user input.
You can then use the atexit module to cleanly exit from your client at any time.
Python: socket.recv() doesn't receive push messages
Hello,
I'm coding a socket based IMAP client in Python3 which successfully establishes a connection to the server, succussfully transmits the IDLE command but then fails to receive incoming data from the server.
If you are wondering why I do not use libimap or sth., the answer is easy: I just want to implement an IDLE command-supporting python client which must be written without that library.
An extract:
import socket
def runIMAPPeek():
#socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(29 * 60)
#connection
s.connect((IMAP_SERVER , 443))
#login
data = b"a1 LOGIN " + USER + b" " + PASSWORD + b"\n"
s.sendall(data)
reply = read(s)
#Idle loop
#As in RFC 3501 connection will be reinitialized every 29 minutes
while True:
# Idle command
print("#Sending IDLE...")
data = b"a2 IDLE\n"
s.sendall(data)
reply = read(s)
if reply.startswith("+ idling"):
print(" #idling.")
else:
print(" #Unexpected answer: {}".format(reply))
#sys.exit()
# waiting for incoming mails ----------------------------------
try:
push_msg = read(s)
# got push message = new message arrived
getNewEnvelope(s, push_msg)
except socket.timeout:
# timeout
print(" #timeout. Reinitializing IDLE...")
#TODO: except (KeyboardInterrupt, SystemExit)
# Quit Idle
data = b"DONE\n"
write(s, data)
reply = read(s)
if reply.startswith(prefix_str + " OK"):
print(" #quit idling.")
else:
print(" #Unexpected answer: {}".format(reply))
#sys.exit()
def read(s):
"""Read socket data, print it, convert to string, replace new lines
and return it.
"""
print("#Receiving...", end=" ")
reply = s.recv(4096)
reply = str(reply)[2:-1] #convert and remove byte indicators
reply = reply.replace("\\r\\n", "\n")
print(reply)
return reply
The problem is marked with the "----". Although messages are received in the mailbox, python does not react but remains in the idling/receiving state. In fact, the print line above the s.recv() command isn't even printed.
I tried everything successfully with Telnet, so there is no server problem.
In addition to my comment above, you have never selected INBOX. You will not receive any push messages, because you haven't told it what folder you want. Technically, IDLE is not valid in the unselected state.
Constructs like this one:
if reply.startswith("+ idling"):
are completely non-compliant. The IDLE RFC specifies that the client shall expect a continuation request, not this particular string (which also happens to be a continuation request).