Python: socket.recv() doesn't receive push messages
Hello,
I'm coding a socket based IMAP client in Python3 which successfully establishes a connection to the server, succussfully transmits the IDLE command but then fails to receive incoming data from the server.
If you are wondering why I do not use libimap or sth., the answer is easy: I just want to implement an IDLE command-supporting python client which must be written without that library.
An extract:
import socket
def runIMAPPeek():
#socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(29 * 60)
#connection
s.connect((IMAP_SERVER , 443))
#login
data = b"a1 LOGIN " + USER + b" " + PASSWORD + b"\n"
s.sendall(data)
reply = read(s)
#Idle loop
#As in RFC 3501 connection will be reinitialized every 29 minutes
while True:
# Idle command
print("#Sending IDLE...")
data = b"a2 IDLE\n"
s.sendall(data)
reply = read(s)
if reply.startswith("+ idling"):
print(" #idling.")
else:
print(" #Unexpected answer: {}".format(reply))
#sys.exit()
# waiting for incoming mails ----------------------------------
try:
push_msg = read(s)
# got push message = new message arrived
getNewEnvelope(s, push_msg)
except socket.timeout:
# timeout
print(" #timeout. Reinitializing IDLE...")
#TODO: except (KeyboardInterrupt, SystemExit)
# Quit Idle
data = b"DONE\n"
write(s, data)
reply = read(s)
if reply.startswith(prefix_str + " OK"):
print(" #quit idling.")
else:
print(" #Unexpected answer: {}".format(reply))
#sys.exit()
def read(s):
"""Read socket data, print it, convert to string, replace new lines
and return it.
"""
print("#Receiving...", end=" ")
reply = s.recv(4096)
reply = str(reply)[2:-1] #convert and remove byte indicators
reply = reply.replace("\\r\\n", "\n")
print(reply)
return reply
The problem is marked with the "----". Although messages are received in the mailbox, python does not react but remains in the idling/receiving state. In fact, the print line above the s.recv() command isn't even printed.
I tried everything successfully with Telnet, so there is no server problem.
In addition to my comment above, you have never selected INBOX. You will not receive any push messages, because you haven't told it what folder you want. Technically, IDLE is not valid in the unselected state.
Constructs like this one:
if reply.startswith("+ idling"):
are completely non-compliant. The IDLE RFC specifies that the client shall expect a continuation request, not this particular string (which also happens to be a continuation request).
Related
I am currently working on a program in Python that works as a client, and needs to connect remotely to a server using the TCP/IP protocol. After the server receives the client's username, the client can send messages to other clients by typing "#<username> <message>", and this input will be further processed and the message that will be sent to the server will be constructed as "SEND <username> <message>", and this will be actually recognized by the server. Then the server will send back an acknowledgement to the sending client, and the actual message to the destination client.
My approach is to use a main function named chat_run(), used for input and constructing the message that will be sent to the server, and in parallel to run a function named OutputRecvMsg() in a different thread that will receive messages from the server and output them in the console.
The problem is, I want the beginning of all the input lines to start with username >, and the messages received from the server to be output immediately on a new line, and the client to wait for a new input.
My current implementation problem seems to be in receiving messages (the OutputRecvMsg() function). After it outputs a message to the console, I need to press Enter to ask for input, because it remains stuck.
For me, there are two questions regarding the current problem, maybe two threads try to access the same resource (console), maybe I made a mistake regarding the construction of the received message (because I know that sock.recv(4096) is blocking and I tried to avoid a blocking state).
import socket
import time
import re
import threading as th
SERVER_REPLY_1 = 'HELLO'
SERVER_REPLY_2 = 'IN-USE'
AT_SYMBOL = '#'
host_port = ('remote_server_add', 5378)
def build_loggin_msg(msg):
return 'HELLO-FROM ' + msg + ' \n'
def chat_run(sock, currentUser):
while True:
rawInput = input(currentUser + '> ')
if rawInput == '!who':
sock.sendall('WHO\n'.encode())
elif rawInput == '!quit':
sock.close()
break
else:
splittedMsg = re.split(r'\s', rawInput, maxsplit = 1)
if len(splittedMsg) > 1 and splittedMsg[0].startswith(AT_SYMBOL):
userNameToSend = splittedMsg[0][1:]
message = 'SEND ' + userNameToSend + ' ' + splittedMsg[1] + ' \n'
sock.sendall(message.encode())
def OutputRecvMsg(sock, currentUser):
OutMsg =''
chunk = ''
while True:
try:
chunk = sock.recv(4096).decode()
if not chunk:
pass
else:
OutMsg += chunk
except BlockingIOError as e:
if OutMsg:
print(OutMsg)
OutMsg = ''
if __name__ == '__main__':
loggedIn = False
currentUser = None
_data = ''
while not loggedIn:
currentUser = input('Add a username please: ')
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect(host_port)
sock.sendall(build_loggin_msg(currentUser).encode())
data = sock.recv(4096).decode()
print(data)
if data.startswith(SERVER_REPLY_1):
loggedIn = True
else:
print('Close connection for retry another username')
sock.close()
time.sleep(1)
sock.setblocking(0)
th.Thread(target=OutputRecvMsg, args=(sock, currentUser)).start()
chat_run(sock, currentUser)
As an example:
Add a username please: Nickname
HELLO Nickname
Nickname> #Nickname hello man -> send to me
Nickname> DELIVERY Nickname hello man
SEND-OK -> here I have to press enter to get the next lines
Nickname>
I asked a question about my server to client code because I had many problems with and someone told me that the solution to the problems I had was to make a peer to peer chat which I have now done.
Server.py
import socket, threading
host = "127.0.0.1"
port = 4000
s = socket.socket()
s.bind((host,port))
s.listen(5)
client_sockets = []
users = []
print("Listening")
def handle_client(conn):
while True:
try:
data = conn.recv(512)
for x in client_sockets:
try:
x.send(data)
except Exception as e:
print(e)
except:
pass
while True:
conn,addr = s.accept()
client_sockets.append(conn)
print("Connections from", addr[0], "on port",addr[1])
threading.Thread(target = handle_client,args = (conn,)).start()
Client.py
import socket,threading
host = "127.0.0.1"
port = 4000
s = socket.socket()
s.connect((host,port))
def echo_data(sock):
while True:
try:
data = sock.recv(512)
print(data)
except:
pass
while True:
threading.Thread(target=echo_data,args=(s,)).start()
msg = input("Enter your message : ")
s.send(msg.encode())
The problems is that when I run the client and try talking to another client the message doesn't get sent unless the other client hits enter and also that brings me to my second problem, when the clients send messages to each other they get received in this format:
b'hi'Enter your message :
This is the link to my previous question
I will start with general problems not directly related to the question:
except: pass is generally a bad idea, specially when things go wrong because it will hide potentially useful messages. It is allowed by the language but should never exist in real code
in client.py you start a receiving thread per message, while you only need one for the whole client. You should start the thread outside the loop:
threading.Thread(target=echo_data,args=(s,)).start()
while True:
msg = input("Enter your message : ")
s.send(msg.encode())
Now for the questions:
the message doesn't get sent unless the other client hits enter
It can be caused by an IDE. Specifically, IDLE is known to behave poorly with multi-threaded scripts. If you correctly use one single receiving thread and starts the script from the command line (python client.py) it should work correctly
the messages get recived in this format: b'hi'Enter your message
sock.recv(sz) returns a byte string. You need to decode it to convert it to a Python 3 unicode string:
data = sock.recv(512)
print(data.decode())
But that is not all. It is fine for tests, but you should at least allow clients to disconnect from the server and when they do, remove them from client_sockets. And it is common not to send back a message to the sender. So you could improve the server.py loop:
while True:
try:
data = conn.recv(512)
for x in client_sockets:
if x != conn: # do not echo to sender
x.send(data)
except Exception as e: # problem in connection: exit the loop
print(e)
break
# clear the connection
conn.close()
client_sockets.remove(conn)
So I'm working on a simple chat server in python.
The server is running fine and on the client I have one thread for receiving incoming data and one for sending messages.
I can send a message from client_1 to the server which passes it to all other clients which will then print the message.
Even though everything technically works fine, there is still one thing that is VERY annoying whenever it happens:
Say client_1 is typing text into the console.
At the same time client_2 sends a message to the server, the server sends it to client_1 and client_1 prints the message.
Now the text client_1 was originally typing into the console is no longer in the line it was supposed to be in.
This is what the consoles looked like before client_2 sent the string "test test": https://ibb.co/hFdeo7
and this is what they looked like after sending: https://ibb.co/mEWAvn
NOTE: If I were to press Enter on client_1 the message "TEST TEST TEST" would still be sent correctly. The problem only lies in the conflict between the text that is being printed and the text in the input() statement.
My code looks like This:
Server.py
connections = []
while True:
readable, writeable, exception = select.select(connections, [], [], 0)
for sock in readable:
if sock == server:
conn, addr = server.accept()
connections.append(conn)
else:
data = str(sock.recv(1024), 'utf8')
if data:
for s in connections:
if s != server_socket and s != sock:
s.send(bytes(data, 'utf8'))
else:
connections.remove(sock)
Client.py
def receive():
while True:
readable, writeable, exception = select.select([0, client], [], [])
for sock in readable:
if sock == client:
data = str(sock.recv(1024), 'utf8')
if data:
print(data)
def send():
while True:
readable, writeable, exception = select.select([0, client], [], [])
for sock in readable:
if sock == client:
pass
else:
msg = input()
client.send(bytes(msg, 'utf8'))
Thread(target=receive).start()
Thread(target=send).start()
Is there any way to solve this problem without running the send() and receive() functions in separate scripts, or using a GUI module like Tkinter?
EDIT: I would like to print the incoming message as soon as it is received but then display the input() prompt and typed text again afterwards.
Here's one approach to showing the incoming message and then redisplaying the input prompt and partial input string (as specified in a comment). It uses readline.get_line_buffer to read the currently-input string and redisplay it. A warning, though: reading and writing to the same stream from different threads without locking is going to be prone to glitches.
It just requires a small modification to the receive function:
if data:
print('\n' + data)
sys.stdout.write(readline.get_line_buffer())
sys.stdout.flush()
The '\n' in the print is so the incoming message doesn't land right on top of whatever's being typed. The flush is necessary so that you can write the current input without a newline but without it getting buffered. I might suggest adding a prompt (like '> ') to the input and the sys.stdout.write, just to make it clearer to the user what's happening.
Lastly, running this may mess up your terminal output. You might need to run reset afterwards. There's probably a way to clean up that prevents that but I don't know offhand what it is.
I am trying to modify a server and client chat script implemented in python. One of the requirement is that the client exits when the user types CTRL-D. My question is how do I read that the user typed (ctrl-D) and implement it within this code. Should I just close the socket like c_socket.close() without any message back to the server that I am exiting?
Thanks!
# telnet program example
import socket, select, string, sys
import readline
def prompt() :
sys.stdout.write('<You> ')
sys.stdout.flush()
#main function
if __name__ == "__main__":
if(len(sys.argv) < 3) :
print 'Usage : python MClient hostname port'
sys.exit()
host = sys.argv[1]
port = int(sys.argv[2])
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(2)
# connect to remote host
try :
s.connect((host, port))
except :
print 'Unable to connect'
sys.exit()
print 'Connected to remote host. Start sending messages'
prompt()
while 1:
socket_list = [sys.stdin, s]
# Get the list sockets which are readable
read_sockets, write_sockets, error_sockets = select.select(socket_list , [], [])
for sock in read_sockets:
#incoming message from remote server
if sock == s:
data = sock.recv(4096)
if not data :
print '\nDisconnected from chat server'
sys.exit()
else :
#print data
sys.stdout.write(data)
prompt()
#user entered a message
else :
msg = sys.stdin.readline()
s.send(msg)
prompt()
Last thing first - whether you need to send something to the server before closing the socket or not depends purely on your protocol. Given the simplistic nature of the presented client code, I'd guess that closing of the socket should be enough (and the server should treat unannounced disconnects anyway as one shouldn't consider network I/O as persistent).
Second, CTRL+D will cause the returned message from sys.stdin.readline() to return as empty so you can test against that, e.g.:
msg = sys.stdin.readline()
if msg:
s.send(msg)
prompt()
else:
s.close()
sys.exit()
If you really need to send something to the server (again, depends on the protocol) you can use socket's shutdown() method before calling close().
However, keep in mind that the code presented here doesn't account for pressing the CTRL+D while reading from the socket (i.e. receiving data) so if there is a particularly long data stream coming in your CTRL+D won't be registered. You can solve this by checking the user input (and writing it to buffer for later sending) during the retrieval procedure, or you can just place your receiving code in a separate thread and leave the main thread just for user input.
You can then use the atexit module to cleanly exit from your client at any time.
[EDIT:]
I'm currently trying to make a small tcp chat application. Sending and receiving messages already works fine... But the problem is:
When i start typing a message while i receive one... it appears after the text I'm writing
Screenshot: http://s7.directupload.net/images/140816/6svxo5ui.png
[User sent > "hello", then I started writing "i am writing..." then user wrote " i sent a..." before i sent my message... so it has been placed after my input...
I want the incoming message always to be before my input !
this is my current code:
Client.py
con = connect.User()
server = raw_input("Type in the server adress \n[leave blank to use xyr.no-ip.info]\n>:")
nick =""
while nick == "":
nick = raw_input("Type in your nickname\n>:")
con.connect(server, nick)
def sender():
print("Sender started")
while 1:
msg = raw_input()
if msg == "q":
break
con.send(msg)
con.disconnect()
def receiver(server):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
if server == "":
server="xyr.no-ip.info"
sock.connect((server, 8000))
sock.send("%pyreceiver\n")
print("Receiver started")
while 1:
msg_in = sock.recv(1024)
if not str(msg_in).startswith("[py]" + nick):
if str(msg_in).startswith("/ua"):
print(str(msg_in)[3:])
elif str(msg_in).startswith("/u "):
print(str(msg_in)[2:])
else:
print(str(msg_in[:-1]))
#
if nick == "":
nick = "guest"
print("Name changed to ""guest""")
time.sleep(.5)
thread.start_new_thread(receiver, (server, ))
time.sleep(.5)
thread.start_new_thread(sender())
Connect.py
import socket
import time
class User():
nickel =""
def connect(self, server="xyr.no-ip.info", nick="guest"):
nickel = nick
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
if server == "":
server="xyr.no-ip.info"
print("server changed to xyr.no-ip.info")
time.sleep(.5)
print("Connecting...")
self.sock.connect((server, 8000))
print("Connected")
time.sleep(.4)
self.sock.send("[py]" + nick + "\n")
self.sock.send(nick + " connected with a python client\n")
print("registered as " + nick)
time.sleep(.3)
def send(self, msg):
self.sock.send(msg + "\n")
def disconnect(self):
self.sock.close()
print("disconnected")
Your code writes everything to stdout. Whenever something arrives to either of your sender/receiver threads, it prints to stdout. The issue with that is, due to the fundamental nature of output streams, you cannot accomplish the following :
place incoming messages above the stuff currently being typed/echoed.
Things happen strictly in the order of occurrence. The moment something comes in, wherever the cursor is, the print statement dumps that data over there. You cannot modify that behaviour without using fancier / more powerful constructs.
In order to do what you want, I would use ncurses. You seem to be using python on Windows, so you're going to have to do some digging on how to get equivalent functionality. Check out this thread : Curses alternative for windows
I had a similar problem and I found that a simpler solution (for me) was to get input via readchar (https://github.com/magmax/python-readchar/tree/master/readchar).
Using readchar, I would buffer each keystroke (checking for key.BACKSPACE and CR - see code snippet below).
All output I would prepend with "/033[1A" (make the cursor move up), print the output line, and then a "/n"...
after each output line, I move the cursor to the beginning and re-print the self.input_buff
while the user is doing input, this handles console input, displaying what they are typing:
keypress = readkey()
if keypress == key.BACKSPACE:
self.input_buff = self.input_buff[:-1]
print("\033[0A%s " % self.input_buff)
continue
if keypress != key.CR:
self.input_buff = "%s%s" % (self.input_buff, keypress)
print("\033[0A%s" % self.input_buff)
continue
This kept the input line at the bottom and all terminal output above it.
I know it comes a year late and if you are a wiz with curses, maybe that is the way to go...