I have this following code which is in my server.py file. It is waiting to receive data from the client. Additionally I cannot use any of the http libraries. Only the socket Library:
def handle_client(conn, addr):
print ('New client from', addr)
x = []
try:
while True:
data = conn.recv(1024)
decoded_data = data.decode('utf-8')
# if "GET / " in data.decode('utf-8'):
# handle_client_get(conn)
# else:
if data:
print(data)
x.append(decoded_data)
else:
print(x)
break
finally:
print("in close now")
conn.close()
The problem I am having is that I can only reach the print(x) statement once I manually CTRL + Cclose the client. Otherwise it doesn't print.
Why is that the case.
ANSWER
You need to send an acknowledgment to the client such that the data sent has been received correctly.
that will terminate the connection and not wait for a timeout.
This is because the client sends: Expect: 100-continue
And you need to send an acknowledgment back to the client
You need to implement the protocol which will have a means to tell you how much data there is to read. In the case of HTTP, the request starts with a CRLF delimited header, and we can read that to get the information we want.
w3.org has a good description of the http request protocol. Its more complicated than I want to implement here but I've included an example that pulls in the request header by reading the socket one character at a time and looking for an empty \n terminated line. By reading one character at a time, I don't have to implement my own line buffer.
The first line is the request method and the remaining lines are other parameters included with the request. For a POST, for instance, there would be more data still to read from the socket.
import re
def handle_client(conn, addr):
print ('New client from', addr)
header = []
line = []
try:
# read ascii http client header.
while True:
c = conn.recv(1)
# check for early termination
if not c:
return None
# check for end of request line
elif c == b"\n":
# make line a string to add to header
line = ''.join(line).decode('ascii').strip()
# are we at the empty line signalling end-of-header?
if not line:
break
header.append(line)
line = []
# filter out \r
elif c == b"\r":
continue
# request is first line of header
request_line = header.pop(0)
method, uri, http_version = re.split(r" +" request_line)
if method.upper() == "GET":
# this function needs more parameters... the uri to get and the protocol
# version to use.
handle_client_get(...)
finally:
print("in close now")
conn.close()
Edit:
Now that you mention this is a HTTP server and is to be done using only socket library, I suggest you remove certain complex elements from the program at this stage like removing the while block until this problem is solved.
server.py
def handle_client(conn, addr):
print ('New client from', addr)
x = []
try:
data = conn.recv(1024)
decoded_data = data.decode('utf-8')
if data:
print(data)
x.append(decoded_data)
else:
print(x)
break
finally:
print("in close now")
conn.close()
Original Answer:
Try sending an additional \n at the sender end and use the flush function later to send out any remaining data on the buffer something like:
sender.py
def send_data(data,sock_obj,addr):
sock_obj.send(data+"\n")
sock_obj.flush()
Related
lines = f.readlines()
client = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
for line in lines:
client.sendto(line.encode(), (serverName,serverport))
res,address = client.recvfrom(1024)
res = res.decode()
The code does not send requests on the second iteration of the for loop. I am confused by how to approach this.
server.py
lines = "hello\nworld\nmulti\nline".splitlines()
client = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
for line in lines:
client.sendto(line.encode(), (serverName,serverport))
# recvfrom will block **until** it gets a reply
res,address = client.recvfrom(1024)
res = res.decode()
client.py
...
while True:
data,address = server.recvfrom(1024)
print(f"RECV: {data!r}")
# allow a small delay to make sure the other end is actively waiting on a response when you send
time.sleep(0.1)
server.sendto(b"OK\r\n",address)
print("Replied so server can go on")
since udp is inherently lossy (although generally not) you might still want to put a timeout on the client in the server.py script ... or remove the expectation of a response by simply removing the res,addres = client.recvfrom(...) line
server.py:
json files from NVD are used here
import socket, json, random, threading, zipfile, requests, re, zipfile
from bs4 import BeautifulSoup
from zipfile import *
def listen_user(user):
for x in range(2018,2021,1):
filename = "nvdcve-1.1-" + str(x) + ".json"
print(filename)
with open(filename, 'rb') as file:
sendfile = file.read()
user.sendall(sendfile)
print('file sent' + str(x))
def start_server():
while True:
user_socket, address = server.accept()
print(f"User <{address[0]}> connected!")
users.append(user_socket)
listen_accepted_user = threading.Thread(
target=listen_user,
args=(user_socket,)
)
listen_accepted_user.start()
if __name__ == '__main__':
users = []
server = socket.socket(
socket.AF_INET,
socket.SOCK_STREAM,
)
server.bind(
("127.0.0.1", 100)
)
server.listen(5)
print('waiting for connection...')
start_server()
client.py
import socket, json, random
from threading import Thread
def start_client(client):
savefilename = str(random.randint(1,10)) + 'new.json'
print(savefilename)
with client,open(savefilename,'wb') as file:
while True:
recvfile = client.recv(4096)
if not recvfile:
print('1 client')
break
file.write(recvfile)
file.close()
print('2 client')
client.close()
if __name__ == '__main__':
client = socket.socket(
socket.AF_INET,
socket.SOCK_STREAM,
)
client.connect(
("127.0.0.1", 100)
)
start_client(client)
when I send files - they are sent almost in full, but the program does not reach the line "print ('1 client')" or "print ('2 client')"
and the *new file contains all lines except a few dozen of the last
please help - how to fix the code?
recvfile = client.recv(4096) is inside the while loop and it is continuously waiting for the next bytes to receive. The client doesn't know the files are sent, so it waits for the next 4096 bytes and doesn't exit the loop.
To let the client know that the file transfer is completed, you can send a message from the server.py which you can validate in the client and break the loop as shown below.
server.py
def listen_user(user):
for x in ["f.json","g.json"]:
filename = x
print(filename)
with open(filename, 'rb') as file:
sendfile = file.read()
user.sendall(sendfile)
print('file sent' + str(x))
user.send(b"Done")
Client.py
def start_client(client):
savefilename = str(random.randint(1,10)) + 'new.json'
print(savefilename)
with client,open(savefilename,'wb') as file:
while True:
recvfile = client.recv(4096)
if recvfile.decode("utf-8") =="Done":
print('1 client')
file.close()
break
file.write(recvfile)
print('2 client')
client.close()
The call client.recv(4096) means that you are waiting for 4096 bytes to be received, then doing something with those bytes. What's likely happening in this case is that you're writing out all of the bytes, minus those that don't quite fill up the buffer at the end. This leaves the client waiting with a buffer with space that is doesn't think it is ready to write out yet.
I'm guessing that you're assuming that client.recv() will return an empty string once you've gotten all the data; this is not the case based on your code. If you want the client to be able to terminate the connection, you're going to need to send some kind of control sequence or try to otherwise assess the bytes received from the server to determined when it's time to close the connection. If you do this, you will probably want to set bufsize when calling client.recv() to 1, and instead use some other method to buffer before you write to a file.
For instance, since you're sending JSON data, you could concatenate the bytes to a variable and then repeatedly try to parse JSON. Once you have managed to successfully parse JSON, you can terminate the connection on the client side (though this would mean you have to open a new connection per file you're sending).
However, that raises the question: why do you need to close from the client side? Usually the server will just close the connection once it is done sending all of the relevant data.
7 socket listener setup. It works great and keeps the connection open, non blocking, all that. From time to time a file will show up that I need to hand back to the client. That works to, but it only send the data in the file if the client sends a character first. I need to have it send the data when the file shows up and not wait. I am coming from php and know what I am doing there. Python is new to me so there are some nuances I don't understand about this code.
while True:
try:
#I want this bit here to fire without waiting for the client to send anything
#right now it works except the client has to send a character first
#check for stuff to send back
for fname in os.listdir('data/%s/in' % dirname):
print(fname)
f = open('data/%s/in/%s' % (dirname, fname), "r")
client.send(f.readline())
data = client.recv(size)
if data:
bucket=bucket+data
else:
raise error('Client disconnected')
except Exception as e:
client.close()
print(e)
return False
Python: socket.recv() doesn't receive push messages
Hello,
I'm coding a socket based IMAP client in Python3 which successfully establishes a connection to the server, succussfully transmits the IDLE command but then fails to receive incoming data from the server.
If you are wondering why I do not use libimap or sth., the answer is easy: I just want to implement an IDLE command-supporting python client which must be written without that library.
An extract:
import socket
def runIMAPPeek():
#socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(29 * 60)
#connection
s.connect((IMAP_SERVER , 443))
#login
data = b"a1 LOGIN " + USER + b" " + PASSWORD + b"\n"
s.sendall(data)
reply = read(s)
#Idle loop
#As in RFC 3501 connection will be reinitialized every 29 minutes
while True:
# Idle command
print("#Sending IDLE...")
data = b"a2 IDLE\n"
s.sendall(data)
reply = read(s)
if reply.startswith("+ idling"):
print(" #idling.")
else:
print(" #Unexpected answer: {}".format(reply))
#sys.exit()
# waiting for incoming mails ----------------------------------
try:
push_msg = read(s)
# got push message = new message arrived
getNewEnvelope(s, push_msg)
except socket.timeout:
# timeout
print(" #timeout. Reinitializing IDLE...")
#TODO: except (KeyboardInterrupt, SystemExit)
# Quit Idle
data = b"DONE\n"
write(s, data)
reply = read(s)
if reply.startswith(prefix_str + " OK"):
print(" #quit idling.")
else:
print(" #Unexpected answer: {}".format(reply))
#sys.exit()
def read(s):
"""Read socket data, print it, convert to string, replace new lines
and return it.
"""
print("#Receiving...", end=" ")
reply = s.recv(4096)
reply = str(reply)[2:-1] #convert and remove byte indicators
reply = reply.replace("\\r\\n", "\n")
print(reply)
return reply
The problem is marked with the "----". Although messages are received in the mailbox, python does not react but remains in the idling/receiving state. In fact, the print line above the s.recv() command isn't even printed.
I tried everything successfully with Telnet, so there is no server problem.
In addition to my comment above, you have never selected INBOX. You will not receive any push messages, because you haven't told it what folder you want. Technically, IDLE is not valid in the unselected state.
Constructs like this one:
if reply.startswith("+ idling"):
are completely non-compliant. The IDLE RFC specifies that the client shall expect a continuation request, not this particular string (which also happens to be a continuation request).
My client sends a string "abcd" and half closes the socket (the write part). My server reads the data and appends it inside the list (collection) until end of file is recieved (half closed socket detected). Then it iterates through the list and sends the data.
My server code:
while True:
try:
sock,address = self.__mySocket.accept()
except:
print "Client is dead"
break
print "Client connect: " + str(address)
collection = []
while True:
data = sock.recv()
if len(data) == 0:
break
data = str(data[::-1])
collection.append(data)
for val in collection:
sock.send(val)
sock.close()
The client:
sslsock.sendall('abcd\n')
time.sleep(1)
sslsock.shutdown(socket.SHUT_WR)
data = ""
while True:
data = sslsock.recv()
if len(data) == 0:
sslsock.close()
sys.exit(1)
print data
Now when I print the data on the client it just print garbage. I've tried using pickle and that didn't work either. Now, when I comment out the shutdown on the client and work my server around it just works fine. It prints the reverse of the sent data.
In server Code. I put the for loop inside the if len(data) ==0 . And, It works. I 'm guessing that break statement was breaking out of even the outside While True. So, it never got to the point of sending.