We Keep Coding

Socket programming Python server side gets stuck when received msg from client

I wanted to create a software that can send video through socket programming. I can send 1 video at a time but when I want to send more than 2, it is stuck.
Below is the server code:
import socket
import os
IP = "127.0.0.1"
PORT = 4456
SIZE = 1024
FORMAT = "utf"
SERVER_FOLDER = "video_folder"
def main():
print("[STARTING] Server is starting.\n")
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind((IP, PORT))
server.listen()
print("[LISTENING] Server is waiting for clients.")
conn, addr = server.accept()
print(f"[NEW CONNECTION] {addr} connected.\n")
""" Receiving the folder_name """
folder_name = conn.recv(SIZE).decode(FORMAT)
""" Creating the folder """
folder_path = os.path.join(SERVER_FOLDER, folder_name)
if not os.path.exists(folder_path):
os.makedirs(folder_path)
conn.send(f"Folder ({folder_name}) created.".encode(FORMAT))
else:
conn.send(f"Folder ({folder_name}) already exists.".encode(FORMAT))
""" Receiving files """
while True:
msg = conn.recv(SIZE).decode(FORMAT)
""" Recv the file name """
print(f"[CLIENT] Received the filename: {msg}.")
file_path = os.path.join(folder_path, msg)
file = open(file_path, "wb")
conn.send(f"{file_path} filename received.".encode(FORMAT))
while True:
msg = conn.recv(SIZE) # it stuck here once the msg become nothing need help
if not msg:
conn.send("The data is saved.".encode(FORMAT))
break
file.write(msg)
print('.', end='', flush=True)
if not msg: break
file.close()
print("\ndone.")
def test():
while True:
x = "World"
print("morning")
while True:
print("Hello" + x)
break
if __name__ == "__main__":
main()
Below is the client code:
import socket
import os
IP = "127.0.0.1"
PORT = 4456
SIZE = 1024
FORMAT = "utf"
CLIENT_FOLDER = "C:/Users/wende/OneDrive/Desktop/client_folder"
def main():
""" Staring a TCP socket. """
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect((IP, PORT))
""" Folder path """
path = os.path.join(CLIENT_FOLDER, "files")
folder_name = path.split("/")[-1]
""" Sending the folder name """
msg = f"{folder_name}"
print(f"[CLIENT] Sending folder name: {folder_name}")
client.send(msg.encode(FORMAT))
""" Receiving the reply from the server """
msg = client.recv(SIZE).decode(FORMAT)
print(f"[SERVER] {msg}\n")
""" Sending files """
files = sorted(os.listdir(path))
for file_name in files:
""" Send the file name """
print(f"[CLIENT] Sending file name: {file_name}")
client.send(file_name.encode(FORMAT))
""" Recv the reply from the server """
msg = client.recv(SIZE).decode(FORMAT)
print(f"[SERVER] {msg}")
""" Send the data """
file = open(os.path.join(path, file_name), "rb")
file_data = file.read()
client.send(file_data)
print("Sending File...")
msg = client.recv(SIZE).decode(FORMAT)
print(f"[SERVER] {msg}")
if __name__ == "__main__":
main()
I have found out where the code is stuck, but I have no idea why it pauses there for no reason.
The problem I found is I keep loop the received at the server and once the received part is NULL then the loop will stop, but the problem I faced is once the received part is NULL, it cannot jump to the next if statement to check. I try transfer 1 video is no problem at all.
How can I solve this?

The solution is here: Sending a file over TCP sockets in Python
You need to put this line in the client code
client.shutdown(socket.SHUT_WR)
after
client.send(file_data)
IMO: Better to use async / await words to do your chat asynchronously. In current realization when you receive one message from a first client the second going to wait until the first will finish.
I think it will help you Simplest async/await example possible in Python

Python socket doesn't send message

I'm trying to create a python application that allows you to transfer files from the client to a server. The operation consists in sending a string to the server to indicate that we are sending a file and then the client sends the file name, its length and finally the contents of the file to the server. finally, the server confirms that the file has been saved and sends the confirmation message to the client
client.py
c.send(f"FILES:{files};{folder[index:]}".encode(FORMAT))
msg = c.recv(SIZE).decode(FORMAT)
print(f"[SERVER]: {msg}")
for i in range(len(files)):
c.send(files[i].encode(FORMAT))
print(f"[SERVER]: {c.recv(SIZE).decode(FORMAT)}") # send fileName
length = os.path.getsize(folder + "\\" + files[i])
c.send(str(length).encode(FORMAT))
print(f"[SERVER]: {c.recv(SIZE).decode(FORMAT)}") # send size
file = open(folder + "\\" + files[i], 'rb')
print(f"File read: {file.read()}")
try:
c.send(file.read()) #send bytes of file
except Exception as e:
print(f"[ERROR]: {e}")
print(f"[SERVER]: {c.recv(SIZE).decode(FORMAT)}") # complete writing on server
server.py
elif cmd == "FILES":
try:
data, path = data.split(';')
conn.send("Received files".encode(FORMAT))
while True:
nameFile = conn.recv(SIZE).decode(FORMAT)
conn.send(f"Received FileName {nameFile}".encode(FORMAT)) # received fileName
file = open(basePath + "\\" + path + "\\" + nameFile, 'wb')
print(f"[SERVER]: Opened: {file}")
length = conn.recv(SIZE).decode(FORMAT)
print(f"[CLIENT]: Length of files: {length}")
conn.send("Received size".encode(FORMAT)) # received size
bytesSend = conn.recv(length)
print(f"[CLIENT] Received bytes: {conn.recv(length)}")
file.write(bytesSend)
file.close
conn.send(f"File {nameFile} receive and saved".encode(FORMAT)) #complete writing
except:
pass
But when I try to send everything works up to c.send(file.read()). practically the client sends (but in reality it does not) the contents of the file to the server and passes to the last c.recv where it waits for the server confirmation but the server does not receive any contents of the file. Consequently, the server waits for the contents of the file to arrive but the client times out as it waits for confirmation from the server.

Python socket file transfer not working only with pdf files

I'm writing a file transfer server and client that transfers csv and pdf files, and checks a hash for integrity, and saves it to a local directory. Everything works fine on the csv, but when I try to send a pdf from client to server, the file will not be written and the server just saves the title in a 0kB pdf file. Not sure what to do from here, recv(bytes_read) keeps coming up empty.
Client send file function (Ignore the gross indentation, this is first stack overflow post)
def putFile(filename, serverIP, port, BUFFER_SIZE,s): #sends file to a server
filesize= os.path.getsize(filename)
#make and send hash
hexhash=getHash(filename)#make hash
s.send(hexhash.encode())#send hash
print("Hash sent",hexhash)
received=s.recv(BUFFER_SIZE).decode()#receive response from server
print(received)#print response
#send file
s.send(f"{filename}[SEPARATOR]{filesize}".encode())#send file name and size
with open(filename, "rb") as f: #open as read in binary to read in chunks
while True:
bytes_read = f.read(BUFFER_SIZE)#.encode() # read the bytes from the file in 4096B
if not bytes_read: # file transmitting is done
print("Sent")
#s.sendall(bytes_read)
#s.send(("").encode())
break
s.sendall(bytes_read) #sendall assures transimission in busy networks
print(bytes_read)
print("waiting")
received=s.recv(BUFFER_SIZE).decode()#receive response from server about hash
print(received)#print response
if received== "Good hash":
print("File stored")
elif received=="Bad Hash":
print("File sent does not match file received")
s.close()
return
#END SEND FILE
server function
storeFile(fileheader, cli, hexhash): #eceives a file from a client
filename, filedata=fileheader.split("[SEPARATOR]")
filename=os.path.basename(filename)#just the file name
print("File:",filename)
#filesize=int(filesize)
cli.setblocking(False)
with open(filename, "wb") as f: #save file
while True:
print("loop")
try:
print("trying")
bytes_read = cli.recv(BUFFER_SIZE)# read up to BUFFSIZE bytes from the client
print("reading:",bytes_read)
except socket.error as e:
print("except")
err = e.args[0]
if err == errno.EAGAIN or err == errno.EWOULDBLOCK:
sleep(1)
print("File Saved")
break
else:
# a "real" error occurred
print(e)
sys.exit(1)
else:
# got a message, do something :)
print("write")
f.write(bytes_read) # write to the file the bytes we just received)
#cli.close()
cli.setblocking(True)
#receive hash
check=checkHash(hexhash,filename)#check the hash given and the file hash, hash and file are given
if check: #hashes match
cli.send(("Good hash").encode())#respond to clent that hashes match
#print(received.split(SEPARATOR))
#s.close()
#END FILE RECEIPT"""
elif not check:
cli.send("Bad hash".encode())
#cli.close()
return

python socket - how to complete/close the connection on the client side?

server.py:
json files from NVD are used here
import socket, json, random, threading, zipfile, requests, re, zipfile
from bs4 import BeautifulSoup
from zipfile import *
def listen_user(user):
for x in range(2018,2021,1):
filename = "nvdcve-1.1-" + str(x) + ".json"
print(filename)
with open(filename, 'rb') as file:
sendfile = file.read()
user.sendall(sendfile)
print('file sent' + str(x))
def start_server():
while True:
user_socket, address = server.accept()
print(f"User <{address[0]}> connected!")
users.append(user_socket)
listen_accepted_user = threading.Thread(
target=listen_user,
args=(user_socket,)
)
listen_accepted_user.start()
if __name__ == '__main__':
users = []
server = socket.socket(
socket.AF_INET,
socket.SOCK_STREAM,
)
server.bind(
("127.0.0.1", 100)
)
server.listen(5)
print('waiting for connection...')
start_server()
client.py
import socket, json, random
from threading import Thread
def start_client(client):
savefilename = str(random.randint(1,10)) + 'new.json'
print(savefilename)
with client,open(savefilename,'wb') as file:
while True:
recvfile = client.recv(4096)
if not recvfile:
print('1 client')
break
file.write(recvfile)
file.close()
print('2 client')
client.close()
if __name__ == '__main__':
client = socket.socket(
socket.AF_INET,
socket.SOCK_STREAM,
)
client.connect(
("127.0.0.1", 100)
)
start_client(client)
when I send files - they are sent almost in full, but the program does not reach the line "print ('1 client')" or "print ('2 client')"
and the *new file contains all lines except a few dozen of the last
please help - how to fix the code?

recvfile = client.recv(4096) is inside the while loop and it is continuously waiting for the next bytes to receive. The client doesn't know the files are sent, so it waits for the next 4096 bytes and doesn't exit the loop.
To let the client know that the file transfer is completed, you can send a message from the server.py which you can validate in the client and break the loop as shown below.
server.py
def listen_user(user):
for x in ["f.json","g.json"]:
filename = x
print(filename)
with open(filename, 'rb') as file:
sendfile = file.read()
user.sendall(sendfile)
print('file sent' + str(x))
user.send(b"Done")
Client.py
def start_client(client):
savefilename = str(random.randint(1,10)) + 'new.json'
print(savefilename)
with client,open(savefilename,'wb') as file:
while True:
recvfile = client.recv(4096)
if recvfile.decode("utf-8") =="Done":
print('1 client')
file.close()
break
file.write(recvfile)
print('2 client')
client.close()

The call client.recv(4096) means that you are waiting for 4096 bytes to be received, then doing something with those bytes. What's likely happening in this case is that you're writing out all of the bytes, minus those that don't quite fill up the buffer at the end. This leaves the client waiting with a buffer with space that is doesn't think it is ready to write out yet.
I'm guessing that you're assuming that client.recv() will return an empty string once you've gotten all the data; this is not the case based on your code. If you want the client to be able to terminate the connection, you're going to need to send some kind of control sequence or try to otherwise assess the bytes received from the server to determined when it's time to close the connection. If you do this, you will probably want to set bufsize when calling client.recv() to 1, and instead use some other method to buffer before you write to a file.
For instance, since you're sending JSON data, you could concatenate the bytes to a variable and then repeatedly try to parse JSON. Once you have managed to successfully parse JSON, you can terminate the connection on the client side (though this would mean you have to open a new connection per file you're sending).
However, that raises the question: why do you need to close from the client side? Usually the server will just close the connection once it is done sending all of the relevant data.

Not able to receive file from client in python

I am trying to program compilation server which compiles a C program sent by client and returns an object file which can then be linked and executed at the client. Here are my client and server programs respectively
client.py:
# Compilation client program
import sys, socket, string
File = raw_input("Enter the file name:")
ssock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ssock.connect(('localhost', 5000))
csock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
csock.connect(('localhost', 5001))
f = open(File, "rb")
data = f.read()
f.close()
ssock.send(File) #send filename
ssock.send(data) #send file
fd=raw_input("Enter a key to start recieving object file:")
data=csock.recv(1024) #receive status
if data=="sucess\n":
File=File.replace(".c",".o") #objectfile name
print "Object file, "+File+", recieved sucessfully"
else:
print "There are compilation errors in " + File
File="error.txt" #errorfile name
print "Errors are reported in the file error.txt"
fobj=open(File,"wb")
while 1:
data=ssock.recv(1024) # if any error in c sourcefile then error gets
# eported in errorfile "error.txt" else objectfile is
# returned from server
if not data:break
fobj.write(data)
fobj.close()
ssock.close()
csock.close()
server.py
#Compilation Server program
import subprocess
import socket, time, string, sys, urlparse, os
ssock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ssock.bind(('', 5000))
ssock.listen(2)
print 'Server Listening on port 5000'
csock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
csock.bind(('', 5001))
csock.listen(2)
print 'Control server listening on port 5001'
client, claddr = ssock.accept()
controlsoc, caddr = csock.accept()
filename=client.recv(1024) #receive filename
print filename
############### This code is not working, i'm not getting the reason #######
############### I want to receive a file more than 1KB from client #######
f = open(filename,"wb") #receive file======
while 1:
data = client.recv(1024)
if not data: break
f.write(data)
f.close()
###############
###############
data="gcc -c " + filename + " 2> error.txt" #shell command to execute c source file
#report errors if any to error.txt
from subprocess import call
call(data,shell=True) #executes the above shell command
fil = filename.replace(".c",".o")
if (os.path.isfile(fil))== True: #test for existence of objectfile
data = "sucess\n" #no objectfile => error in compilation
filename = filename.replace(".c",".o")
else:
data = "unsucessful\n"
print data+"hi"
filename = "error.txt"
controlsoc.send(data)
f = open(filename,"rb")
data=f.read()
f.close()
print data
client.send(data)
client.close()
controlsoc.close()
I'm not able to recieve files of multiple KB. Is there any flaw in my code or how should i modify my code in order to achieve my objective of coding a compilation server.
Please help me with this regard..Thanks in advance

The problem here is you assume that ssock.send(File) will result in filename=client.recv(1024) reading exactly the filename and not more, but in fact the receiving side has no idea where the filename ends and you end up getting the file name and part of the data in the filename variable.
TCP connection is a bi-directional stream of bytes. It doesn't know about boundaries of your messages. One send might correspond to more then one recv on the other side (and the other way around). You need an application-level protocol on top of raw TCP.
The easiest in your case would be to send a text line in the form file-size file-name\n as a header. This way your server would be able to not only separate header from file data (via newline) but also know how many bytes of file content to expect, and reuse same TCP connection for multiple files.