I've written a simple Python script that reads a file and streams the contents over a socket. The application I've connected to currently just reads the data from the socket and writes it to a file. However the data on the receiving end is incomplete. The first ~150 lines of the file do not get received, nor does the last line. I don't see anything glaringly wrong with my Python code, but if someone can point out what I've done wrong I would appreciate it. If there's an alternative method that can accomplish this task that may be helpful as well. Thanks.
EDIT: I'm pretty sure it's an issue with this code and not the receiving side because I have a C++ version of this Python code that the receiving end works fine with. However, I don't know what could be wrong here.
import socket
import sys
import time
__all__=['Stream_File_to_Socket']
def Stream_File_to_Socket(port,input_file):
# create socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = ('localhost',port)
sock.bind(server_address)
sock.listen(1)
# open file and send data
f = open(input_file,"r")
while True:
#print('waiting for a connection')
connection, client_address = sock.accept()
if connection.fileno() != -1 :
break
#print("no connection!")
return(-1)
time.sleep(0.5)
buffer_size = 1024
while True:
data = f.readline()
if not data:
connection.send(data)
break
connection.send(data)
time.sleep(0.01)
f.close()
return(0)
Related
I've been having a problem with coding the file I/O program using a socket in python. The purpose of my server program is to send a text file to the client program via socket. Even though I confirmed that a client program receives some data from a server program, the client can't write the data to the file. What is the problem??? Please help me..
[server.py]
import socketserver
from time import sleep
class TCPSocketHandler(socketserver.BaseRequestHandler):
def handle(self):
sock = self.request
while True:
with open('imageNames.txt', 'r') as file:
try:
names=file.readlines()
for name in names:
print(name)
sock.send((name.encode('utf-8')))
except Exception as exc:
print(exc)
sleep(60)
def main():
HOST, PORT = "192.168.xx.xx", 7799
server = socketserver.TCPServer((HOST, PORT), TCPSocketHandler)
server.serve_forever()
if __name__ == '__main__':
main()
[client.py]
import socket
def main():
HOST, PORT = "192.168.xx.xx", 7799
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.connect((HOST, PORT))
data = (sock.recv(1024)).decode('utf-8')
with open('imageNames2.txt', 'w') as file:
try:
while data:
print(data)
file.write(data)
data = (sock.recv(1024)).decode('utf-8')
except Exception as e:
print(e)
if __name__ == '__main__':
main()
Big Edit
After some more digging I found that what I wrote before was only partially correct. The real reason is the buffering, both in Python and on the file system level.
If a TextIO object is created, there's a buffer that accumulates input to avoid excessive system calls. Once this buffer fills up, the content will be flushed towards the file system where it might end up in another buffer until this one fills up and the content is finally written to the file.
Now, the content manager exiting, thus internally calling close() on the TextIO implicitly enforces flushing the buffers.
With the loop in the server sleeping for 60 seconds, probably both, you and I have not been patient enough to wait for the buffers to fill up to a level that triggers flushing to see content in the file on disk.
If you want to enforce every chunk that the client receives to show up in the file immediately, you'll have to explicitly flush the TextIO buffer and call os.fsync() to trigger flushing of the file system buffer as well.
So, here's how you could alter the client while keeping the server in its infinite loop over the file content:
import os
import socket
def main():
HOST, PORT = "127.0.0.1", 7799
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.connect((HOST, PORT))
data = (sock.recv(1024)).decode('utf-8')
with open('imageNames2.txt', 'w') as file:
try:
while data:
file.write(data)
file.flush()
os.fsync(file.fileno())
print(data)
data = (sock.recv(1024)).decode('utf-8')
except Exception as e:
print(e)
if __name__ == '__main__':
main()
Please note that in some IDEs, such as PyCharm, the new file might not show up in the project overview until the file has been closed. On the file system level, however, it does exist.
Also, my earlier side note about losing the first line was wrong. Sorry about that.
I'm trying to send a file over a socket in Python 2.7.x . My client and server code is below. It's working, but only after the client connection kills the socket. So I added raw_input('finished') to the end of both for debugging.
So if I start the server, then run the client... It looks like all but the last bit of the file sends, until I forcefully kill the client and then it's all there. So the problem is definitely in the server loop... I just don't know how to fix it. if not data: break isn't being triggered. But, if I do something like if len(data) < 1024: break it won't work for bigger files.
Any help is appreciated!
# client.py
import socket
conn = socket.socket()
conn.connect(('localhost', 1337))
f = open('test.jpg', 'rb')
data = f.read(1024)
while data:
conn.send(data)
data = f.read(1024)
f.close()
raw_input('finished')
# server.py
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('localhost', 1337))
s.listen(5)
conn, addr = s.accept()
f = open('test.jpg', 'wb')
while True:
data = conn.recv(1024)
if not data:
break
f.write(data)
f.close()
raw_input('finished')
From your posted code:
while data:
conn.send(data)
data = f.read(1024)
From the Python socket documentation:
socket.send(string[, flags])
[...]
Returns the number of bytes sent. Applications are responsible for checking
that all data has been sent; if only some of the data was transmitted, the
application needs to attempt delivery of the remaining data.
That should tell you what the problem is, but just to be explicit about it: send() may or may not accept all of the bytes you asked it to send before returning, and it's up to you to handle it correctly in the case where it only accepts the first (n) bytes rather than the entire buffer. If you don't check send()'s return value, then you will sometimes drop some of the bytes of your file without knowing it. You need to check send()'s return value and if it is less than len(data), call send() again (as many times as necessary) with the remaining bytes. Alternatively you could call conn.sendall() instead of conn.send(), since sendall() will perform that logic for you.
Im having trouble getting my very basic and simple TCP Server to properly work with http requests. This is what I have so far
from socket import *
import sys
serverSocket = socket(AF_INET, SOCK_STREAM)
serverSocket.bind(('', 4567))
serverSocket.listen(1)
while True:
print('Ready to serve...')
connectionSocket, addr = serverSocket.accept()
print("connected from: ", addr)
try:
message = connectionSocket.recv(1024)
filename = message.split()[1]
f = open(filename[1:])
outputdata = f.read()
connectionSocket.send("HTTP/1.1 200 OK\r\n")
for i in range(0, len(outputdata)):
connectionSocket.send(outputdata[i].encode())
connectionSocket.send("\r\n".encode())
connectionSocket.close()
except IOError:
connectionSocket.send("file not found")
serverSocket.close()
sys.exit()
The error comes from the open statement. I dont fully understand how this line of code's return value is organized.
message = connectionSocket.recv(1024)
I know that the return value is in bytes but when I try to use a fuction to turn it into a string like decode() i get errors as well
I have the .py file and the html file sitting in the same directory on my local machine and the way I test this is I just run this and open up a browser and type in
http://127.0.0.1:4567/helloworld.html
My code then promptly crashes after receiving the HTTP request.
Any and all help will be greatly appreciated!
There are numerous problems with your code and since you don't state what specific issues you are concerned about, here is what I see:
connectionSocket.send(outputdata[i].encode())
connectionSocket.send("\r\n".encode())
That appears to send a newline after every character you send back to the client.
Also, it doesn't deal with the client disconnecting because you're sending back invalid data.
Even if what you were trying to do didn't have these errors in it, you don't appear to be attempting to send back a valid http response.
https://www.w3.org/Protocols/rfc2616/rfc2616-sec6.html
This is my code:
socketcheck.py
import time
import subprocess
subprocess.Popen(["python", "server.py"])
for i in range(10):
time.sleep(2)
print i
def print_from_server(data):
print data
server.py
import socket
from socketcheck import print_from_server
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('localhost',3005))
client_connected = 1
while 1:
s.listen(1)
conn, addr = s.accept()
data = conn.recv(1024)
if data:
client_connected = 0
else: break
if client_connected == 0:
print 'data received'
print_from_server(data)
client_connected = 1
conn.sendall(data)
client.py
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('localhost',3005))
s.sendall('Hello, world')
data = s.recv(1024)
#s.close()
print 'Received', repr(data)
What I am trying to do here is, run socketcheck.py which runs server.py in background and listens for a client connection. So whatever data the client sends, I want to pass it on to socketcheck.py. Is this valid? If so, then how do I achieve it?
Now when I try and run socketcheck.py, the for loop is running indefinitely.
Thanks :)
EDIT:
This initially I tried as a single program, but until the client gets connected, the rest of the program doesn't execute(blocking), with the setblocking(0) the program flow wouldn't stop but when the client connects to server it doesn't print(do anything). The server code looked something like this:
import socket
from socketcheck import print_from_server
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('localhost',3005))
s.setblocking(0)
while 1:
try:
s.listen(1)
conn, addr = s.accept()
conn.setblocking(0)
data = conn.recv(1024)
if not data: break
print 'data received'
conn.sendall(data)
except:
print 'non blocking'
print 'the lengthy program continues from here'
The reason why your program crashes your computer is simple:
You have a while loop which calls print_from_server(data), and each time it calls it, a new subprocess gets created via subprocess.Popen(["python", "server.py"]).
The reason for creating a new popen each time is a bit more complicated: You open a new server.py program in socketcheck.py. If this new server.py program calls print_from_server(data), this is the first time print_from_server(data) gets called (for the new server.py program). So the global commands (such as popen) are executed, since they are always executed once.
The number of processes running will explode quickly and you computer crashes.
One additional remark: You cannot print to console with a print command in a subprocess, since there is no console attached to that subprocess, you can only print to file. If you do that, you'll see that this output explodes quickly from all the processes.
Put socketcheck.py and server.py into one program and everything works fine, or explain why you need two programs.
The functionality can be easily achieved with multithreading :)
This is the client and server program where a client sends a file to server to save in the server. There is a issuse in that same file name is not getting copied on the server with same file size
Please help me in this
Client program
import socket
import sys
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("localhost",9999))
path=raw_input("Please enter the complete PATH of your file : ")
f=open (path, "rb")
l = f.read(256)
while (l):
s.sendall(l)
l = f.read(10000)
s.close()
Server Program
import socket
import sys
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(("localhost",9999))
s.listen(10)
while True:
s, address = s.accept()
print address
i=1
f = open( str(i),'wb') #open in binary
#i=i+1
while (True):
l=s.recv(256)
#while (l):
f.write(l)
l=s.recv(256)
print 'File recieve succesfully'
f.close()
#sc.close()
s.close()
Thanks in advance
Start by walking through the code and thinking about what the client knows about the data it is sending and what the server knows about the data it is receiving. You will have to send 2 types of messages: the data and the filename. How you do that is up to you.
Without over-thinking it, maybe try writing the filename first (followed by a newline or special character) then send the file data. On the server side accept the connection, read in data until you find a newline character (that's the filename), then receive the rest of the data and write it to the file.
Also, the server code you've provided doesn't work, at least I don't think, since you never break out of your while True loops.