Usually, when I want to transfer a web server text file to client, here is what I did
import cgi
print "Content-Type: text/plain"
print "Content-Disposition: attachment; filename=TEST.txt"
print
filename = "C:\\TEST.TXT"
f = open(filename, 'r')
for line in f:
print line
Works very fine for ANSI file. However, say, I have a binary file a.exe (This file is in web server secret path, and user shall not have direct access to that directory path). I wish to use the similar method to transfer. How I can do so?
What content-type I should use?
Using print seems to have corrupted content received at client side. What is the correct method?
I use the following code.
#!c:/Python27/python.exe -u
import cgi
print "Content-Type: application/octet-stream"
print "Content-Disposition: attachment; filename=jstock.exe"
print
filename = "C:\\jstock.exe"
f = open(filename, 'rb')
for line in f:
print line
However, when I compare the downloaded file with original file, it seems there is an extra whitespace (or more) for after every single line.
Agree with the above posters about 'rb' and Content-Type headers.
Additionally:
for line in f:
print line
This might be a problem when encountering \n or \r\n bytes in the binary file. It might be better to do something like this:
import sys
while True:
data = f.read(4096)
sys.stdout.write(data)
if not data:
break
Assuming this is running on windows in a CGI environment, you will want to start the python process with the -u argument, this will ensure stdout isn't in text-mode
When opening a file, you can use open(filename, 'rb') - the 'b' flag marks it as binary. For a general handler, you could use some form of mime magic (I'm not familiar with using it from Python, I've only ever used it from PHP a couple of years ago). For the specific case, .exe is application/octet-stream.
Content-type of .exe is tipically application/octet-stream.
You might want to read your file using open(filename, 'rb') where b means binary.
To avoid the whitespace problem, you could try with:
sys.stdout.write(open(filename,"rb").read())
sys.stdout.flush()
or even better, depending on the size of your file, use the Knio approach:
fo = open(filename, "rb")
while True:
buffer = fo.read(4096)
if buffer:
sys.stdout.write(buffer)
else:
break
fo.close()
For anyone using Windows Server 2008 or 2012 and Python 3, here's an update...
After many hours of experimentation I have found the following to work reliably:
import io
with io.open(sys.stdout.fileno(),"wb") as fout:
with open(filename,"rb") as fin:
while True:
data = fin.read(4096)
fout.write(data)
if not data:
break
Related
What I need to do is to write some messages on a .txt file, close it and send it to a server. This happens in a infinite loop, so the code should look more or less like this:
from requests_toolbelt.multipart.encoder import MultipartEncoder
num = 0
while True:
num += 1
filename = f"example{num}.txt"
with open(filename, "w") as f:
f.write("Hello")
f.close()
mp_encoder = MultipartEncoder(
fields={
'file': ("file", open(filename, 'rb'), 'text/plain')
}
)
r = requests.post("my_url/save_file", data=mp_encoder, headers=my_headers)
time.sleep(10)
The post works if the file is created manually inside my working directory, but if I try to create it and write on it through code, I receive this response message:
500 - Internal Server Error
System.IO.IOException: Unexpected end of Stream, the content may have already been read by another component.
I don't see the file appearing in the project window of PyCharm...I even used time.sleep(10) because at first, I thought it could be a time-related problem, but I didn't solve the problem. In fact, the file appears in my working directory only when I stop the code, so it seems the file is held by the program even after I explicitly called f.close(): I know the with function should take care of closing files, but it didn't look like that so I tried to add a close() to understand if that was the problem (spoiler: it was not)
I solved the problem by using another file
with open(filename, "r") as firstfile, open("new.txt", "a+") as secondfile:
secondfile.write(firstfile.read())
with open(filename, 'w'):
pass
r = requests.post("my_url/save_file", data=mp_encoder, headers=my_headers)
if r.status_code == requests.codes.ok:
os.remove("new.txt")
else:
print("File not saved")
I make a copy of the file, empty the original file to save space and send the copy to the server (and then delete the copy). Looks like the problem was that the original file was held open by the Python logging module
Firstly, can you change open(f, 'rb') to open("example.txt", 'rb'). In open, you should be passing file name not a closed file pointer.
Also, you can use os.path.abspath to show the location to know where file is written.
import os
os.path.abspath('.')
Third point, when you are using with context manager to open a file, you don't close the file. The context manger supposed to do it.
with open("example.txt", "w") as f:
f.write("Hello")
My client requests a page from a server written in python 3.
The server return an html page that is presented by client.
Therefore, I did a dummy.html page and when client asks for it, my python reads it and returns it to the client:
filename = "dummy.html"
fh = open(filename, 'rt')
line = fh.readline()
while line:
print(line)
line = fh.readline()
fh.close()
However, this code does not read the <!DOCTYPE html> that is placed in the top of my dummy.html file (and thus, things like bootstrap don't work for me...).
I also tried printing it manually print('<!DOCTYPE html>') but that also does not work.
print('<!DOCTYPE html>') <---- IT IS PRINTED TO SDOUT BUT WHEN PRINTED TO CLIENT, THE PAGE DOES NOT HAVE THIS LINE ....
filename = CURRENTPATH+"\\..\\su.html"
fh = open(filename, 'rt')
line = fh.readline()
print('hello')
print('<'+'!'+'DOCTYPE html>')
while line:
print(line)
line = fh.readline()
fh.close()
How can I fix it?
It looks like you're trying to reimplement a web server in Python. Please consider using an existing web framework, such as Django (https://www.djangoproject.com/), Flask (http://flask.pocoo.org/) or Pyramid (http://www.pylonsproject.org/), which will do most of the work for you (including built-in support for a wide variety of HTML templating libraries, and actual performance).
As for your actual answer, a bare print statement prints to stdout, as expected. You need, instead, to write to the file-like object whose contents will be sent to the client (is it a socket? a file? who knows? stop reinventing the wheel).
I have a script that regularly reads a text file on a server and over writes a copy of the text to a local copy of the text file. I have an issue of the process adding extra carriage returns and an extra invisible character after the last character. How do I make an identical copy of the server file?
I use the following to read the file
for link in links:
try:
f = urllib.urlopen(link)
myfile = f.read()
except IOError:
pass
and to write it to the local file
f = open("C:\\localfile.txt", "w")
try:
f.write(myfile)
except NameError:
pass
finally:
f.close()
This is how the file looks on the server
!http://i.imgur.com/rAnUqmJ.jpg
and this is how the file looks locally. Besides, an additional invisible character after the last 75
!http://i.imgur.com/xfs3E8D.jpg
I have seen quite a few similar questions, but not sure how to handle the urllib to read in binary
Any solution please?
If you want to copy a remote file denoted by a URL to a local file i would use urllib.urlretrieve:
import urllib
urllib.urlretrieve("http://anysite.co/foo.gz", "foo.gz")
I think urllib is reading binary.
Try changing
f = open("C:\\localfile.txt", "w")
to
f = open("C:\\localfile.txt", "wb")
I'm coding a small website with Python and CGI where users can upload zip files and download files uploaded by other users.
Currently I'm able to upload correctly the zip's, but I'm having some trouble to correctly send files to the user. My first approach was:
file = open('../../data/code/' + filename + '.zip','rb')
print("Content-type: application/octet-stream")
print("Content-Disposition: filename=%s.zip" %(filename))
print(file.read())
file.close()
But soon I realized that I had to send the file as binary, so I tried:
print("Content-type: application/octet-stream")
print("Content-Disposition: filename=%s.zip" %(filename))
print('Content-transfer-encoding: base64\r')
print( base64.b64encode(file.read()).decode(encoding='UTF-8') )
And different variants of it. It just doesn't works; Apache raises "malformed header from script" error, so I guess I should encode the file in some other way.
You need to print an empty line after the headers, and you Content-disposition header is missing the type (attachment):
print("Content-type: application/octet-stream")
print("Content-Disposition: attachment; filename=%s.zip" %(filename))
print()
You may also want to use a more efficient method of uploading the resulting file; use shutil.copyfileobj() to copy the data to sys.stdout.buffer:
from shutil import copyfileobj
import sys
print("Content-type: application/octet-stream")
print("Content-Disposition: attachment; filename=%s.zip" %(filename))
print()
with open('../../data/code/' + filename + '.zip','rb') as zipfile:
copyfileobj(zipfile, sys.stdout.buffer)
You should not use print() for binary data in any case; all you get is b'...' byte literal syntax. The sys.stdout.buffer object is the underlying binary I/O buffer, copy binary data directly to that.
The header is malformed because, for some reason, Python sends it after sending the file.
What you need to do is flush stdout right after the header:
sys.stdout.flush()
Then put the file copy
This is what worked for me, I am running Apache2 and loading this script via cgi. Python 3 is my language.
You may have to replace first line with your python 3 bin path.
#!/usr/bin/python3
import cgitb
import cgi
from zipfile import ZipFile
import sys
# Files to include in archive
source_file = ["/tmp/file1.txt", "/tmp/file2.txt"]
# Name and Path to our zip file.
zip_name = "zipfiles.zip"
zip_path = "/tmp/{}".format(zip_name)
with ZipFile( zip_path,'w' ) as zipFile:
for f in source_file:
zipFile.write(f);
# Closing File.
zipFile.close()
# Setting Proper Header.
print ( 'Content-Type:application/octet-stream; name="{}"'.format(zip_name) );
print ( 'Content-Disposition:attachment; filename="{}"\r\n'.format(zip_name) );
# Flushing Out stdout.
sys.stdout.flush()
bstdout = open(sys.stdout.fileno(), 'wb', closefd=False)
file = open(zip_path,'rb')
bstdout.write(file.read())
bstdout.flush()
Im writing a python scrpit to upload files to my file server:
host = "myhost.dev"
tn = telnetlib.Telnet()
tn.open(host, 5202)
print tn.read_until("\n")
fp = "./output"
f = open(fp, "r")
f_body = f.read()
tn.write(f_body)
tn.write("\n")
f.close()
If file has a new line character- 0a, and it is part of a binary data in gzip file, what should I do to escape it ? Can python telnetlib do it by itself ? or should I do it ?
best regards
I think that telnet is not the best option for transfering files, but if you still want to use it for uploading files. You may try to do the following (haven't tried, but I think should work)
#On client side
...
import base64
with open('test.gz','rb') as f:
content = f.read()
content_serialized = base64.b64encode(content)+'\n'
...
#On server side
...
import base64
content = base64.b64decode(content_serialized.rstrip('\n'))
with open('test.gz','wb') as f:
f.write(content)
...
Maybe telnet is not the best solution for this, it is better to use FTP o HTTP.
The telnet protocol is not suitable for transmiting files, and it has some control flow processes that make if dificult to transmit files and special chars.
If you want to use a non standard protocol, it is better so use the socket module, with sockets you don't have this problems with 0a.