We Keep Coding

Python doesn't release file after it is closed

What I need to do is to write some messages on a .txt file, close it and send it to a server. This happens in a infinite loop, so the code should look more or less like this:
from requests_toolbelt.multipart.encoder import MultipartEncoder
num = 0
while True:
num += 1
filename = f"example{num}.txt"
with open(filename, "w") as f:
f.write("Hello")
f.close()
mp_encoder = MultipartEncoder(
fields={
'file': ("file", open(filename, 'rb'), 'text/plain')
}
)
r = requests.post("my_url/save_file", data=mp_encoder, headers=my_headers)
time.sleep(10)
The post works if the file is created manually inside my working directory, but if I try to create it and write on it through code, I receive this response message:
500 - Internal Server Error
System.IO.IOException: Unexpected end of Stream, the content may have already been read by another component.
I don't see the file appearing in the project window of PyCharm...I even used time.sleep(10) because at first, I thought it could be a time-related problem, but I didn't solve the problem. In fact, the file appears in my working directory only when I stop the code, so it seems the file is held by the program even after I explicitly called f.close(): I know the with function should take care of closing files, but it didn't look like that so I tried to add a close() to understand if that was the problem (spoiler: it was not)

I solved the problem by using another file
with open(filename, "r") as firstfile, open("new.txt", "a+") as secondfile:
secondfile.write(firstfile.read())
with open(filename, 'w'):
pass
r = requests.post("my_url/save_file", data=mp_encoder, headers=my_headers)
if r.status_code == requests.codes.ok:
os.remove("new.txt")
else:
print("File not saved")
I make a copy of the file, empty the original file to save space and send the copy to the server (and then delete the copy). Looks like the problem was that the original file was held open by the Python logging module

Firstly, can you change open(f, 'rb') to open("example.txt", 'rb'). In open, you should be passing file name not a closed file pointer.
Also, you can use os.path.abspath to show the location to know where file is written.
import os
os.path.abspath('.')
Third point, when you are using with context manager to open a file, you don't close the file. The context manger supposed to do it.
with open("example.txt", "w") as f:
f.write("Hello")

Is possible to save a temporaly file in a Azure Function Linux Consuption Plan in Python?

first of all sorry for my English. I have an Azure Function Linux Consuption Plan using Python and I need to generate an html, transform to pdf using wkhtmltopdf and send it by email.
#generate temporally pdf
config = pdfkit.configuration(wkhtmltopdf="binary/wkhtmltopdf")
pdfkit.from_string(pdf_content, 'report.pdf',configuration=config, options={})
#read pdf and transform to Bytes
with open('report.pdf', 'rb') as f:
data = f.read()
#encode bytes
encoded = base64.b64encode(data).decode()
#Send Email
EmailSendData.sendEmail(html_content,encoded,spanish_month)
Code is running ok in my local development but when I deploy the function and execute the code I am getting an error saying:
Result: Failure Exception: OSError: wkhtmltopdf reported an error: Loading pages (1/6) [> ] 0% [======> ] 10% [==============================> ] 50% [============================================================] 100% QPainter::begin(): Returned false Error: Unable to write to destination
I think that error is reported because for any reason write permission is not available. Can you help me to solve this problem?
Thanks in advance.

The tempfile.gettempdir() method returns a temporary folder, which on Linux is /tmp. Your application can use this directory to store temporary files generated and used by your functions during execution.
So use /tmp/report.pdf as the file directory to save temporary file.
with open('/tmp/report.pdf', 'rb') as f:
data = f.read()
For more details, you could refer to this article.

Final correct code:
config = pdfkit.configuration(wkhtmltopdf="binary/wkhtmltopdf")
local_path = os.path.join(tempfile.gettempdir(), 'report.pdf')
logger.info(tempfile.gettempdir())
pdfkit.from_string(pdf_content, local_path,configuration=config, options={})

Writing to data in Python to a local file and uploading to FTP at the same time does not work

I have this weird issue with my code on Raspberry Pi 4.
from gpiozero import CPUTemperature
from datetime import datetime
import ftplib
cpu = CPUTemperature()
now = datetime.now()
time = now.strftime('%H:%M:%S')
# Save data to file
f = open('/home/pi/temp/temp.txt', 'a+')
f.write(str(time) + ' - Temperature is: ' + str(cpu.temperature) + ' C\n')
# Login and store file to FTP server
ftp = ftplib.FTP('10.0.0.2', 'username', 'pass')
ftp.cwd('AiDisk_a1/usb/temperature_logs')
ftp.storbinary('STOR temp.txt', f)
# Close file and connection
ftp.close()
f.close()
When I have this code, script doesn't write anything to the .txt file and file that is transferred to FTP server has size of 0 bytes.
When I remove this part of code, script is writing to the file just fine.
# Login and store file to FTP server
ftp = ftplib.FTP('10.0.0.2', 'username', 'pass')
ftp.cwd('AiDisk_a1/usb/temperature_logs')
ftp.storbinary('STOR temp.txt', f)
...
ftp.close()
I also tried to write some random text to the file and run the script, and the file transferred normally.
Do you have any idea, what am I missing?

After you write the file, the file pointer is at the end. So if you pass file handle to FTP, it reads nothing. Hence nothing is uploaded.
I do not have a direct explanation for the fact the local file ends up empty. But the strange way of combining "append" mode and reading may be the reason. I do not even see a+ mode defined in open function documentation.
If you want to both append data to a local file and FTP, I suggest your either:
Append the data to the file – Seek back to the original position – And upload the appended file contents.
Write the data to memory and then separately 1) dump the in-memory data to a file and 2) upload it.

Passing Binary file over HTTP POST

I have a local python file that decodes binary files. This python file first reads from the file, opens it as binary and then saves it in a buffer and interprets it. Reading it is simply:
with open(filepath, 'rb') as f:
buff = f.read()
read_all(buff)
This works fine locally. Now I'd like to setup a Azure Python job where I can send the file, approx. 100kb, over a HTTP POST and then read the interpreted meta data which my original python script does well.
I've first removed the read function so that I'll now work with the buffer only.
In my Azure Python Job I have the following, triggered by a HttpRequest
my_data = reader.read_file(req.get_body())
To test my sending I've tried the following in python
import requests
url = 'http://localhost:7071/api/HttpTrigger'
files = {'file': open('test', 'rb')}
with open('test', 'rb') as f:
buff = f.read()
r = requests.post(url, files=files) #Try using files
r = requests.post(url, data=buff) #Try using data
I've also tried in Postman adding the file to the body as a binary and setting the headers to application/octet-stream
All this doesn't send the binary file the same way as the original f.read() did. So I'm getting a wrong interpretation of the binary file.
What is file.read doing differently to how I'm sending it over as a HTTP Body message?
Printing out the first line from the local python read file gives.
b'\n\n\xfe\xfe\x00\x00\x00\x00\\\x18,A\x18\x00\x00\x00(\x00\x00\x00\x1f\x00\x00\
Whereas printing it out at the req.get_body() shows me
b'\n\n\xef\xbf\xbd\xef\xbf\xbd\x00\x00\x00\x00\\\x18,A\x18\x00\x00\x00(\x00\x00\x00\x1f\x00\
So something is clearly wrong. Any help why this could be different?
Thanks
EDIT:
I've implemented a similar function in Flask and it works well.
The code in flask is simply grabbing the file from a POST. No encoding/decoding.
if request.method == 'POST':
f = request.files['file']
#f.save(secure_filename(f.filename))
my_data = reader.read_file(f.read())
Why is the Azure Function different?

You can try UTF-16 to decode and do the further action in your code.
Here is the code for that:
with open(path_to_file,'rb') as f:
contents = f.read()
contents = contents.rstrip("\n").decode("utf-16")
Basically after doing re.get_body, perform the below operation:
contents = contents.rstrip("\n").decode("utf-16")
See if it gives you the same output as your receive in local python file.
Hope it helps.

How to download and rewrite a file in Python?

The link to the code is here (didn´t copy it here to give the guy credit):
I don´t want it to change the name with the date as is currently doing, but to download the file "finviz.csv" and rewrite it each day (with the scheduler task) to keep the data updated in my data system.
I´ve tried some tweaks, but I´m no developer I don´t have a clue how to do it.
Can you please help?

The comments in the code described it quite clearly:
# we're going to name the file by the date it was downloaded (e.g. 2012-3-18.csv)
fname = now.strftime("%Y-%m-%d")+".csv";
So just change the line to
fname = "finviz.csv";
And fix the file existence check logic:
# check if the file does not already exist
if not os.path.isfile(savepath+"/"+fname):
# open a file to save the data to ("wb" means write binary mode)
outfile = open(savepath+"/"+fname, "wb");
# download the data from the url specified above
infile = urllib2.urlopen(url);
# read the downloaded data and write it to our output file
outfile.write(infile.read());
# close the output file once we're done
outfile.close();
else:
print "'"+fname+"' ALREADY EXISTS in the save directory '"+savepath+"'.";
to:
# open a file to save the data to ("wb" means write binary mode)
outfile = open(savepath+"/"+fname, "wb");
# download the data from the url specified above
infile = urllib2.urlopen(url);
# read the downloaded data and write it to our output file
outfile.write(infile.read());
# close the output file once we're done
outfile.close();

You have to change the line
fname = now.strftime("%Y-%m-%d")+".csv";
for
fname = "finviz.csv";
And you also need to delete this if (and its corresponding else):
if not os.path.isfile(savepath+"/"+fname):