We Keep Coding

Writing to data in Python to a local file and uploading to FTP at the same time does not work

I have this weird issue with my code on Raspberry Pi 4.
from gpiozero import CPUTemperature
from datetime import datetime
import ftplib
cpu = CPUTemperature()
now = datetime.now()
time = now.strftime('%H:%M:%S')
# Save data to file
f = open('/home/pi/temp/temp.txt', 'a+')
f.write(str(time) + ' - Temperature is: ' + str(cpu.temperature) + ' C\n')
# Login and store file to FTP server
ftp = ftplib.FTP('10.0.0.2', 'username', 'pass')
ftp.cwd('AiDisk_a1/usb/temperature_logs')
ftp.storbinary('STOR temp.txt', f)
# Close file and connection
ftp.close()
f.close()
When I have this code, script doesn't write anything to the .txt file and file that is transferred to FTP server has size of 0 bytes.
When I remove this part of code, script is writing to the file just fine.
# Login and store file to FTP server
ftp = ftplib.FTP('10.0.0.2', 'username', 'pass')
ftp.cwd('AiDisk_a1/usb/temperature_logs')
ftp.storbinary('STOR temp.txt', f)
...
ftp.close()
I also tried to write some random text to the file and run the script, and the file transferred normally.
Do you have any idea, what am I missing?

After you write the file, the file pointer is at the end. So if you pass file handle to FTP, it reads nothing. Hence nothing is uploaded.
I do not have a direct explanation for the fact the local file ends up empty. But the strange way of combining "append" mode and reading may be the reason. I do not even see a+ mode defined in open function documentation.
If you want to both append data to a local file and FTP, I suggest your either:
Append the data to the file – Seek back to the original position – And upload the appended file contents.
Write the data to memory and then separately 1) dump the in-memory data to a file and 2) upload it.

python stream data between sources

I have a large file in my FTP, and I need to write it to google cloud storage.
My first thought would be download it from ftp to local file, and then transmit local file to remote storage. But I would prefer to do it without downloading it localy.
So far I came up with the following code:
from ftplib import FTP
import io
ftp = ftp = FTP('example.com')
ftp.voidcmd('TYPE I')
sock = ftp.transfercmd('RETR file.csv')
raw = io.BytesIO()
file = io.BufferedRandom(raw)
blob = bucket.blob('blobname.csv', chunk_size=262144) # gcs blob
blob.upload_from_file(file, content_type='text/csv', rewind=True)
But I get:
Traceback (most recent call last):
File "/home/tsh/example.py", line 65, in <module>
file = io.BufferedRandom(raw)
io.UnsupportedOperation: File or stream is not seekable.
Is there a way to pipe data from ftp to cloud storage (or any other remote resource) without downloading it to local machine? I am using python 3.6.

I think you can achieve what you want, without python, using rclone. If you must use python, maybe they provide a wrapper or you could use subprocess module.

Upload FTP file wont change on server

I´m using cron in a Raspberry Pi (Raspbian) to run a script in Python to upload a file every x minutes to my server, this file is created every x minutes from another script (which is loaded from cron every x-1 minutes). When running manually the upload.py it works well and the file is uploaded correctly, but when using cron to run upload.py, the uploaded file is always the same even if i delete it from the server.
Example:
-File in Raspberry Pi is 40kb or 100kb or 200kb.
-The file in Server is always 40kb. If i delete it from the server, the next time upload.py runs from cron, it will appear the same 40kb file even if the file in the Raspberry Pi is 100kb or 200kb.
Create file .py
import os
import shutil
with open("temp.dat", 'r') as fsrc:
with open("up.dat", 'w+') as fdest:
fsrc.seek(-324279, os.SEEK_END)
shutil.copyfileobj(fsrc, fdest)
fsrc.close()
fdest.close()
Upload .py
from ftplib import FTP
HOST = 'host.com'
FTP_NAME = 'username'
FTP_PASS = 'password'
ftp = FTP(HOST)
ftp.login(FTP_NAME, FTP_PASS)
file = open('up.dat', 'r')
ftp.storlines('STOR up.dat', file)
ftp.quit()
file.close()
Cant understand why this happen. Any help is welcome.
Thanks.

PIL.Image.save() to an FTP server

Right now, I have the following code:
pilimg = PILImage.open(img_file_tmp) # img_file_tmp just contains the image to read
pilimg.thumbnail((200,200), PILImage.ANTIALIAS)
pilimg.save(fn, 'PNG') # fn is a filename
This works just fine for saving to a local file pointed to by fn. However, what I would want this to do instead is to save the file on a remote FTP server.
What is the easiest way to achieve this?

Python's ftplib library can initiate an FTP transfer, but PIL cannot write directly to an FTP server.
What you can do is write the result to a file and then upload it to the FTP server using the FTP library. There are complete examples of how to connect in the ftplib manual so I'll focus just on the sending part:
# (assumes you already created an instance of FTP
# as "ftp", and already logged in)
f = open(fn, 'r')
ftp.storbinary("STOR remote_filename.png", f)
If you have enough memory for the compressed image data, you can avoid the intermediate file by having PIL write to a StringIO, and then passing that object into the FTP library:
import StringIO
f = StringIO()
image.save(f, 'PNG')
f.seek(0) # return the StringIO's file pointer to the beginning of the file
# again this assumes you already connected and logged in
ftp.storbinary("STOR remote_filename.png", f)

Can I upload an object in memory to FTP using Python?

Here's what I'm doing now:
mysock = urllib.urlopen('http://localhost/image.jpg')
fileToSave = mysock.read()
oFile = open(r"C:\image.jpg",'wb')
oFile.write(fileToSave)
oFile.close
f=file('image.jpg','rb')
ftp.storbinary('STOR '+os.path.basename('image.jpg'),f)
os.remove('image.jpg')
Writing files to disk and then imediately deleting them seems like extra work on the system that should be avoided. Can I upload an object in memory to FTP using Python?

Because of duck-typing, the file object (f in your code) only needs to support the .read(blocksize) call to work with storbinary. When faced with questions like this, I go to the source, in this case lib/python2.6/ftplib.py:
def storbinary(self, cmd, fp, blocksize=8192, callback=None):
"""Store a file in binary mode. A new port is created for you.
Args:
cmd: A STOR command.
fp: A file-like object with a read(num_bytes) method.
blocksize: The maximum data size to read from fp and send over
the connection at once. [default: 8192]
callback: An optional single parameter callable that is called on
on each block of data after it is sent. [default: None]
Returns:
The response code.
"""
self.voidcmd('TYPE I')
conn = self.transfercmd(cmd)
while 1:
buf = fp.read(blocksize)
if not buf: break
conn.sendall(buf)
if callback: callback(buf)
conn.close()
return self.voidresp()
As commented, it only wants a file-like object, indeed it not even be particularly file-like, it just needs read(n). StringIO provides such "memory file" services.

import urllib
import ftplib
ftp = ftplib.FTP(...)
f = urllib.urlopen('http://localhost/image.jpg')
ftp.storbinary('STOR image.jpg', f)

You can use any in-memory file-like object, like BytesIO:
from io import BytesIO
It works both in binary mode with FTP.storbinary:
f = BytesIO(b"the contents")
ftp.storbinary("STOR /path/file.txt", f)
as well as in ascii/text mode with FTP.storlines:
f = BytesIO(b"the contents")
ftp.storlines("STOR /path/file.txt", f)
For more advanced examples, see:
Python - Upload a in-memory file (generated by API calls) in FTP by chunks
Python - Transfer a file from HTTP(S) URL to FTP/Dropbox without disk writing (chunked upload)
How to send CSV file directly to an FTP server