I am using Paramiko to create an SFTP client to create a backup copy of a JSON file, read in the contents of the original, then update (the original). I am able to get this snippet of code to work:
# open sftp connection stuff
# read in json create backup copy - but have to 'open' twice
read_file = sftp_client.open(file_path)
settings = json.load(read_file)
read_file = sftp_client.open(file_path)
sftp_client.putfo(read_file, backup_path)
# json stuff and updating
new_settings = json.dumps(settings, indent=4, sort_keys = True)
# update remote json file
with sftp_client.open(file_path, 'w') as f:
f.write(new_settings)
However when I try to clean up the code and combine the backup file creation and JSON load:
with sftp_client.open(file_path) as f:
sftp_client.putfo(f, backup_path)
settings = json.load(f)
The backup file will be created but json.load will fail to due not having any content. And if I reverse the order, the json.load will read in the values, but the backup copy will be empty.
I'm using Python 2.7 on a Windows machine, creating a remote connection to a QNX (Linux) machine. Appreciate any help.
Thanks in advance.
If you want to read the file second time, you have to seek file read pointer back to the file beginning:
with sftp_client.open(file_path) as f:
sftp_client.putfo(f, backup_path)
f.seek(0, 0)
settings = json.load(f)
Though that is functionally equivalent to your original code with two open's.
If you aim was to optimize the code, to avoid downloading the file twice, you will have to read/cache the file to memory and then upload and load the contents from the cache.
f = BytesIO()
sftp_client.getfo(file_path, f)
f.seek(0, 0)
sftp_client.putfo(f, backup_path)
f.seek(0, 0)
settings = json.load(f)
Related
from io import BytesIO
import zipfile
mem_zip = BytesIO()
with zipfile.ZipFile(mem_zip, mode="w",compression=zipfile.ZIP_DEFLATED) as zf:
zf.writestr("filename.txt", b"test")
data = mem_zip.getvalue()
with open('/path/to/file/test.zip', 'wb') as f:
f.write(data)
The code sinppet above creates a zip file in-memory with and writes to disk as a zip file. This works as expected. I can extract it and see the text file with the content "test".
I wish to password protect my zipfile. How do I do that?
I tried using the setpassword method but that had no effect on the output. The file written to disk was not password protected.
with zipfile.ZipFile(mem_zip, mode="w",compression=zipfile.ZIP_DEFLATED) as zf:
zf.setpassword(b"test_password")
zf.writestr("filename.txt", b"test")
I am writing to disk here just to test if the zipfile looks as I expect. My goal is to send the file as an email attachment and I wish to keep the zip file in-memory. So using pyminizip is not an option for me.
I'm appending values to a log file every 6th second. Every 30 sec I'm transferring this log to an FTP server as a file. But instead of transfering the whole file, I just want to append the collected data to the file on my server. I haven't been able to figure out how to open the server file and then append the values.
My code so far:
session = ftplib.FTP(authData[0],authData[1],authData[2])
session.cwd("//"+serverCatalog()+"//") # open server catalog
file = open(fileName(),'rb')
with open(fileName(), 'rb') as f:
f = f.readlines()
for line in f:
collected = line
# In some way open server file, write lines to it
session.storbinary('STOR ' + fileName(), open(fileName(), 'a'), 1024)
file.close()
session.quit()
Instead, do I have to download the server file open and append, then send it back?
Above was my question, the full solution is below:
session.cwd("//"+serverCatalog()+"//") # open server catalog
localfile = open("logfile.txt",'rb')
session.storbinary('APPE serverfile.txt', localfile)
localfile.close()
Use APPE instead of STOR.
Source: http://www.gossamer-threads.com/lists/python/python/22960 (link to web.archive.org)
I have this weird issue with my code on Raspberry Pi 4.
from gpiozero import CPUTemperature
from datetime import datetime
import ftplib
cpu = CPUTemperature()
now = datetime.now()
time = now.strftime('%H:%M:%S')
# Save data to file
f = open('/home/pi/temp/temp.txt', 'a+')
f.write(str(time) + ' - Temperature is: ' + str(cpu.temperature) + ' C\n')
# Login and store file to FTP server
ftp = ftplib.FTP('10.0.0.2', 'username', 'pass')
ftp.cwd('AiDisk_a1/usb/temperature_logs')
ftp.storbinary('STOR temp.txt', f)
# Close file and connection
ftp.close()
f.close()
When I have this code, script doesn't write anything to the .txt file and file that is transferred to FTP server has size of 0 bytes.
When I remove this part of code, script is writing to the file just fine.
# Login and store file to FTP server
ftp = ftplib.FTP('10.0.0.2', 'username', 'pass')
ftp.cwd('AiDisk_a1/usb/temperature_logs')
ftp.storbinary('STOR temp.txt', f)
...
ftp.close()
I also tried to write some random text to the file and run the script, and the file transferred normally.
Do you have any idea, what am I missing?
After you write the file, the file pointer is at the end. So if you pass file handle to FTP, it reads nothing. Hence nothing is uploaded.
I do not have a direct explanation for the fact the local file ends up empty. But the strange way of combining "append" mode and reading may be the reason. I do not even see a+ mode defined in open function documentation.
If you want to both append data to a local file and FTP, I suggest your either:
Append the data to the file – Seek back to the original position – And upload the appended file contents.
Write the data to memory and then separately 1) dump the in-memory data to a file and 2) upload it.
file_name = "r1.csv"
client = storage.Client()
bucket = client.get_bucket('upload-testing')
blob = bucket.get_blob(file_name)
blob.download_to_filename("csv_file")
Want to Open r1.csv file in read only Mode.
Getting this Error
with open(filename, 'wb') as file_obj:
Error: [Errno 30] Read-only file system: 'csv_file'
so the function download_to_filename open files in wb mode is there any way threw which i can open r1.csv in read-only mode
As mentioned in previous answer you need to use the r mode, however you don't need to specify that since that's the default mode.
In order to be able to read the file itself, you'll need to download it first, then read its content and treat the data as you want. The following example downloads the GCS file to a temporary folder, opens that downloaded object and gets all its data:
storage_client = storage.Client()
bucket = storage_client.get_bucket("<BUCKET_NAME>")
blob = bucket.blob("<CSV_NAME>")
blob.download_to_filename("/tmp/test.csv")
with open("/tmp/test.csv") as file:
data = file.read()
<TREAT_DATA_AS_YOU_WISH>
This example is thought to run inside GAE.
If you want to open a read only file you should use 'r' mode, 'wb' means write binary:
with open(filename, 'r') as file_obj:
I am having a problem when writing to a config file. I have two Python scripts that read and write to the same file. the problem is when I write to it from one script it overwrites the content from the other script.
Here is my code:
authfile = "Users/.ahs" # .ahs is a hidden file
config = ConfigParser.ConfigParser()
tmpfile = open(authfile, "w+")
config.add_section(s)
config.set(s, k, t)
config.write(tmpfile)
tmpfile.close()
w+ truncates the file when it opens, are you sure you didn't mean a or a+?
See Confused by python file mode "w+"