I'm appending values to a log file every 6th second. Every 30 sec I'm transferring this log to an FTP server as a file. But instead of transfering the whole file, I just want to append the collected data to the file on my server. I haven't been able to figure out how to open the server file and then append the values.
My code so far:
session = ftplib.FTP(authData[0],authData[1],authData[2])
session.cwd("//"+serverCatalog()+"//") # open server catalog
file = open(fileName(),'rb')
with open(fileName(), 'rb') as f:
f = f.readlines()
for line in f:
collected = line
# In some way open server file, write lines to it
session.storbinary('STOR ' + fileName(), open(fileName(), 'a'), 1024)
file.close()
session.quit()
Instead, do I have to download the server file open and append, then send it back?
Above was my question, the full solution is below:
session.cwd("//"+serverCatalog()+"//") # open server catalog
localfile = open("logfile.txt",'rb')
session.storbinary('APPE serverfile.txt', localfile)
localfile.close()
Use APPE instead of STOR.
Source: http://www.gossamer-threads.com/lists/python/python/22960 (link to web.archive.org)
Related
I have this weird issue with my code on Raspberry Pi 4.
from gpiozero import CPUTemperature
from datetime import datetime
import ftplib
cpu = CPUTemperature()
now = datetime.now()
time = now.strftime('%H:%M:%S')
# Save data to file
f = open('/home/pi/temp/temp.txt', 'a+')
f.write(str(time) + ' - Temperature is: ' + str(cpu.temperature) + ' C\n')
# Login and store file to FTP server
ftp = ftplib.FTP('10.0.0.2', 'username', 'pass')
ftp.cwd('AiDisk_a1/usb/temperature_logs')
ftp.storbinary('STOR temp.txt', f)
# Close file and connection
ftp.close()
f.close()
When I have this code, script doesn't write anything to the .txt file and file that is transferred to FTP server has size of 0 bytes.
When I remove this part of code, script is writing to the file just fine.
# Login and store file to FTP server
ftp = ftplib.FTP('10.0.0.2', 'username', 'pass')
ftp.cwd('AiDisk_a1/usb/temperature_logs')
ftp.storbinary('STOR temp.txt', f)
...
ftp.close()
I also tried to write some random text to the file and run the script, and the file transferred normally.
Do you have any idea, what am I missing?
After you write the file, the file pointer is at the end. So if you pass file handle to FTP, it reads nothing. Hence nothing is uploaded.
I do not have a direct explanation for the fact the local file ends up empty. But the strange way of combining "append" mode and reading may be the reason. I do not even see a+ mode defined in open function documentation.
If you want to both append data to a local file and FTP, I suggest your either:
Append the data to the file – Seek back to the original position – And upload the appended file contents.
Write the data to memory and then separately 1) dump the in-memory data to a file and 2) upload it.
I am using Paramiko to create an SFTP client to create a backup copy of a JSON file, read in the contents of the original, then update (the original). I am able to get this snippet of code to work:
# open sftp connection stuff
# read in json create backup copy - but have to 'open' twice
read_file = sftp_client.open(file_path)
settings = json.load(read_file)
read_file = sftp_client.open(file_path)
sftp_client.putfo(read_file, backup_path)
# json stuff and updating
new_settings = json.dumps(settings, indent=4, sort_keys = True)
# update remote json file
with sftp_client.open(file_path, 'w') as f:
f.write(new_settings)
However when I try to clean up the code and combine the backup file creation and JSON load:
with sftp_client.open(file_path) as f:
sftp_client.putfo(f, backup_path)
settings = json.load(f)
The backup file will be created but json.load will fail to due not having any content. And if I reverse the order, the json.load will read in the values, but the backup copy will be empty.
I'm using Python 2.7 on a Windows machine, creating a remote connection to a QNX (Linux) machine. Appreciate any help.
Thanks in advance.
If you want to read the file second time, you have to seek file read pointer back to the file beginning:
with sftp_client.open(file_path) as f:
sftp_client.putfo(f, backup_path)
f.seek(0, 0)
settings = json.load(f)
Though that is functionally equivalent to your original code with two open's.
If you aim was to optimize the code, to avoid downloading the file twice, you will have to read/cache the file to memory and then upload and load the contents from the cache.
f = BytesIO()
sftp_client.getfo(file_path, f)
f.seek(0, 0)
sftp_client.putfo(f, backup_path)
f.seek(0, 0)
settings = json.load(f)
I am using the following code to upload a SQLITE3 Database file. For some reason, the script does not completely upload the file (the uploaded filesize is less than the original)
FTP = ftplib.FTP('HOST','USERNAME','PASSWORD')
FTP.cwd('/public_html/')
FILE = 'Database.db';
FTP.storbinary("STOR " + FILE, open(FILE, 'r'))
FTP.quit()
When I go to open the uploaded file in SQLite Browser, it says it is an invalid file.
What am I doing incorrectly?
In the open() call, you need to specify that the file is a binary file, like so:
FTP.storbinary("STOR " + FILE, open(FILE, 'rb'))
I'm writing a script in python and I'm trying to wrap my head around a problem. I've a URL that when opened, downloads a document. I'm trying to write a python script that opens the https URL that downloads this document, and automatically send that document to a server I have opened using python's pysftp module.
I can't wrap my head around how to do this... Do you think I'd be able to just do:
server.put(urllib.open('https://......./document'))
EDIT:
This is the code I've tried before the above doesn't work...
download_file = urllib2.urlopen('https://somewebsite.com/file.csv')
file_contents = download_file.read().replace('"', '')
columns = [x.strip() for x in file_contents.split(',')]
# Write Downloaded File Contents To New CSV File
with open('file.csv', 'wb') as f:
writer = csv.writer(f)
writer.writerow(columns)
# Upload New File To Server
srv.put('./file.csv', './SERVERFOLDER/file.csv')
ALSO:
How would I go about getting a FILE that is ONE DAY old from the server? (Examining age of each file)... using paramiko
Good day all,
I am trying to write a python script that picks up IP address(es) from a textfile, check if the IP addresses already present in CSV file, if IP address is already present then do nothing (pass), if the IP is not present in the CSV file then append the IP address in the CSV file.
I can add the IP addresses to the CSV file but I am not able to get my head around with how to check if the IP address is already present in the file.
if os.path.isfile('IP_file.txt'):
logfile = open('IP_file.txt', 'r')
csv_file = open('csv_file.csv', 'ab')
for line in logfile:
if line in csv_file: # This is not working
pass
else:
csv_file.write(line)
csv_file.close()
logfile.close()
else:
print('No file with filename ' logfile 'created')
I also tried to use the CSV module but no luck. Any help is much appreciated.
CSV is just a text format. Read it line by line, test every line against your IP. Like
ip="192.168.1.1"
for line in csv_file:
if ip in line:
found = True
break
I am a bit concerned against your cycling over file on the disk. Probably it is better to read files into memories (find all IPs in CSV file first and put in a list) and then check against the list instead of opening and iterating over whole CSV file for every line of IP file.
Keep a list or dict that holds the unique fetched values:
found_address = []
with open('IP_file.txt', 'r') as logfile:
with open('csv_file.csv', 'ab') as csv_file:
for line in logfile:
if not line in found_address:
csv_file.write(line)
found_address.append(line)
You are opening csv_file.csv in append mode ab. And then you are trying to check line in csv_file. In append-mode file-pointer points to the end-of-file.
little crazy functional-style python:
ip_file = set(map(str.strip, open('IP_file.txt').readlines()))
if CSV file is: 192.168.1.1,192.168.1.2,192.168.1.3,...
csv_file = set(map(str.strip, open('csv_file.csv').read().split(',')))
diff_ip = ip_file - csv_file
open('csv_file.csv','a').write(''.join(map(lambda x: ',{}'.format(x), list(diff_ip))))