I am performing a daily database dump in a python script and I am looking for the most python efficient way to delete the current file only after the recent one was written successfully, i.e. delete the backup-12-11-2019.sql only after backup-13-11-2019 was created successfully
you can use try:...except:...esle: as the following code.
import datetime
import os
now = datetime.datetime.utcnow()
try:
pg_dumpall('-h', DB_HOST, '-U', DB_USERNAME, '-p', DB_PORT, _out=BACKUP_FOLDER + 'backup-' + str(now.day) + '-' + str(now.month) + '-' + str(now.year) + '.sql')
except Exception as e:
print(e)
else:
previous_day = now - datetime.timedelta(days=1)
os.remove(BACKUP_FOLDER + 'backup-' + str(now.day - 1) + '-' + str(now.month) + '-' + str(now.year) + '.sql')
If the pg_dumpall does not raise and Exception it will delete the previous backup file
Best regard
From DBA StackExchange
pg_dumpall -U pg_user > /tmp/tmp.txt
DUMP_STATUS=$?
echo "Dump status: $DUMP_STATUS" > /tmp/status.txt
And SO: Get return value
subprocess.check_output([SCRIPT, "-d", date], shell=True).
We're able to come up with something that will run the command you want to run, and check its return value at the same time.
output = subprocess.check_output(['pg_dumpall', '-h', DB_HOST, '-U', DB_USERNAME, '-p', DB_PORT], stdout=outfile)
if output == 0:
print("Success")
os.remove(oldfile)
else:
print("Failure")
Related
So I need to have a script that will execute the nsupdate command, to add a record the the ddns database:
#!/usr/local/bin/python3.7
import sys
import os
import time
import subprocess
try:
hostname = sys.argv[1]
IP = sys.argv[2]
except Exception as e:
print('Error: Please enter a number\n')
zone = hostname[-5:]
update = ''
if zone == 'zone1':
print('\nThis will be added to zone1\n')
#add to zone 1
update = 'update add ' + hostname + ' 86400' + ' A ' + IP
os.system('nsupdate -k zone1.key')
update = 'update add ' + hostname + ' 86400' + ' A ' + IP
if zone == 'zone2':
print('This will be added to zone2')
#add to zone 2
os.system('nsupdate -k zone2.key')
#if statement to check IP address is in range
#update = 'update add ' + hostname + ' 86400' + ' A ' + IP
time.sleep(2)
os.system(update)
time.sleep(2)
os.system('send')
time.sleep(2)
os.system('quit')
#time.sleep(3)
print('\nHostname:')
print(hostname)
print('\nIP address: ')
print(IP)
print('\nZone: ')
print (zone)
Above is the code I'm currently working with, please ignore it if its badly written, dont write much python.
I find it executes the first os.system fine, but will not run the next three.
I believe that's because the command line changes from # to > but have got no clue how to still have it execute those commands.
Have tried os.system and subprocess.run/call
even tried call it all in one os.system by going os.system('nsupdate; {update}; send; quit').
Any help would be great.
so i am working on a python script as my school project, which tries to run exe's one by one. and if any exe is un openable then it output's "Unable to open" and if it's the opposite , then shows the opposite.
Now the problem is, it shows un openable even if the script was able to open/run exe, and the only time it shows opened, is when i deliberately end/terminal the exe after some time (shows unopenable if i end it immediately). And it doesn't start/run other exe's if i dont terminate the opened exe file.
SO what i want is , if the script is successful in opening a exe, then the script should output as 'opened' and shouldn't wait for me to terminate it.
here's the problematic code`
total = len(glob.glob("*.exe"))
notopened = 0
opened = 0
current = 0
print("opening exe's")
for fn in glob.glob("*.exe"):
try:
current += 1
p = subprocess.Popen([fn])
result = p.wait()
if result == 1:
opened += 1
print("[" + ("0" if current < 10 else "") + str(current) + "/" + str(total) + "]" + fn + " opened [" + str(round(((total - opened)/total) * 100, 2)) + "%]")
else:
notopened += 1
print("[" + ("0" if current < 10 else "") + str(current) + "/" + str(total) + "]" + fn + " notopened [" + str(round(((total - opened)/total) * 100, 2)) + "%]")
except:
notopened += 1
print("[" + str(current) + "/" + str(total) + "] " + fn + " notopened [" + str(round(((total - opened)/total) * 100, 2)) + "%]")
pass
The result from wait is an error code; zero indicates success. You are basically inverting the logic, and calling the successes failures.
Furthermore, though, even if the result code indicates a failure, it means that the process actually ran. For example, on Unix-like systems, the false command will report failure when you run it, by design; the absence of a result code is what indicates that running the command failed. In Python, this will always result in an exception.
As an aside, you should probably be avoiding Popen; like the subprocess documentation says, you should prefer run or one of the other higher-level functions in every situation where you can avoid raw Popen.
As a further aside, the print is basically identical in all cases, and should simply be refactored to execute outside the condition. I also cleaned it up a bit. You never use notopened so I took that out (it should be the difference between current and opened anyway).
files = glob.glob("*.exe")
total = len(files)
for current, fn in enumerate(files, 1):
try:
subprocess.run([fn])
opened += 1
whether = ""
except Exception:
whether = "not "
print("[%02i/%i] %s %sopened [%1.2f%%]" % (
current, total, fn, whether, 100*(total-opened)/total))
Anyway, randomly opening executables and waiting for them to finish is extremely risky. What if the executable formats the disk, shuts down the computer, or removes all files? You should probably be rethinking your approach.
You can start multiple processes using ThreadPoolExecutor.
Here is a possible code that could help.
Note that I used logger here (you need to configure it to stream to console, by the way), since logging is thread-safe, unlike print.
Note that this code does not print "opened" and i don't know how to do that on a successful opening of the process.
import glob
import logging
import subprocess
from concurrent.futures import ThreadPoolExecutor, as_completed
logger = logging.getLogger(__name__)
def run_multiple_files(files):
with ThreadPoolExecutor(max_workers=8) as executor:
future_exec = {}
for exe_file in files:
future = executor.submit(run_one_file, exe_file)
future_exec[future] = exe_file
for future in as_completed(future_exec):
exe_file = future_exec[future]
try:
future.result()
except Exception as exc:
logger.error(f'{exe_file} generated an exception: {exc}')
def run_one_file(exe_file):
try:
subprocess.run(
[exe_file],
stdout=subprocess.DEVNULL,
stderr=subprocess.STDOUT,
check=True,
)
except subprocess.CalledProcessError:
msg = f'Failed {exe_file}'
logger.error(msg)
raise subprocess.CalledProcessError(msg)
else:
msg = f'Finished: {exe_file}'
logger.info(msg)
if __name__ == '__main__':
run_multiple_files(glob.glob('*.exe'))
I have the below Python script and it works very well, but I would like to introduce some fail safe options .. That fail safe options being ..
1) if I cannot find (in this example) Michael I would like to write to the file ERROR ..
2) If the database does not allow me to connect for whatever reason I would like to write to another file CONNECTION_ERROR
Here is my script:
#! /usr/bin/python
import pymssql
import sys
sys.path.insert(0, '/opt/mount/safe')
from secrets import password
conn = pymssql.connect(
server="server",
port=port,
user="user",
password=password,
database="database")
conn
cursor = conn.cursor()
cursor.execute("SELECT name, address FROM database WHERE name = 'michael'")
with open('safe/file.txt', 'w') as f:
for row in cursor.fetchall():
print ( "Person " + (row[0])),
print ( "has this address " + (row[1]))
f.write(str( "Person " + (row[0])))
f.write("%s\n" % str( " has this address " + (row[1])))
conn.close()
Took me a while but the below works really really well
import sys, pymssql, string, os, calendar, datetime, traceback, socket, platform
try:
d = datetime.datetime.now()
log = open("LogFile.txt","a")
log.write("----------------------------" + "\n")
log.write("----------------------------" + "\n")
log.write("Log: " + str(d) + "\n")
log.write("\n")
# Start process...
starttime = datetime.datetime.now()
log.write("Begin process:\n")
log.write(" Process started at "
+ str(starttime) + "\n")
log.write("\n")
xxxxxx
CODE HERE
XXXXXX
endtime = datetime.datetime.now()
# Process Completed...
log.write(" Completed successfully in "
+ str(endtime - starttime) + "\n")
log.write("\n")
log.close()
except:
tb = sys.exc_info()[2]
tbinfo = traceback.format_tb(tb)[0]
# Concatenate information together concerning
# the error into a message string
pymsg = "PYTHON ERRORS:\nTraceback info:\n" + tbinfo + "\nError Info:\n" + str(sys.exc_info())
# Return python error messages for use in
# script tool or Python Window
log.write("" + pymsg + "\n")
log.close()
I have a configuration file I'm reading with ConfigParser.
One of the values is a file path containing spaces in the path:
[Configuration]
mysqldumpexecutable_location = C:/Program Files (x86)/MySQL/MySQL Server 5.7/bin/
It seems that the value returned with get() for this parameter is only "C:/Program" - up to the space.
Here's the relevant code:
import os
import time
import datetime
import configparser
def BackupDB():
try:
config = configparser.RawConfigParser()
config.read(r"etc\configuration.txt")
DB_HOST = config.get('Configuration', 'db_host')
DB_USER = config.get('Configuration', 'db_username')
DB_USER_PASSWORD = config.get('Configuration', 'db_password')
#DB_NAME = '/backup/dbnames.txt'
DB_NAME = config.get('Configuration', 'db_schema')
BACKUP_PATH = config.get('Configuration', 'backup_path')
MYSQLDUMP_LOCATION = config.get('Configuration', 'mysqldumpexecutable_location')
DATETIME = time.strftime('%d%m%Y-%H%M%S')
TODAYBACKUPPATH = BACKUP_PATH + DATETIME
db = DB_NAME
dumpcmd = MYSQLDUMP_LOCATION + "mysqldump -u " + DB_USER + " -p" + DB_USER_PASSWORD + " " + db + " > " + TODAYBACKUPPATH + "/" + db + ".sql"
os.system(dumpcmd)
except Exception as e:
print("something here")
return None
I'm getting this error:
'c:/Program' is not recognized as an internal or external command,
operable program or batch file.
How can I pass this Windows path correctly?
Thanks!
I'm new at exporting data, I research all over the net but it was really hard for me to understand, can someone help me to know the basic about it.
This is my main problem: I want to download a specific data from mysql base on the date range I choose in my client, then when I click the download button, I want these data from mysql to be save in my computer together the user have the option to save it as CSV/Excel, I'm using python for my webservice. Thank you
This is my code right know in my webservice:
#api.route('/export_file/', methods=['GET', 'POST'])
def export_file():
if request.method == 'POST':
selectAttendance = """SELECT * FROM attendance"""
db.session.execute(selectAttendance)
db.session.commit()
f = csv.writer(open("file.csv", "w"))
for row in selectAttendance:
f.writerow([str(row)])
return jsonify({'success': True})
In general:
Set the mime header "Content-Type" part of the http header to the corresponding MIME-Type matching your data.
This tells the browser what type of data the webserver is going to send.
Send the actual data in the 'body'
With flask:
Forcing application/json MIME type in a view (Flask)
http://flask.pocoo.org/docs/0.10/patterns/streaming/
def get(self):
try:
os.stat(BACKUP_PATH)
except:
os.mkdir(BACKUP_PATH)
now = datetime.now() # current date and time
year = now.strftime("%Y")
month = now.strftime("%m")
day = now.strftime("%d")
time = now.strftime("%H:%M:%S")
date_time = now.strftime("%d_%m_%Y_%H:%M:%S")
TODAYBACKUPPATH = BACKUP_PATH + '/' + date_time
try:
os.stat(TODAYBACKUPPATH)
except:
os.mkdir(TODAYBACKUPPATH)
print ("checking for databases names file.")
if os.path.exists(DB_NAME):
file1 = open(DB_NAME)
multi = 1
print ("Databases file found...")
print ("Starting backup of all dbs listed in file " + DB_NAME)
else:
print ("Databases file not found...")
print ("Starting backup of database " + DB_NAME)
multi = 0
if multi:
in_file = open(DB_NAME,"r")
flength = len(in_file.readlines())
in_file.close()
p = 1
dbfile = open(DB_NAME,"r")
while p <= flength:
db = dbfile.readline() # reading database name from file
db = db[:-1] # deletes extra line
dumpcmd = "mysqldump -h " + DB_HOST + " -u " + DB_USER + " -p" + DB_USER_PASSWORD + " " + db + " > " + pipes.quote(TODAYBACKUPPATH) + "/" + db + ".sql"
os.system(dumpcmd)
gzipcmd = "gzip " + pipes.quote(TODAYBACKUPPATH) + "/" + db + ".sql"
os.system(gzipcmd)
p = p + 1
dbfile.close()
else:
db = DB_NAME
dumpcmd = "mysqldump -h " + DB_HOST + " -u " + DB_USER + " -p" + DB_USER_PASSWORD + " " + db + " > " + pipes.quote(TODAYBACKUPPATH) + "/" + db + ".sql"
os.system(dumpcmd)
gzipcmd = "gzip " + pipes.quote(TODAYBACKUPPATH) + "/" + db + ".sql"
os.system(gzipcmd)
# t = ("Your backups have been created in '" + TODAYBACKUPPATH + "' directory")
return "Your Folder have been created in '" + TODAYBACKUPPATH + "'."