We Keep Coding

Use cursor.executemany instead of cursor.execute with tying panda header to sql variables

I have a python script that updates rows in an oracle sql table correctly, however I am using cursor.execute and try/except so if one update fails, it kills the whole run.
I want to be able to have it run through the whole update and just log the error and move onto the next one, which is where cursor.executemany comes in.
https://cx-oracle.readthedocs.io/en/latest/user_guide/batch_statement.html
Here is the script, it works great, except for the all or nothing error approach.
#oracle sql update statement for SHRTCKN
banner_shrtckn_update = """
UPDATE SATURN.SHRTCKN A
SET A.SHRTCKN_COURSE_COMMENT = :course_comment,
A.SHRTCKN_REPEAT_COURSE_IND = :repeat_ind,
A.SHRTCKN_ACTIVITY_DATE = SYSDATE,
A.SHRTCKN_USER_ID = 'STU00940',
A.SHRTCKN_DATA_ORIGIN = 'APPWORX'
WHERE A.SHRTCKN_PIDM = gb_common.f_get_pidm(:id) AND
A.SHRTCKN_TERM_CODE = :term_code AND
A.SHRTCKN_SEQ_NO = :seqno AND
A.SHRTCKN_CRN = :crn AND
A.SHRTCKN_SUBJ_CODE = :subj_code AND
A.SHRTCKN_CRSE_NUMB = :crse_numb
"""
def main():
# get time of run and current year
now = datetime.datetime.now()
year = str(now.year)+"40"
# configure connection strings for banner PROD
db_pass = os.environ['DB_PASSWORD']
dsn = cx_Oracle.makedsn(host='FAKE', port='1521', service_name='TEST.FAKE.BLAH')
try: # initiate banner db connection -- PROD
banner_cnxn = cx_Oracle.connect(user=config.db_test['user'], password = db_pass, dsn=dsn)
writeLog("---- Oracle Connection Made ----")
insertCount = 0
for index, row in df.iterrows():
shrtcknupdate(row,banner_cnxn)
insertCount = insertCount + 1
banner_cnxn.commit()
banner_cnxn.close()
writeLog(str(insertCount)+" rows updated")
except Exception as e:
print("Error: "+str(e))
writeLog("Error: "+str(e))
def writeLog(content):
print(content)
log.write(str(datetime.date.today())+" "+content+"\n")
#define the variable connection between panda/csv and SHRTCKN table
def shrtcknupdate(row, connection):
sql = banner_shrtckn_update
variables = {
'id' : row.Bear_Nbr,
'term_code' : row.Past_Term,
'seqno' : row.Seq_No,
'crn' : row.Past_CRN,
'subj_code' : row.Past_Prefix,
'crse_numb' : row.Past_Number,
'course_comment' : row.Past_Course_Comment,
'repeat_ind' : row.Past_Repeat_Ind
}
cursor = connection.cursor()
cursor.execute(sql, variables)
if __name__ == "__main__":
writeLog("-------- Process Start --------")
main()
writeLog("-------- Process End --------")
The executemany option, I can turn on batcherrors=True
and it will do exactly what I need.
The problem I am running into, is if I get rid of the for loop that runs through the excel/panda dataframe to update the oracle sql rows, which is not needed when doing the update in batch, then how do I attach the column headers to the sql update statement.
If I leave in the for loop, I get this error when using executemany:
Error: parameters should be a list of sequences/dictionaries or an integer specifying the number of times to execute the statement

For named binds, you need to provide a list of dictionaries. This list can be obtained by to_dict(orient='records'):
‘records’ : list like [{column -> value}, … , {column -> value}]
banner_shrtckn_update = """
UPDATE SATURN.SHRTCKN A
SET A.SHRTCKN_COURSE_COMMENT = :Past_Course_Comment,
A.SHRTCKN_REPEAT_COURSE_IND = :Past_Repeat_Ind,
A.SHRTCKN_ACTIVITY_DATE = SYSDATE,
A.SHRTCKN_USER_ID = 'STU00940',
A.SHRTCKN_DATA_ORIGIN = 'APPWORX'
WHERE A.SHRTCKN_PIDM = gb_common.f_get_pidm(:Bear_Nbr) AND
A.SHRTCKN_TERM_CODE = :Past_Term AND
A.SHRTCKN_SEQ_NO = :Seq_No AND
A.SHRTCKN_CRN = :Past_CRN AND
A.SHRTCKN_SUBJ_CODE = :Past_Prefix AND
A.SHRTCKN_CRSE_NUMB = :Past_Number
"""
def main():
# get time of run and current year
now = datetime.datetime.now()
year = str(now.year)+"40"
# configure connection strings for banner PROD
db_pass = os.environ['DB_PASSWORD']
dsn = cx_Oracle.makedsn(host='FAKE', port='1521', service_name='TEST.FAKE.BLAH')
try: # initiate banner db connection -- PROD
banner_cnxn = cx_Oracle.connect(user=config.db_test['user'], password = db_pass, dsn=dsn)
writeLog("---- Oracle Connection Made ----")
# batch execute banner_shrtckn_update
cursor = banner_cnxn.cursor()
data = df[['Bear_Nbr', 'Past_Term', 'Seq_No', 'Past_CRN', 'Past_Prefix', 'Past_Number', 'Past_Course_Comment', 'Past_Repeat_Ind']].to_dict('records')
cursor.executemany(banner_shrtckn_update, data, batcherrors=True)
for error in cursor.getbatcherrors():
writeLog(f"Error {error.message} at row offset {error.offset}")
banner_cnxn.commit()
banner_cnxn.close()
except Exception as e:
print("Error: "+str(e))
writeLog("Error: "+str(e))
This isn't described in detail in the documenation, but you can find an example for named binds in python-cx_Oracle/samples/bind_insert.py.
(Please note that I adjusted the variable names in your sql statement to the dataframe column names to avoid renaming of the columns at creating data.)

Why does one of my login functions work and the other does not, despite the fact that they both have the identical code?

So I am trying a login system for my bank management project and I created two of the login system.
one for the admin system and the other for the customer. This is the code and text file for both. Why would my first function work and not the second? FYI I can't use any global function and dictionary and the error I been getting is ValueError: too many values to unpack (expected 2)
def LAdminAccount():
EnteredStaffAccountNumber = str(input("========== Please Type in Your Account Number:"))
EnteredStaffPassword = str(input("========== Please Type in Your Password:"))
A= open("AdminUser.txt","r")
for line in A.readlines():
us,pw = line.strip().split("|")
if (EnteredStaffAccountNumber == us ) and (EnteredStaffPassword == pw):
print ("Login successful")
A.close()
AdminMenu()
print ("Wrong username/password")
return
def LCustomerAccount():
EnteredID = str(input("========== Please Type in Your Account ID:"))
EnteredPassword = str(input("========== Please Type in Your Password:"))
B= open("Customerlogin.txt","r")
for line in B.readlines():
id,pw = line.split("|",1)
print (id)
print (pw)
if (EnteredID == id ) and (EnteredPassword == pw):
print ("Customer Login successful")
B.close()
CustomerMenu()
print ("Wrong Account Number/password")
menu()
AdminUser.txt
00001|1234
Customerlogin.txt
000403100865|3088
Output is:
000403100865
3088
Customer Login successful
Wrong Account Number/password

The error suggests that the problem is the following line:
id,pw = line.split("|")
If you have more than one "|" in your text your will not be able to split them this way.
To guarantee the string is split at most once try replacing with:
id,pw = line.split("|", 1)

DbfRead: get the record values from one database and use it in another?

I have this Python program with a for loop(Not EOF loop). Inside the for loop, I assign record values to strSsn and strStoreid, but I want to use them inside another database loop. When I use them AFTER the loop I set them in, I only find the value for the last record in the loop I set them in.
with DBF(r'C:\Sonichr\\empInfo.DBF',lowernames=True,load =True,ignore_missing_memofile=True) as empInfo_table:
for empInfo_rec in empInfo_table:
#Append the records in a list
strSsn = empInfo_rec['ssn']
strStoreid = empInfo_rec['storeid']
s20data_table = DBF('C:\Sonichr\\s20data.DBF',lowernames=True,load =True,ignore_missing_memofile=True)
for s20data_rec in s20data_table:
storeId = s20data_rec['storeid']
ssn = s20data_rec['ssn']
positionde = s20data_rec['positionde']
payrate = s20data_rec['payrate']
if storeId + ssn == strStoreId + strSsn:
if positionde == 'Carhop' :
abraPayRate1 = payrate
else:
abraPayRate0 = payrate
Updated!!

Flask-SQLAlchemy Check if row exists

I have already looked at this SO question, so far it has not helped.
So I am trying to check if a row exists in a table with Flask-SQLAlchemy, and so far nothing has helped.
My current code:
#app.route('/guilds/<guildid>')
def guildsettings(guildid):
discord = make_session(token=session.get('oauth2_token'))
user = discord.get(API_BASE_URL + '/users/#me').json()
guilds = discord.get(API_BASE_URL + '/users/#me/guilds').json()
row = Settings.query.all()
exists = Settings.query.filter_by(db.exists().where(Settings.guildid==guildid)).scalar()
print(exists)
if exists == True:
info = Settings.query.filter_by(guildid=guildid).first()
return render_template("guild.html", id=guildid, guilds=guilds, User=user, prefix=info.prefix, logschannel=info.logschannel, modrole=info.modrole, adminrole=info.adminrole, welcomechannel=info.welcomechannel, welcomemessage=info.welcomemessage, dadbot=info.dadbot, music=info.music, funcmds=info.funcmds, weather=info.weather, wapikey=info.wapikey)
else:
return render_template("notinserver.html", guildid=guildid, link="https://discordapp.com/api/oauth2/authorize?client_id=xxxxxx&permissions=8&redirect_uri=xxxxxxx%2Fme&scope=bot&guild_id={}".format(guildid))
I am not sure on what to do now. Any help is appreciated.

If it needs to be excactly one result:
try:
Settings.query.filter_by(db.exists().where(Settings.guildid==guildid)).one()
except sqlalchemy.orm.exc.NoResultFound:
return False
except sqlalchemy.orm.exc.MultipleResultsFound:
#do what ever you want to do if there is more than one result
return True

Python will not pull values from dictionary properly using for loop

Im trying to use a dictionary to check a given number of servers listed for a particular SQL backup success or fail. My problem so far is when I run this code:
for serverChk in srvrDict['Server']:
it returns the server name as single characters on each new line like:
S
E
R
V
E
R
So in my trial I see this "Error connecting to T to check OS version" where T is the fist character of the servername. I can't seem to put my finger on it and all the searching I've done has lead me to asking.
Thanks!
class checkstatus:
#def getServers(self):
chkbkpstats = csv.reader(file('c://temp//networkerservers.csv'))
for row in chkbkpstats:
srvrDict = {}
srvrDict['Server'] = row[0]
srvrDict['Instance'] = row[1]
print srvrDict
for serverChk in srvrDict['Server']:
try:
c = wmi.WMI(server)
for os in c.Win32_OperatingSystem():
osVer = os.caption
except:
print 'Error connecting to %s to check OS version' % serverChk
if '2003' in osVer:
print 'w2k3'
if '2008' in osVer:
print 'w2k8'

I suppose you have stored a string in your dictionary. So the line for serverChk in srvrDict['Server'] translates to for serverChk in yourSavedString. This is why you are getting individual characters. To access individual dictionary items you should do for k,v in srvrDict.iteritems() where k is the key and v is the value.

You are overwriting the Server and Instance values in srvrDict each iteration of your loop through chkbkpstats, not actually generating a sequence of data with an entry for each item in your log file as it appears you expect. You need to make that a list containing dictionaries, which you append to each iteration. You are probably looking for something more like:
chkbkpstats = csv.reader(file('c://temp//networkerservers.csv'))
srvrs = []
for for row in chkbkpstats:
srvrs.append({'Name' : row[0], 'Instance' : row[1]})
for srvr in srvrs:
try:
c = wmi.WMI(srvr['Instance'])
except:
print 'Error connecting to %s to check OS version' % srvr['Name']
else:
osVer = c.Win32_OperatingSystem()[0].Caption
if '2003' in osVer:
print 'w2k3'
elif '2008' in osVer:
print 'w2k8'

There are a few problems with your code.
First, you create a new srvrDict each time you go through the first for loop, overwriting the value that was stored in this variable the last time. I think, what you actually intended to do is the following:
srvrDict = {}
for row in chkbkpstats:
srvrDict[row[0]] = row[1]
Now, srvrDict will contain an entry like {'P1RT04': ['THP06ASU']} for each row in chkbkpstats, mapping server names to lists of instances running on that server.
Then, in the second loop, use for serverChk in srvrDict: to iterate over all the entries in the dictionary. However, I'm not sure where the variable server in c = wmi.WMI(server) comes from. If this is what has been row[1] in the first loop, then you should use srcvDict[serverChk] to retrieve the value from the dictionary.
This way, the whole procedure would look something like this:
chkbkpstats = csv.reader(file('c://temp//networkerservers.csv'))
srvrDict = {}
for row in chkbkpstats:
name, instance = row[0], row[1]
if name not in srvrDict:
srvrDict[name] = []
srvrDict[name].append(instance)
for server in srvrDict:
for instance in srvrDict[server]:
try:
c = wmi.WMI(instance)
except:
print 'Error connecting to %s to check OS version' % server
else:
osVer = c.Win32_OperatingSystem()[0].caption
if '2003' in osVer:
print 'w2k3'
elif '2008' in osVer:
print 'w2k8'
else:
print 'unknown OS'
PS.: I'm not sure what's the return value of c.Win32_OperatingSystem(). [...] Update: Thanks to sr2222 for pointing this out. Code fixed.
Update: Edited the code to allow for one server hosting multiple instances.