I am working on a Python script (Python version 2.5.1 on Windows XP) that involves connecting to a Microsoft Access (.mdb) database to read values from a table. I'm getting some unexpected results with one record whereby the field of interest precision is getting rounded.
I know the Access table field of interest is a Double data type. But, the value that caused me to discover this in the table is 1107901035.43948. When I read the value in the Python code and print it out, it's showing 1107901035.44.
Is there a pyODBC connection parameter or other that must be set? I couldn't find anything in the documentation
Here's what my code looks like (the intention is to resolve unneeded records by identifying the record that has the greatest value for my field of interest):
conn = pyodbc.connect('DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};DBQ=' + pGDB)
conn.autocommit = True
cursor = conn.cursor()
tableList = []
for x in cursor.tables():
val = str(x[2])
if val[0:3] <> "MSy":
if val[0:3] <> "GDB":
if val[-5:] <> "Index":
tableList.append(val)
for x in tableList:
try:
SQL = "SELECT * FROM %s" % (x)
cursor.execute(SQL)
rows = cursor.fetchall()
counter = 0
for row in rows:
counter +=1
if counter > 1:
print "Site %s is a multipart basin" % (x)
SQL = "SELECT MAX(Shape_Area) AS AREA FROM %s" % (x)
cursor.execute(SQL)
row = cursor.fetchone()
val = row.AREA
print str(val)
SQL = "DELETE * FROM %s WHERE Shape_Area < %s" % (x, val)
cursor.execute(SQL)
thanks,
Tom
Django uses the Jinja template, so you can use its round filer. It works as follows:
template.html
<p>{{ VALUE| round(2, 'floor') }}</p>
Check out Jinja's documentation on topic
The SQL round function can also do this job:
SQL = "SELECT ROUND(MAX(Shape_Area), 2) AS AREA FROM %s" % (x)
Related
I have an ever growing and changing database that reflects a permits passed by the State and EPA.
As the database changes and updates I need to transfer the relevant information.
The script does two things; first it checks which fields are the same and creates a list of fields and data that will be inserted into the new database. Second to insert the data into the new database.
Problem is I cannot get it to insert. I have matched everything like it says online in various ways but i get error ('42000', '[42000] [Microsoft][ODBC Microsoft Access Driver] Syntax error in INSERT INTO statement. (-3502) (SQLExecDirectW)').
I cannot figure out how to prevent it.
Code:
import pyodbc
importDatabase = r"J:\ENVIRO FIELD\AccessDatabases\MS4\MS4 Town Databases\~Template\MS4_Apocalypse Import DEV 1.accdb"
"Create the Import Database Connection"
connectionImport = pyodbc.connect(r'Driver={Microsoft Access Driver (*.mdb, *.accdb)};DBQ=%s;' %(importDatabase))
cursorImport = connectionImport.cursor()
"####---Outfall Section---####"
"Import the outfall names into the new database"
tbl = "tbl_Outfall_1_Profile"
exportList = []
importList = []
for row in cursorImport.columns(table = "tblExportMigration_Outfall_1_Profile"):
field = row.column_name
exportList.append(field)
for row in cursorImport.columns(table = "tbl_Outfall_1_Profile"):
field = row.column_name
importList.append(field)
matchingList = []
for field in exportList:
if field != "outfallID":
if field in importList:
matchingList.append(field)
else:
continue
sqlValue = ""
for field in matchingList:
sqlValue += "[%s], " %(field)
sqlValue = sqlValue[:-2]
sql = "SELECT %s from %s" %(sqlValue, "tblExportMigration_Outfall_1_Profile")
for rowA in cursorImport.execute(sql):
tupleList = list(rowA)
tupleList = ["" if i == None else i for i in tupleList]
tupleValues = tuple(tupleList)
sqlUpdate = """INSERT INTO tbl_Outfall_1_Profile (%s) Values %s;""" %(sqlValue, tupleValues)
cursorImport.execute(sqlUpdate)
cursorImport.close()
This is the sql string I create
"INSERT INTO tbl_Outfall_1_Profile ([profile_OutfallName], [profile_HistoricalName1], [profile_HistoricalName2], [profile_HistoricalName3], [profile_HistoricalName4]) Values ('756', '', '', '', '');"
Taking what #Gord Thompson said I was actually able to create a dynamic parameter flow
First created a module to create the ?
def Defining_Paramters(length):
parameterString = ""
for x in range(1,length):
parameterString += "?, "
parameterString += "?"
return parameterString
Then stuck it into the string for the sql update
sqlUpdate = sqlUpdate = "INSERT INTO %s (%s) Values (%s);" %(table, sqlFrameworkSubStr, parameters)
Run the cursor and commit it
cursorTo.execute(sqlUpdate, (dataTuple))
connectionTo.commit()
It would seem that you have to create the query in its entirety then have your data in tuple format for entry
This is the sql string [I think] I create
Try this:
sqlUpdate = """INSERT INTO tbl_Outfall_1_Profile (%s) Values (%s);""" %(sqlValue, tupleValues)
or perhaps:
sqlUpdate = "INSERT INTO tbl_Outfall_1_Profile (%s) Values (%s);" %(sqlValue, tupleValues)
this is my python code to get all tickets from a sqlite database, where the "IR" number is the same. When I run it and search a value, sqlite prints only one row, for the value "IR". But there are two rows in my database. This is my Database:
Database content
def seek(IR):
conn = sqlite3.connect("Test.db")
cur = conn.cursor()
sql = "SELECT IR FROM Tickets WHERE IR = ?"
cur.execute(sql, (IR))
fetch = cur.fetchall()
print("Printing IR ", IR)
print("Total rows are: ", len(fetch))
for row in fetch:
print("IR: ", row[0])
print("Stellplatz: ", row[2])
conn.close()
I solve the issue on my own. I forgot the "*" in the SQL Statment.
I'm quite new to mysql as in manipulating the database itself. I succeeded to store new lines in a table but my next endeavor will be a little more complex.
I'd like to fetch the column names from an existing mysql database and save them to an array in python. I'm using the official mysql connector.
I'm thinking I can achieve this through the information_schema.columns command but I have no idea how to build the query and store the information in an array. It will be around 100-200 columns so performance might become an issue so I don't think its wise just to iterate my way through it for each column.
The base code to inject code into mysql using the connector is:
def insert(data):
query = "INSERT INTO templog(data) " \
"VALUES(%s,%s,%s,%s,%s)"
args = (data)
try:
db_config = read_db_config()
conn = MySQLConnection(db_config)
cursor = conn.cursor()
cursor.execute(query, args)
#if cursor.lastrowid:
# print('last insert id', cursor.lastrowid)
#else:
# print('last insert id not found')
conn.commit()
cursor.close()
conn.close()
except Error as error:
print(error)
As said this above code needs to be modified in order to get data from the sql server. Thanks in advance!
Thanks for the help!
Got this as working code:
def GetNames(web_data, counter):
#get all names from the database
connection = create_engine('mysql+pymysql://user:pwd#server:3306/db').connect()
result = connection.execute('select * from price_usd')
a = 0
sql_matrix = [0 for x in range(counter + 1)]
for v in result:
while a == 0:
for column, value in v.items():
a = a + 1
if a > 1:
sql_matrix[a] = str(('{0}'.format(column)))
This will get all column names from the existing sql database
I am trying to make a SELECT statement to Mysql datbase using pymysql.
This is the code. I am passing a variable to the select statement, and to my surprise this is a huge pain in the lemon. Any idea what am I missing here?
def getUrlFromDatabase(n):
stmt = "SELECT * FROM jsonTes ORDER BY website LIMIT %s-1,1"
cur.execute(stmt,str(n))
return cur.fetchone()
conn = pymysql.connect(host='localhost', port=3306, user='root', passwd='passwd', db='email_database', charset='utf8')
cur = conn.cursor()
cur.execute("USE database")
getUrlFromDatabase(0)
Error:
This is what I try to achieve: Return the nth record from MySQL query
pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ''0'-1,1' at line 1")
LIMIT in MySQL takes numeric arguments, which must both be nonnegative integer constants. You have to calculate the expression in Python and then pass the integer as a single parameter. Also, you need to put the parameter in a tuple:
def getUrlFromDatabase(n):
stmt = "SELECT * FROM jsonTes ORDER BY website LIMIT %s, 1"
cur.execute(stmt, (n-1 if n > 0 else 0,))
return cur.fetchone()
You are not passing the value 1 for %s in the string format.
stmt = "SELECT * FROM jsonTes ORDER BY website LIMIT %s" %n for limit n
you can use like that
def getUrlFromDatabase(n):
stmt = "SELECT * FROM jsonTes ORDER BY website LIMIT {}, 1"
cur.execute(stmt.format(n-1 if n > 0 else n))
return cur.fetchone()
Sorry if this question is stupid, I am 2 days into learning python
I have been beating my head against a wall trying to understand why my python script can run SELECT statements but not UPDATE or DELETE statements.
I believe this would be a MySQL issue and not a Python issue but I am no longer able to troubleshoot
pcheck.py
import re
import time
import json
import MySQLdb
import requests
from array import *
conn = MySQLdb.connect([redacted])
cur = conn.cursor()
sql1 = "SELECT pkey,pmeta FROM table1 WHERE proced = 0 LIMIT 1"
cur.execute(sql1)
row = cur.fetchone()
while row is not None:
print "row is: ",row[0]
rchk = [
r"(SHA256|MD5)",
r"(abc|def)"
]
for trigger in rchk:
regexp = re.compile(trigger)
pval = row[1]
if regexp.search(pval) is not None:
print "matched on: ",row[0]
sql2 = """INSERT INTO table2 (drule,dval,dmeta) VALUES('%s', '%s', '%s')"""
try:
args2 = (trigger, pval, row[1])
cur.execute(sql2, args2)
print(cur._last_executed)
except UnicodeError:
print "pass-uni"
break
else:
pass
sql3 = """UPDATE table1 SET proced=1 WHERE pkey=%s"""
args3 = row[0]
cur.execute(sql3, args3)
print(cur._last_executed)
row = cur.fetchone()
sql3 = """DELETE FROM table1 WHERE proced=1 AND last_update < (NOW() - INTERVAL 6 MINUTE)"""
cur.execute(sql3)
print(cur._last_executed)
cur.close()
conn.close()
print "Finished"
And the actual (and suprisingly expected) output:
OUTPUT
scrape#:~/python$ python pcheck.py
row is: 0GqQ0d6B
UPDATE table1 SET proced=1 WHERE pkey='0GqQ0d6B'
DELETE FROM table1 WHERE proced=1 AND last_update < (NOW() - INTERVAL 6 MINUTE)
Finished
However, the database is not being UPDATED. I checked that the query was making it to MySQL:
MySQL Log
"2015-12-14 22:53:56","localhost []","110","0","Query","SELECT `pkey`,`pmeta` FROM `table1` WHERE `proced`=0 LIMIT 200"
"2015-12-14 22:53:57","localhost []","110","0","Query","UPDATE `table1` SET `proced`=1 WHERE `pkey`='0GqQ0d6B'"
"2015-12-14 22:53:57","localhost []","110","0","Query","DELETE FROM table1 WHERE proced=1 AND last_update < (NOW() - INTERVAL 6 MINUTE)"
However proced value for row 0GqQ0d6B is still NOT 1
If I make the same queries via Sqlyog (logged in as user) the queries work as expected.
These kind of issues can be very frustrating. You sure there's no extra spaces here?
print "row is:*"+row[0]+"*"
Perhaps comment out the
for trigger in rchk:
section, and sprinkle some print statements around?
As the commenter Bob Dylan was able to deduce the cursor needed to be committed after the change.