Im trying to save three values from SQLite in a Python list. All values are from a different column but in the same row. If the value is null I dont want to add it to the list. This is the code I wrote:
def create_list(self, chat):
list = []
for x in range(1, 3):
column_name = "list" + str(x)
value = c.execute("SELECT (?) FROM user WHERE (?) NOTNULL AND id = (?)", (column_name, column_name, chat)).fetchall()
if value != None:
list.append(value[0][0])
print(list)
Instead of printing the SQLite values in a list it just prints: ['list1', 'list2', 'list3'] (If one of the values in the table is null it doesnt print that one. For example if the value in column liste3 is null it just prints ['list1', 'list2'])
How can I fix this so that it saves the actual SQLite values in the list?
I had the same problem.
SQLite package ignores NULL values by default after fetching the data. you need to put a condition that prints 'Empty' or something when it faces NULL values. something like:
conn = sqlite3.connect("someDataBase.db")
db = conn.cursor()
db.execute('''
SELECT name, email
FROM person
WHERE name = 'some dude'
''')
result = db.fetchone()
if result is None: #if a value IS NULL
print 'Empty'
else:
var = result[0]
print var
Related
Hi im trying to make a python script which will update a line each time the script is run.
im having a bit of a headscratcher to how best to tackle the part where i have to update each line (a select statement) based on the value of a dataframe.
simplified i got a string 'select null r_cnt, null t_cnt, null r_dur, null t_dur from my_table'
and i got a list containing the fields for this line [t_cnt, r_dur]
I then want the new string output to be the first string where we have removed the null in front of the the values which was present in my list but kept null in front of those not in the list.
'select null r_cnt, t_cnt, r_dur, null t_dur from my_table'
my whole code looks something like this now where im stuck at the point i mentioned above
str_to_execute = f"select * from {db}.table_desc where grp_id in (400,500,300,1200) and id not in(127,140,125)"
cursor.execute(str_to_execute)
df = as_pandas(cursor)
for index, row in df.iterrows():
# print(row['name'])
str_to_execute = f"SHOW COLUMN STATS {db}.ctrl_{row['id']}"
cursor.execute(str_to_execute)
loop = as_pandas(cursor)
for index, row in loop.iterrows():
print(row['Column'])
str_to_execute = f"select concat(cast(ctrl_id as string),cast(ctrl_date as string)) primarykey, ctrl_id, ctrl_date,null r_cnt, null t_cnt, null r_dur, null t_dur,null r_amt, null t_amt,null p_cnt, null p_dur,null p_amt, null ro_vol, null t_vol, null r_vol, null p_vol, null ro_amt, null ro_cnt, from {db}.ctrl_{row['id']}"
if #This is where im stuck
Try:
s = 'select null r_cnt, null t_cnt, null r_dur, null t_dur from my_table'
lst = ['t_cnt', 'r_dur']
checklist = ['null r_cnt', 'null t_cnt', 'null r_dur']
checkliststr = ','.join(checklist)
for itm in lst:
if itm in checkliststr:
print('null ' + itm)
s=s.replace('null ' + itm, itm)
print(s)
You could just split the functionality of your string in this case Select and from my table from the input of your statement, which is more or less if I understand your correctly a list of strings.
One possible solution would be the following:
Define or get your lists of strings:
origin_list = ["null r_cnt", "null t_cnt", "null r_dur", "null t_dur"]
goal_list = ["t_cnt", "r_dur"]
For each element of your origin_list you want to edit the element according to its existence in the goal_list, I would do it with a map and lambda:
edited_list = list(map(lambda x: edit(x, goal_list), origin_list))
Now we have to define the function and logic from your post I derived the following logic or something similar:
Now you have the adjusted string and can merge it back together with your functionality.
I have a list of tuples of which i'm inserting into a Table.
Each tuple has 50 values. How do i insert without having to specify the column names and how many ? there is?
col1 is an auto increment column so my insert stmt starts in col2 and ends in col51.
current code:
l = [(1,2,3,.....),(2,4,6,.....),(4,6,7,.....)...]
for tup in l:
cur.execute(
"""insert into TABLENAME(col2,col3,col4.........col50,col51)) VALUES(?,?,?,.............)
""")
want:
insert into TABLENAME(col*) VALUES(*)
MySQL's syntax for INSERT is documented here: http://dev.mysql.com/doc/refman/5.7/en/insert.html
There is no wildcard syntax like you show. The closest thing is to omit the column names:
INSERT INTO MyTable VALUES (...);
But I don't recommend doing that. It works only if you are certain you're going to specify a value for every column in the table (even the auto-increment column), and your values are guaranteed to be in the same order as the columns of the table.
You should learn to use code to build the SQL query based on arrays of values in your application. Here's a Python example the way I do it. Suppose you have a dict of column: value pairs called data_values.
placeholders = ['%s'] * len(data_values)
sql_template = """
INSERT INTO MyTable ({columns}) VALUES ({placeholders})
"""
sql = sql_template.format(
columns=','.join(keys(data_values)),
placeholders=','.join(placeholders)
)
cur = db.cursor()
cur.execute(sql, data_values)
example code to put before your code:
cols = "("
for x in xrange(2, 52):
cols = cols + "col" + str(x) + ","
test = test[:-1]+")"
Inside your loop
for tup in l:
cur.execute(
"""insert into TABLENAME " + cols " VALUES {0}".format(tup)
""")
This is off the top of my head with no error checking
I've created a nested dictionary calling values from a table, and I need to update the attribute table for a feature class using that data. I have it working with two hard-coded fields as a test, but I need to figure out how to automate getting the length of featFields and using that to indicate the index position for each field to be updated. So, instead of hard-coding row[1], row[2], etc. and 'LOCDESC' and 'RIMELEV', I'd be using a variable to step through index positions for each one.
I am working in Python. End goal is a toolbox for use in ArcMap 10.2 or 10.3.
import arcpy
arcpy.env.workspace = r"C:/SARP10/MACP_Tool"
#Define fields to update and the field to use as join field
Table = "Test2.csv"
Input = "Test.gdb/MHs"
csvFields = ['Location_Details', 'Elevation']
featFields = ['LOCDESC', 'RIMELEV']
csvKey = "Manhole_Number"
featKey = "FACILITYID"
csvFields.insert(0, csvKey)
featFields.insert(0, featKey)
print csvFields
#Create dictionary to store values from the update table
UpdateDict = {}
#Iterates through the values in the table and stores them in UpdateDict
with arcpy.da.SearchCursor(Table, csvFields) as cursor:
for row in cursor:
UpdateDict[row[0]] = dict(zip(featFields[1:], row[1:]))
print UpdateDict
MHNum = len(UpdateDict) # gets # of MHs to be updated
MHKeys = UpdateDict.keys() # gets key values, i.e. MH numbers
print "You are updating fields for the following {} manholes: {}".format(MHNum, MHKeys)
#Iterates through feature class attribute table and updates desired attributes
with arcpy.da.UpdateCursor(Input, featFields) as cursor:
i = 0
z = 0
for row in cursor:
i += 1
for f in UpdateDict.keys():
if f == row[0]:
row[1] = UpdateDict.values()[z]['LOCDESC']#uses counter and subdict key to call correct value
row[2] = UpdateDict.values()[z]['RIMELEV']#uses counter and subdict key to call correct value
cursor.updateRow(row)
z +=1 #counter keeps track of rows and provides index location for dictionary
print "Updating {} of {} manholes in this submittal: {}.".format(z, MHNum, f)
else:
pass
print "Updated {} of {} rows.".format(MHNum, i)
print "Script completed."
Since the (currently hard-coded) iteration of row[n] steps through the values of featFields, you can set up an for loop that iterates through them both, something like:
if f == row[0]:
# loop set by length of featFields list
for j in range(0, len(featFields) - 1):
row[j + 1] = UpdateDict.values()[z][featFields[j]]
cursor.updateRow(row)
# etc.
Note the "offset" -- row[1] should be using featFields[0] and so on -- that needs to be accounted for.
The problem was with accessing the right fields in the data dictionary. Final code accesses a list of outer keys and a list of inner key: value pairs with a variable (z) set to keep the index number equal in both lists. Thanks for your help, #Erica!
Here's what works:
import arcpy
arcpy.env.workspace = r"C:/SARP10/MACP_Tool"
#Defines fields to update and the field to use as join field
Table = "Test2.csv"
Input = "Test.gdb/MHs"
csvFields = ['Location_Details', 'Elevation', 'Rim_to_Invert', 'Rim_to_Grade', 'Cover_Size', 'Wall_Material', 'Wall_Diam', 'Wall_Lining_Interior', 'Photo2_Link', 'MH_InspectReportLink'] #update table fields
featFields = ['LOCDESC', 'RIMELEV', 'RIMTOINVERT', 'RIMTOGRADE','COVERSIZE','WALLMAT','DIAMETER','LINERTYPE','HYPERLINK_PHOTO2','HYPERLINK_RPT']#fc field names
csvKey = "Manhole_Number"
featKey = "FACILITYID"
csvFields.insert(0, csvKey)
featFields.insert(0, featKey)
print "Your table contains the following fields to be updated: {}\n".format(str(csvFields))
#Process: Create dictionary to store values from the update table, iterate through values and store in UpdateDict
UpdateDict = {}
with arcpy.da.SearchCursor(Table, csvFields) as cursor:
for row in cursor:
UpdateDict[row[0]] = dict(zip(featFields[1:], row[1:]))
## debug print "You have created update dictionary 'UpdateDict': \n{}\n\n".format(UpdateDict)
MHNum = len(UpdateDict) # gets # of MHs to be updatedMHKeys = sorted(UpdateDict.keys()) # gets key values, i.e. MH numbers
MHKeys = UpdateDict.keys() #calls outer keys (MH numbers, which are join values) into a list of keys
MHVals = UpdateDict.values()#calls inner nested key:value pairs to a list
##debug print "Dictionary keys: {}\n\n Dictionary values: {}\n\n".format(str(MHKeys),str(MHVals))
print "You are updating fields for the following {} manholes: {}".format(MHNum, str(MHKeys))
#Process: Iterates through feature class attribute table and updates desired attributes
with arcpy.da.UpdateCursor(Input, featFields) as curs:
i = 0 #attribute table row counter
for row in curs:
i += 1
for f in MHKeys:
if f == row[0]:
z = MHKeys.index(f)#get index location in MHKeys
for y in range(0,len(featFields)-1):
row[y+1] = MHVals[z][featFields[y+1]]#use z to pull corresponding value in MHVals to correct key in MHKeys
print "Current MH: {} \nUpdating Values: {} \n\n".format(f, UpdateDict.values()[z])
curs.updateRow(row)
else:
pass
print "Updated {} of {} rows.".format(MHNum, i)
print "Script completed."
I have a tuple with a single value that's the result of a database query (it gives me the max ID # currently in the database). I need to add 1 to the value to utilize for my subsequent query to create a new profile associated with the next ID #.
Having trouble converting the tuple into an integer so that I can add 1 (tried the roundabout way here by turning the values into a string and then turning into a int). Help, please.
sql = """
SELECT id
FROM profiles
ORDER BY id DESC
LIMIT 1
"""
cursor.execute(sql)
results = cursor.fetchall()
maxID = int(','.join(str(results)))
newID = maxID + 1
If you are expecting just the one row, then use cursor.fetchone() instead of fetchall() and simply index into the one row that that method returns:
cursor.execute(sql)
row = cursor.fetchone()
newID = row[0] + 1
Rather than use an ORDER BY, you can ask the database directly for the maximum value:
sql = """SELECT MAX(id) FROM profiles"""
Is there any way to get the column names from the pymssql results? If i specify as_dict=True I get back a dictionary, which does contain all the column headers, but since it is a dictionary they are not ordered.
pymssql claims to support the Python DB-API, so you should be able to get the .description attribute from your cursor object.
.description
This read-only attribute is a sequence of 7-item
sequences.
Each of these sequences contains information describing
one result column:
(name,
type_code,
display_size,
internal_size,
precision,
scale,
null_ok)
So, the first item in each of the "inner" sequences is the name for each column.
You can create a list of ordered column names using list comprehension on the cursor description attribute:
column_names = [item[0] for item in cursor.description]
To get the column names on a single comma separated line.
colNames = ""
for i in range(len(cursor.description)):
desc = cursor.description[i]
if i == 0:
colNames = str(desc[0])
else:
colNames += ',' + str(desc[0])
print colNames
Alternatively, pass the column names to a list and use .join to get them as string.
colNameList = []
for i in range(len(cursor.description)):
desc = cursor.description[i]
colNameList.append(desc[0])
colNames = ','.join(colNameList)
print colNames
It's a basic solution and need optimizing but the below example returns both column header and column value in a list.
import pymssql
def return_mssql_dict(sql):
try:
con = pymssql.connect(server, user, password, database_name)
cur = con.cursor()
cur.execute(sql)
def return_dict_pair(row_item):
return_dict = {}
for column_name, row in zip(cur.description, row_item):
return_dict[column_name[0]] = row
return return_dict
return_list = []
for row in cur:
row_item = return_dict_pair(row)
return_list.append(row_item)
con.close()
return return_list
except Exception, e:
print '%s' % (e)