I have searched the docs and SO and could not find anything to resolve my issue. I am trying to call a select from my sqlite database and add it to a dictionary with the columns as keys. When I do this it returns a row for each column/key. It is has 14 columns and if I only have 4 rows it repeats for each one. This was the first attempt
columns = [desc[0] for desc in cursor.description]
results = []
for row in r:
Summary = {}
items = zip(columns, row)
for (items, values) in items:
Summary[items] = row
results.append(Summary)
Then I also tried the row_factory as given in the docs. That didn't work. My end goal is to be able to print out to a text file verticly by using
for x in results:
print x[name]
print x[email]
etc
Any help is appreciated
You are creating your dictionary incorrectly. Use:
for row in r:
summary = dict(zip(columns, row))
results.append(summary)
instead.
Your code sets the whole row sequence as the value for each key in Summary, instead of the individual column value, then appending that same dictionary to the results list for each column key..
Related
When I run a SQLAlchemy query to get values from a SQLAlchemy.String() column;
query = db.session.query(self.hosts_class.cluster.distinct())
for row in query.all()
Iterating and printing the result object results in output like;
('sometext',)
If I browse the row object I can see that row contains a key, value pair of "0": "sometext". What's the correct way to access this value from row?
row is a single-element tuple, so you could access by index:
for row in query:
print(row[0])
or by unpacking the tuple
for row, in query: # <- note the comma
print(row)
I am defining a function here and making a query.
def fetch(temp_pass,temp_accno):
cur.execute('''SELECT id, name, acc_no, ph_no,address, email,balance
FROM accounts
WHERE id = %s and acc_no = %s''',
(str(temp_pass), str(temp_accno)));
row = cur.fetchall()
print(row[2])
In this row should be a list of length 7 but when I run print(row[2])
it gives me error that list index out of range.
This is the error I get
File "Accounting.py", line 13, in fetch
print(row[2])
IndexError: list index out of range
row = cur.fetchall() won't give you a row but a list of rows, so row is not a row at all and row[2] is the third row in the list, not the third field in a row. If you want only a row use cur.fetchone().
Note the the query might return several rows and it is not clear what you want to do in that case so I won't deal with it here. cur.fetchone() will give you only one row anyway.
row[2] returns the 3rd row of the list of rows. row[0][2] returns the 3rd column of the 1st row.
You could run a snippet like this to visualize what gets returned:
cur.execute(...)
for row in cur:
print(row)
After saving some data in a variable with cursor.fetchall(), it looks as follows:
mylist = [('abc1',), ('abc2',)] this is apparently a list.
That is not the issue.
The problem is that the following doesn't work:
if 'abc1' in mylist
it can't find 'abc1'. Is there a way in Python to do it easily or do I have to use a loop?
fetchall() returns a row list, i.e., a list containing rows.
Each row is a tuple containing column values. There is a tuple even when there is only one column.
To check whether a row is in the row list, you have to check for the row instead of the column value alone:
if ('abc1',) in mylist
This is problem with using select * statement.
Instead use select col1,col2 from table_name
Below code might help
sql = "select col1,col2,col3 from table_name"
cursor.execute(sql) # initialize cursor in your way
input_dict = dict( (row[0],(row[1],row[2])) for row in cursor.fetchall() )
Can I rely on keys() to always return column names in the same order as the query results when selecting all columns? Based on my output it appears to be true, but I can't find any documentation that guarantees it. I'm inclined to think it is true because while dictionaries may be unordered, they should be consistent.
# Loop through each table in the database where all tables
# are being reflected at once.
for table in Base.metadata.tables.values():
# Select all columns
rows = Session.query(table).all()
# Prepend a header row
rows.insert(0, rows[0].keys())
# Output to file
fh = open(filename, 'wb')
outcsv = unicodecsv.writer(fh)
outcsv.writerows(rows)
fh.close
Similarly, column_descriptions also appears to return names in the same order as the values, but again I am not certain if it will always be true.
# Prepend list of column names as the first row
rows.insert(0, [col['name'] for col in Session.query(table).column_descriptions])
Any help will be much appreciated. Thanks!
The rows returned are KeyedTuples; the ordering of them in 1 query is dictated by the order of the columns in the original select, which will absolutely guarantee that the order is the same as returned by .keys(), and the same for each item of in the same query results.
my question is very similar to here: Find unique values in a Pandas dataframe, irrespective of row or column location
I am very new to coding, so I apologize for the cringing in advance.
I have a .csv file which I open as a pandas dataframe, and would like to be able to return unique values across the entire dataframe, as well as all unique strings.
I have tried:
for row in df:
pd.unique(df.values.ravel())
This fails to iterate through rows.
The following code prints what I want:
for index, row in df.iterrows():
if isinstance(row, object):
print('%s\n%s' % (index, row))
However, trying to place these values into a previously defined set (myset = set()) fails when I hit a blank column (NoneType error):
for index, row in df.iterrows():
if isinstance(row, object):
myset.update(print('%s\n%s' % (index, row)))
I get closest to what I was when I try the following:
for index, row in df.iterrows():
if isinstance(row, object):
myset.update('%s\n%s' % (index, row))
However, my set prints out a list of characters rather than the strings/floats/values that appear on my screen when I print above.
Someone please help point out where I fail miserably at this task. Thanks!
I think the following should work for almost any dataframe. It will extract each value that is unique in the entire dataframe.
Post a comment if you encounter a problem, i'll try to solve it.
# Replace all nones / nas by spaces - so they won't bother us later
df = df.fillna('')
# Preparing a list
list_sets = []
# Iterates all columns (much faster than rows)
for col in df.columns:
# List containing all the unique values of this column
this_set = list(set(df[col].values))
# Creating a combined list
list_sets = list_sets + this_set
# Doing a set of the combined list
final_set = list(set(list_sets))
# For completion's sake, you can remove the space introduced by the fillna step
final_set.remove('')
Edit :
I think i know what happens. You must have some float columns, and fillna is failing on those, as the code i gave you was replacing missing values with an empty string. Try those :
df = df.fillna(np.nan) or
df = df.fillna(0)
For the first point, you'll need to import numpy first (import numpy as np). It must already be installed as you have pandas.