I have created a dictionary as follows:
dict = { 'NASDAQ': {'AMZN%', 'AAPL%', 'ABNB%'}, 'TSX': {'SHOP%', 'L%', 'RY%', 'XIF%'}}
I want to write a for loop within a query to fetch data from a table TICKER_TABLE with a column called TICKER which contains the dict values.
The query I am writing is part of a broader Python loop that looks like this:
for key in dict.keys():
query = """SELECT * FROM "TICKER_TABLE"
WHERE "TICKER" LIKE (FOR ITEM IN dict.VALUES)""""
Is there a way to do this?
If TICKER_TABLE and TICKER it's varable use f before string and {} to add varable. I'm not very advanced with sql, so I will use an extra for loop in python, although it can probably be done better but I don't know what u actually need:
for key in dict:
for value in dict[key]:
query = f"SELECT * FROM {TICKER_TABLE}
WHERE {TICKER} LIKE {value}"
Every iteretion of second for loop to AMZN%, AAPL%, SHOP% etc
Maby something for this will be helpful for u.
If you really wanted to do this using SQLAlchemy you could do it like this, combining the "likes" for each key into a single query:
import sqlalchemy as sa
engine = sa.create_engine(<connection_url>, echo=True, future=True)
# Reflect the database table into an object.
tbl = sa.Table('ticker_table', sa.MetaData(), autoload_with=engine)
with engine.connect() as conn:
for k, v in dict_.items():
q = sa.select(tbl).where(sa.or_(tbl.c.ticker.like(t) for t in v))
rows = conn.execute(q)
for row in rows:
print(row)
Generates SQL like this:
SELECT ticker_table.id, ticker_table.ticker
FROM ticker_table
WHERE ticker_table.ticker LIKE ?
OR ticker_table.ticker LIKE ?
OR ticker_table.ticker LIKE ?
OR ticker_table.ticker LIKE ?
If you don't need to process the results by key you can get them all in a single query:
import functools
import sqlalchemy as sa
# Merge the dictionary values into a single set.
vals = functools.reduce(set.union, dict_.values(), set())
with engine.connect() as conn:
q = sa.select(tbl).where(sa.or_(tbl.c.ticker.like(t) for t in vals))
rows = conn.execute(q)
for row in rows:
print(row)
print()
Related
Im using psycopg2 to access postgres database using the below query. In order to return a dictionary from the executed query, im using DictCursor in my cursor but still my output is a list and not a dictonary.
Here is the program and output below.
import psycopg2.extras
try:
conn = psycopg2.connect("user='postgres' host='localhost' password='postgres'",
)
except:
print "I am unable to connect to the database"
cur = conn.cursor(cursor_factory=psycopg2.extras.DictCursor)
cur.execute("""SELECT datname from pg_database""")
rows = cur.fetchall()
print "\nShow me the databases:\n"
print rows
Output:-
[['template1'], ['template0'], ['postgres'], ['iip'], ['test'], ['test_postgres'], ['testdb']]
It looks like a list, smells like a list, but it's a DictRow.
rows = cur.fetchall()
for row in rows :
print(type(row))
#>>> <class 'psycopg2.extras.DictRow'>
This means that you can still use the column names as keys to access the data :
rows = cur.fetchall()
print([row['datname'] for row in rows])
This class inherits directly from the builtinlist and add all the needed methods to implement a dictionary logic, but it doesn't change the representation __repr__ or __str__, so the output is the same as a list.
class DictRow(list):
"""A row object that allow by-column-name access to data."""
fetchall() packs all the queried rows in a list without specifying the exact type.
Btw, maybe you are looking for this kind of cursor : RealDictCursor ?
For those who came where because they really like the easy reference of the dictionary for column:value record representation, the answer by PRMoureu which notes that the DictRow has all the usual dictionary logic means that you can iterate over the DictRow with .items() and get the key:value pairs.
rows = cur.fetchall()
row_dict = [{k:v for k, v in record.items()} for record in rows]
Will turn your list of DictRow records into a list of dict records.
I am trying to get the last record from my tinyDB, so I want to make a query that looks like that:
"SELECT * FROM table ORDER BY id DESC LIMIT 1"
which should give me the last row. though I can't figure out how to do it with TinyDB.
If you want to order db by time descending for example:
od = sorted(db.all(), key=lambda k: k['time'])
print(od[-1])
According to the documentation, the following would return the doc id of the final element in the db in TinyDB 4.7.0:
el = db.all()[-1]
record = db.get(doc_id=el.doc_id)
Using Document IDs
How about:
table = db.table('table_name')
table.get(doc_id=len(table))
See the way of doc_id here
Using a Query and doing an update :
with TinyDB('db.json') as db:
my_table = db.table('a_table_name')
my_query= Query()
first_of_table_for_this_query = my_table.search(my_query.some_field == some_value)[1]
last_of_table_for_this_query = my_table.search(my_query.some_field == some_value)[-1]
# exemple of use for updating first inserted and last inserted only
my_table.update({'some_field': some_value+42}, doc_ids=[
first_of_table_for_this_query,
last_of_table_for_this_query
])
I have a sql query like this running in python:
select sum(a), date from table group by date
Then I execute this query and get the result:
cursor.execute (query, queryParameters)
rows = cursor.fetchall();
As expected rows is an array:
(('2014-05-13', Decimal('1.6666666667')), ('2014-05-14', Decimal('33.0151515152')), ('2014-05-15', Decimal('66.4850000000')), ('2014-05-16', Decimal('49.8274022154')), ('2014-05-18', Decimal('4.0000000000')))
But i want it as an hash, where the date is the key and the sum is the value(every row is a key-value pair). And since it is a group by date, it makes sense to have it in a hash. Something like this:
{ '2014-05-13' => '1.6666666667', '2014-05-14' => '33.0151515152'....}
How can I achieve this?
Change the order of the fields and pass fetchall() results to dict():
query = "select date, sum(a) from table group by date"
cursor.execute (query, queryParameters)
result = dict(cursor.fetchall())
Demo:
>>> from decimal import Decimal
>>> data = (('2014-05-13', Decimal('1.6666666667')), ('2014-05-14', Decimal('33.0151515152')), ('2014-05-15', Decimal('66.4850000000')),
>>> dict(data)
{'2014-05-18': Decimal('4.0000000000'), '2014-05-13': Decimal('1.6666666667'), '2014-05-15': Decimal('66.4850000000'), '2014-05-14': Decimal('33.0151515152'), '2014-05-16': Decimal('49.8274022154')}
I have this code :
cur.execute("SELECT * FROM foo WHERE date=?",(date,))
for row in cur:
list_foo.append(row[2])
cur.execute("SELECT * FROM bar WHERE date=?",(date,))
for row in cur:
list_bar.append(row[2])
It works fine, but I’d like to automize this. I have made a list of the tables in my sqlite database, and I’d like something like this :
table_list = ['foo','bar']
for t in table_list:
cur.execute("SELECT * FROM "+t+" WHERE date=?",(date,))
for row in cur:
# and here I’d like to append to the list which name depends of t (list_foo, then list_bar, etc.)
But I don’t know how to do that. Any idea ?
Use a dictionary to collect your data. Don't try to set new local names for each list.
You could use string templating too, and a list comprehension to turn your result rows into lists:
data = {}
for t in table_list:
cur.execute("SELECT * FROM {} WHERE date=?".format(t), (date,))
data[t] = [row[2] for row in cur]
One caveat: only do this with a pre-defined list of table names; don't ever interpolate untrusted input like that without hefty escaping to prevent SQL injection attacks.
I have a table that has about 50 columns. To retrieve the variables from the table I am doing:
cursor.execute("SELECT * FROM title WHERE vendor_id = '%s'"%vendor_id)
data=cursor.fetchone()
provider, language, type, subtype, vendor_id = data[0], data[1], data[2], data[3], data[4]
etc...
Is there a way to do this more concisely: the variables I want to define are also the names of the columns. Perhaps something like (in pseudocode) --
values=cursor.fetchone; columns=cursor.fetchcolumns()
data = zip(columns, values)
select * is going to be particularly challenging here as you don't know what columns are being returned in which order. Depending on the database you're using and the wrapper you're using, you should be able to retrieve your rows as dicts instead which allows you to reference the columns as dict keys in the rows.
For instance, MySQLdb supports this through a DictCursor. See http://www.kitebird.com/articles/pydbapi.html
For other libraries, they should offer a similar feature.
You can use cursor.description and then convert the result to a dict:
import sqlite3
cnx = sqlite3.connect(r"g:\Python\Test\dabo\turnos\turnos.sqlite")
cur = cnx.execute("select * from Paciente")
rec = cur.fetchone()
fields = [i[0] for i in cur.description]
values = dict(zip(fields, rec))
print values["PacID"], values["PacNombre"] # ,...