I just started with coding in Python and I have one problem. I am using ibm_dbi driver.
Here is my sql example:
FOR v AS cur CURSOR FOR
SELECT S.NAME FROM SYSIBM.SYSTABLES AS S
JOIN (SELECT DISTINCT TBNAME FROM SYSIBM.SYSCOLUMNS WHERE CREATOR = 'SOME_SCHEMA_DB') AS C.TBNAME = S.NAME WHERE S.CREATOR = 'SOME_SCHEMA' AND S.TYPE = 'T'
DO
SET TABLE = SCHEMA1 ||'.'||v.Name
...
SELECT 1 INTO TEMP_TABLE FROM SYSIBM.SYSTABLES WHERE TYPE='T' AND CREATOR = SCHEMA1 AND NAME = v.NAME
END FOR;
So far i have in Python:
#make a connection to db
conn = ibm_db_dbi.connect("DATABASE=%s;HOSTNAME=%s,....)
#define a cursor
cur = conn.cursor()
sql="""SELECT S.NAME FROM SYSIBM.SYSTABLES AS S
JOIN (SELECT DISTINCT TBNAME FROM SYSIBM.SYSCOLUMNS WHERE CREATOR = 'SOME_SCHEMA_DB') AS C.TBNAME = S.NAME WHERE S.CREATOR = 'SOME_SCHEMA' AND S.TYPE = 'T'"""
resultSet = cur.execute(sql)
I don't know how to iterate through result from query and set values from the cursor. How to set value for this piece of code
SET TABLE = SCHEMA1 ||'.'||v.Name
...
SELECT 1 INTO TEMP_TABLE FROM SYSIBM.SYSTABLES WHERE TYPE='T' AND CREATOR = SCHEMA1 AND NAME = v.NAME
Tip: get your SQL working outside of python first. Your question shows that query that has syntax mistakes or typos.
To iterate through a result-set from a query, use one of the documented fetch methods for the DBI interface (example, fetchmany() , or fetchall() or others).
Here is a fragment, that uses fetchmany() .
try:
cur = conn.cursor()
sql="""SELECT S.NAME FROM SYSIBM.SYSTABLES AS S
inner JOIN (SELECT DISTINCT TBNAME
FROM SYSIBM.SYSCOLUMNS
WHERE TBCREATOR = 'SOME_SCHEMA') AS C
ON C.TBNAME = S.NAME
WHERE S.CREATOR = 'SOME_SCHEMA'
AND S.TYPE = 'T';"""
cur.execute(sql)
result = cur.fetchmany()
while result:
for i in result:
print(i)
result = cur.fetchmany()
except Exception as e:
print(e)
raise
Related
def LiraRateApiCall():
R = requests.get(url)
timestamp = R.json()['buy'][-1][0]/1000
format_date = '%d/%m/%y'
date = datetime.fromtimestamp(timestamp)
buyRate = R.json()['buy'][-1][1]
print(date.strftime(format_date))
print(buyRate)
#ADDDING TO SQL SERVER
conn = odbc.connect("Driver={ODBC Driver 17 for SQL Server};"
'Server=LAPTOP-36NUUO53\SQLEXPRESS;'
'Database=test;'
'Trusted_connection=yes;')
cursor = conn.cursor()
cursor.execute('''
INSERT INTO Data_table (Time1,Price)
VALUES
('date',140),
('Date2' , 142)
''')
conn.commit()
cursor.execute('SELECT * FROM Data_table')
for i in cursor:
print(i)
How do i pass the variables date and buy rate to the table instead of putting in values liek i did (i put in'date' , 140 for example but i want to pass variables not specific values)
You'll need to check the driver version that you're using, but what you're looking for is the concept of bind variables. I'd suggest you look into the concept of fast_executemany as well - that should help speed things up. I've edited your code to show how bind variables typically work (using the (?, ?) SQL syntax), but there are other formats out there.
def LiraRateApiCall():
R = requests.get(url)
timestamp = R.json()['buy'][-1][0]/1000
format_date = '%d/%m/%y'
date = datetime.fromtimestamp(timestamp)
buyRate = R.json()['buy'][-1][1]
print(date.strftime(format_date))
print(buyRate)
#ADDDING TO SQL SERVER
conn = odbc.connect("Driver={ODBC Driver 17 for SQL Server};"
'Server=LAPTOP-36NUUO53\SQLEXPRESS;'
'Database=test;'
'Trusted_connection=yes;')
cursor = conn.cursor()
#Setup data
data = [('date',140), ('Date2' , 142)]
#Use executemany since we have a list
cursor.executemany('''
INSERT INTO Data_table (Time1,Price)
VALUES (?, ?)
''', data)
conn.commit()
cursor.execute('SELECT * FROM Data_table')
for i in cursor:
print(i)
I dont understand at all your question
If you want to pass the variables:
insert_sql = 'INSERT INTO Data_table (Time1,Price) VALUES (' + date + ',' + str(buyRate) + ')'
cursor.execute(insert_sql)
If you want to do dynamic Insert:
You can only insert knowing the values or by inserting with a select
INSERT INTO table
SELECT * FROM tableAux
WHERE condition;
That or you could iterate through the fields you have in a table, extract them and compare it to your variables to do a dynamic insert.
With this select you can extract the columns.
SELECT *
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = N'table1'
I am writing a function that will retrieve data from sqlite table based on the parameters user provide. This is the function so far
def database_retrieve(db_file, id):
try:
conn = sqlite3.connect(db_file)
with conn:
sql_command = "SELECT * FROM my_table WHERE id = "+id
cur = conn.cursor()
cur.execute(sql_command)
result = cur.fetchall()
return result
except Exception as e:
print(e)
db_file = 'testdb.db'
print(database_retrieve(db_file, 'subject1'))
This gives me the following error
no such column: subject1
None
When I add subject1, which is an entry under the id column in my_table, directly to the sql command like this
sql_command = "SELECT * FROM my_table WHERE id = 'subject1'"
it works fine and prints all the data.
I am new to sqlite3. Please help. Thanks in advance
These are the links I used to come this far
Python sqlite3 string variable in execute
https://www.dummies.com/programming/databases/how-to-retrieve-data-from-specific-rows-in-mysql-databases/
When you do this
sql_command = "SELECT * FROM my_table WHERE id = "+id
The value of sql_command is
"SELECT * FROM my_table WHERE id = subject1"
As you can see, subject1 is not in quotes. sqlite thinks it is a column, that's why you see that error.
Instead, do this
sql_command = "SELECT * FROM my_table WHERE id = ?"
cur.execute(sql_command, [id])
? acts as a placeholder for the variable id.
The official sqlite3 documentation mentions few others methods
https://docs.python.org/2/library/sqlite3.html
The sql_command string being generated should be something like this (Formatted string):
sql_command = "SELECT * FROM my_table WHERE id = %s AND name = %s" % (212212, 'shashank')
When running the following code in python I am given the error: Incorrect number of bindings supplied. The current statement uses 0, and there are 1 supplied. For the query statement.
conn = sqlite3.connect('viber_messages2')
cur = conn.cursor()
cur = cur.execute("""SELECT DISTINCT messages.conversation_id
FROM messages
INNER JOIN participants ON messages.participant_id = participants._id
WHERE messages.conversation_id IS NOT NULL;""")
query = ("""SELECT messages._id, messages.body, messages.conversation_id, messages.participant_id, participants_info.number, participants_info.contact_name
FROM messages
INNER JOIN
participants ON messages.participant_id = participants._id
INNER JOIN
participants_info ON participants.participant_info_id = participants_info._id;""")
with open('messages.html', 'w') as h, open('test.txt', 'w') as t:
for convo in cur.fetchall():
df = pd.read_sql_query(query, conn, params=convo)
# HTML WRITE
h.write(df.to_html())
h.write('<br/>')
# TXT WRITE
t.write(df.to_string())
t.write('\n\n')
cur.close()
conn.close()
Let's take a close look at your query
SELECT messages._id, messages.body, messages.conversation_id, messages.participant_id, participants_info.number, participants_info.contact_name
FROM messages
INNER JOIN
participants ON messages.participant_id = participants._id
INNER JOIN
participants_info ON participants.participant_info_id = participants_info._id
There are no place holder here to which parameters can be bound. But how are you executing this query?
df = pd.read_sql_query(query, conn, params=convo)
You should instead do
df = pd.read_sql_query(query, conn)
I have 700 tables in a test.db file, and was wondering how do I loop through all these tables and return the table name if columnA value is -?
connection.execute('SELECT * FROM "all_tables" WHERE "columnA" = "-"')
How do I put all 700 tables in all_tables?
To continue on a theme:
import sqlite3
try:
conn = sqlite3.connect('/home/rolf/my.db')
except sqlite3.Error as e:
print('Db Not found', str(e))
db_list = []
mycursor = conn.cursor()
for db_name in mycursor.execute("SELECT name FROM sqlite_master WHERE type = 'table'"):
db_list.append(db_name)
for x in db_list:
print "Searching",x[0]
try:
mycursor.execute('SELECT * FROM '+x[0]+' WHERE columnA" = "-"')
stats = mycursor.fetchall()
for stat in stats:
print stat, "found in ", x
except sqlite3.Error as e:
continue
conn.close()
SQLite
get all tables name:
SELECT name FROM sqlite_master WHERE type='table' ORDER BY name;
Cycle
for table in tables:
...
connection.execute('SELECT * FROM "table1" WHERE "columnA" = "-"')
or one SQL request UNION
sql = []
for table in tables
sql.append('(SELECT * FROM "' + table + '" WHERE "columnA" = "-";)')
' UNION '.join(sql)
You could query the sqlite_master to get all the table names within your database: SELECT name FROM sqlite_master WHERE type = 'table'
sqlite_master can be thought of as a table that contains information about your databases (metadata).
A quick but most likely inefficient way (because it will be running 700 queries with 700 separate resultsets) to get the list of table names, loop through those tables and return data where columnA = "-":
for row in connection.execute('SELECT name FROM sqlite_master WHERE type = "table" ORDER BY name').fetchall()
for result in connection.execute('SELECT * FROM ' + row[1] + ' WHERE "columnA" = "-"').fetchall()
# do something with results
Note: Above code is untested but gives you an idea on how to approach this.
Issue: I can't figure out how to run a query in the correct way so that it returns a mapped dictionary. The query will use counts from multiple tables.
I am using psycopg2 for a postgresql database, and I will be using the results to create a report on day to day deltas on these counts.
Given that, can someone provide an example on how to execute multiple queries and return a dictionary that I can use for comparison purposes? Thanks! I image in a for loop is needed somewhere in here.
tables = ['table1', 'table2']
def db_query():
query = "select count(*) from (a_table) where error_string != '';"
conn = psycopg2.connect(database=db, user=user, password=password, host=host)
cur = conn.cursor(cursor_factory=psycopg2.extras.DictCursor)
cur.execute(query, tables)
output = cur.fetchall()
conn.close()
return output
I haven't used postgresql, so you might want to also check this out as a reference: How to store count values in python.
That being said, rearrange your code into something like this. Be sure to make conn global so you don't have to make more than one connection, and make sure you're also closing cur:
conn = None
def driverFunc():
global conn
try:
conn = psycopg2.connect(database=db, user=user, password=password, host=host)
tables = ['table1', 'table2']
countDict = {}
for thisTable in tables:
db_query(thisTable, countDict)
finally:
if not conn == None:
conn.close()
def db_query(tableName, countDict):
# Beware of SQL injection with the following line:
query = "select count(*) from " + tableName + " where error_string != '';"
cur = None
try:
cur = conn.cursor(cursor_factory=psycopg2.extras.DictCursor)
cur.execute(query)
countDict[tableName] = int(cur.fetchone())
finally:
if not cur == None:
cur.close()