I want to update a row. So I issued the command -
conn = sqlite3.connect("W:\\webtracker\\database\\webtracker.db")
cur = conn.cursor()
cur.execute("SELECT URL FROM WebsiteDetail where screenshot_processed = false")
rows = cur.fetchall()
for row in rows:
cur.execute('''UPDATE WebsiteDetail SET screenshot_processed = true WHERE URL = ?''',(row[0],))
However, it doesnt update the value. What is the problem?
There is not error being thrown
row[0] is a string
You need to commit these changes. Your for loop should look like this:
for row in rows:
cur.execute('''UPDATE WebsiteDetail SET screenshot_processed = true WHERE URL = ?''',(row[0],))
conn.commit()
Related
I have a table with 100+ rows. Using Python I'm trying to get the first row preform some action on column A and update column B to true once done.
Then I would like to go to the next row and repeat this process.
current code
def _down():
params = config()
connection = psycopg2.connect(**params)
req = ("BEGIN;")
req2 = ("SELECT id FROM table_name WHERE col2 = False LIMIT 1 FOR UPDATE SKIP LOCKED;")
cursor = connection.cursor()
cursor.execute(req), cursor.execute(req2)
data = cursor.fetchall()
connection.commit()
cursor.close(), connection.close()
I have a query that updates column b to true but it does it for the entire column rather than the specific row
req3 = ("update tablename set col2 = True")
I am trying to capture the only the record from a PostgreSQL statement. The select statements outputs one row with column named as updated_at and the value is a timestamp- '2008-01-01 00:50:01'. I want to just capture/collect that value so when I call that variable, it just outputs '2008-01-01 00:50:01'.
Here is my code:
def get_etl_record():
pg_hook = PostgresHook(postgre_conn_id="post", schema='schema1')
connection = pg_hook.get_conn()
cursor = connection.cursor()
cursor2 = connection.cursor()
latest_update_query = "select max(updated_at) from my_table group by updated_at"
cursor.execute(latest_update_query)
#results= cursor.fetchall()
columns = [col[0] for col in cursor.description]
rows = [dict(zip(columns, row[0])) for row in cursor.fetchall()]
print(rows)
However this code doesnt give me an output.
Any ideas or suggestions?
There is no way to do what you want, but 3 ways to do very similar:
1.
cursor = connection.cursor()
cursor.execute(sql)
result = cursor.fetchone()
max_updated_at = result[0]
2.
dict_cur = connection.cursor(cursor_factory=psycopg2.extras.DictCursor)
dict_cur.execute('select max(updated_at) as max_updated_at ...')
result = dict_cur.fetchone()
max_updated_at = result['max_updated_at']
3.
nt_cur = connection.cursor(cursor_factory=psycopg2.extras.NamedTupleCursor)
nt_cur.execute('select max(updated_at) as max_updated_at ...')
result = nt_cur.fetchone()
max_updated_at = result.max_updated_at
I am trying to fetch records after a regular interval from a database table which growing with records. I am using Python and its pyodbc package to carry out the fetching of records. While fetching, how can I point the cursor to the next row of the row which was read/fetched last so that with every fetch I can only get the new set of records inserted.
To explain more,
my table has 100 records and they are fetched.
after an interval the table has 200 records and I want to fetch rows from 101 to 200. And so on.
Is there a way with pyodbc cursor?
Or any other suggestion would be very helpful.
Below is the code I am trying:
#!/usr/bin/python
import pyodbc
import csv
import time
conn_str = (
"DRIVER={PostgreSQL Unicode};"
"DATABASE=postgres;"
"UID=userid;"
"PWD=database;"
"SERVER=localhost;"
"PORT=5432;"
)
conn = pyodbc.connect(conn_str)
cursor = conn.cursor()
def fetch_table(**kwargs):
qry = kwargs['qrystr']
try:
#cursor = conn.cursor()
cursor.execute(qry)
all_rows = cursor.fetchall()
rowcnt = cursor.rowcount
rownum = cursor.description
#return (rowcnt, rownum)
return all_rows
except pyodbc.ProgrammingError as e:
print ("Exception occured as :", type(e) , e)
def poll_db():
for i in [1, 2]:
stmt = "select * from my_database_table"
rows = fetch_table(qrystr = stmt)
print("***** For i = " , i , "******")
for r in rows:
print("ROW-> ", r)
time.sleep(10)
poll_db()
conn.close()
I don't think you can use pyodbc, or any other odbc package, to find "new" rows. But if there is a 'timestamp' column in your database, or if you can add such a column (some databases allow for it to be automatically populated as the time of insertion so you don't have to change the insert queries) then you can change your query to select only the rows whose timestamp is greater than the previous timestamp. And you can keep changing the prev_timestamp variable on each iteration.
def poll_db():
prev_timestamp = ""
for i in [1, 2]:
if prev_timestamp == "":
stmt = "select * from my_database_table"
else:
# convert your timestamp str to match the database's format
stmt = "select * from my_database_table where timestamp > " + str(prev_timestamp)
rows = fetch_table(qrystr = stmt)
prev_timestamp = datetime.datetime.now()
print("***** For i = " , i , "******")
for r in rows:
print("ROW-> ", r)
time.sleep(10)
import sqlite3
#connect to the sqlite database
conn = sqlite3.connect('database.db')
#create a cursor
c = conn.cursor()
#select query to return a single row
c.execute('SELECT NAME FROM T1')
#row contains the returned result
row = c.fetchone()
#print the result
print(row)
It prints something like --> (u'John',), but I only want John
You are printing the whole row, which is always going to be a tuple.
If you wanted to print just the first column, use subscription:
print(row[0])
I'm having a problem while trying to simply execute data from rows from db (sqlite3). The DB input has 4 fields, therefore once entered they're being saved. But here's my problem, where I execute all of the 4 rows, if one of the fields was not filled I get an error.
That's the database execute code:
def ids(self):
con = lite.connect('foo.db')
with con:
cur = con.cursor()
cur.execute("SELECT Id FROM foo")
while True:
ids = cur.fetchall()
if ids == None:
continue
return ids
And since there are 4 rows, my output code:
print ''.join(ids[0]) + ',' + ''.join(ids[1]) + ',' + ''.join(ids[2])
+ ',' + ''.join(ids[3])
so my question is how to make an exception when there's no existing row to not show anything and just leave the ones that actually exist? I tried doing if ids[0] is not None: #do something but that would make my code really slow and it's non-pythonic way I guess. Is there any better way to make that work? Any help will be appreciated.
You don't seem to have 4 rows. Make it generic and just join an arbitrary number of rows:
ids = someobject.ids()
print ','.join(''.join(row) for row in ids)
You can simplify your database query, there is no need to 'poll' the query:
def ids(self):
with lite.connect('foo.db') as con:
cur = con.cursor()
cur.execute("SELECT Id FROM foo")
return cur.fetchall()
You could also just loop directly over the cursor, the database will handle buffering as you fetch:
def ids(self):
with lite.connect('foo.db') as con:
cur = con.cursor()
cur.execute("SELECT Id FROM foo")
return cur # just the cursor, no fetching
ids = someobject.ids()
# this'll loop over the cursor, which yields rows as required
print ','.join(''.join(row) for row in ids)