I want to keep last.fm's user recent music tracks list to postgresql database table using pylast interface.But when I tried to insert values to the table it shows errors.Code example:
import pylast
import psycopg2
import re
from md5 import md5
import sys
import codecs
import psycopg2.extensions
psycopg2.extensions.register_type(psycopg2.extensions.UNICODE)
user_name = raw_input("Enter last.fm username: ")
user_password = raw_input("Enter last.fm password: ")
api_key = '*********'
api_secret = '********'
#Lastfm network authentication
md5_user_password = md5(user_password).hexdigest()
network = pylast.get_lastfm_network(api_key, api_secret,user_name,md5_user_password)
used=pylast.User(user_name, network)
recent_tracks=used.get_recent_tracks(10)
# Database connection
try:
conn=psycopg2.connect("dbname='**' user='postgres' host='localhost' password='*'")
conn.set_client_encoding('UNICODE')
except:
print "I am unable to connect to the database, exiting."
sys.exit()
cur=conn.cursor()
for i, artist in enumerate(recent_tracks):
for key in sorted(artist):
cur.execute("""
INSERT INTO u_recent_track(Playback_date,Time_stamp,Track)
VALUES (%s,%s,%s)""", (key, artist[key]))
conn.commit()
cur.execute("SELECT * FROM u_recent_track;")
cur.fetchone()
for row in cur:
print ' '.join(row[1:])
cur.close()
conn.close()
Here "recent_tracks" tuple have the values for example:
artist 0
- playback_date : 5 May 2010, 11:14
- timestamp : 1273058099
- track : Brian Eno - Web
I want to store these value under u_recent_track(Tid, Playback_date, Time_stamp, Track).Can anybody have idea how to sort out this problem? when I tried to run, it shows error:
Traceback (most recent call last):
File "F:\JavaWorkspace\Test\src\recent_track_database.py", line 50, in <module>
VALUES (%s,%s,%s)""", (key, artist[key]))
IndexError: tuple index out of range
sorted(artist) returns a ordered list of artist, when you're iterating over it it returns still elements of artist. So when you're trying to access artist[key] it is actually trying to access an element of artist indexed by the index, which is an element of artist itself. Tuples do not work this way.
It seems you're using python2.5 or lower and therefore you could do:
cur.executemany("""
INSERT INTO u_recent_track(Playback_date,Time_stamp,Track)
VALUES (%(playback_date)s,%(timestamp)s,%(track)s)""", recent_tracks)
conn.commit()
This should work.
This error isn't anything to do with Postgres, but with the artist variable. You're firstly saying:
for key in sorted(artist):
implying that it's a list, but then you're accessing it as if it were a dictionary, which is raising an error. Which is it? Can you show an example of the full contents?
(Playback_date,Time_stamp,Track) indicates you want to insert three values into a row.
VALUES (%s,%s) should therefore be VALUES (%s,%s,%s)
and (key, artist[key]) should be a tuple with 3 elements, not 2.
Try:
for track in recent_tracks:
cur.execute("""
INSERT INTO u_recent_track(Playback_date,Time_stamp,Track)
VALUES (%s,%s,%s)""", (track.get_date(), track.get_timestamp(), track.get_track()))
conn.commit()
PS. This is where I'm getting my information about the pylast API.
PPS. If my reading of the documentation is correct, track.get_track() will return a Track object. It has methods like get_album, get_artist, get_id and get_title. Exactly what do you want stored in the Track column of the u_recent_track database table?
Related
I selected column "user_list" in 'users' table and fetched to a python variable called "u_list". I appended 'item' in it and tried to update "user_list", but got a lot of errors. I tried searching on stackoverflow, but nothing helped.
code:
cursor.execute(f'SELECT user_list FROM users WHERE id=442392434899681280')
u_list = cursor.fetchone()[0]
u_list.append('item')
cursor.execute('UPDATE users SET user_list = {} WHERE id = 442392434899681280'.format(u_list))
data_base.commit()
but got an error:
Traceback (most recent call last):
File "d:\workspace\sabo\test.py", line 30, in <module>
cursor.execute('UPDATE users SET user_list = {} WHERE id = 442392434899681280'.format(u_list))
psycopg2.errors.SyntaxError: syntax error at or near "["
LINE 1: UPDATE users SET user_list = ['item'] WHERE id = 4423924348996...
Another try and error
code:
cursor.execute(f'SELECT user_list FROM users WHERE id=442392434899681280')
u_list = cursor.fetchone()[0]
u_list.append('item')
cursor.execute("UPDATE users SET user_list= (%s) WHERE id = 442392434899681280", (u_list))
data_base.commit()
error:
File "d:\workspace\sabo\test.py", line 33, in <module>
cursor.execute("UPDATE users SET user_list= (%s) WHERE id = 442392434899681280", (u_list))
psycopg2.errors.InvalidTextRepresentation: malformed array literal: "item"
LINE 1: UPDATE users SET user_list= ('item') WHERE id = 4423924348996...
^
DETAIL: Array value must start with "{" or dimension information.
Think about it like if you were typing the query yourself. As the error statement specifies, to PostgreSQL arrays must star with '{' so they will have to be within curly braces. If you were to write the query in SQL yourself it would look like this:
UPDATE users SET user_list = '{"foo", "bar", "item"}' WHERE id = 442392434899681280;
Handcrafted way
So in Python, it would have to be done like this:
cursor.execute(f'SELECT user_list FROM users WHERE id=442392434899681280')
u_list = cursor.fetchone()[0]
u_list.append('item')
cursor.execute('UPDATE users SET user_list = \'{{{}}}\' WHERE id = 442392434899681280'.format(','.join(['"{}"'.format(v) for v in u_list])))
data_base.commit()
Notice the three curly braces in the formatting, two are to escape so one '{' remains an the third is for the formatting. Also, if your list is not of strings you will have to convert it before joining.
Let psycopg2 handle it
psycopg2's docs also state that Python lists are converted to PostgreSQL ARRAY so the above would be done like like this:
cursor.execute("UPDATE users SET user_list= %s WHERE id = 442392434899681280", (u_list,))
You are missing a comma after the list and there is an extra parentheses surrounding %s in your code sample.
I'm wondering about if it's possible to get data from a database with this query: SELECT Name FROM fraleon WHERE Relay = "value from a string"?
When I try to put in a string in the query it comes with a message that the column doesn't exists.
I have just started with programming, so hope my question isn't to stupid ;-) I'm trying to make an app where i have a QtableWidget and a QScrollbar where the value from the scrollbar (1 - 100) is used to change to the next relay.
con = sqlite3.connect("C:\Leonclient\LocalDB\LeonDBlite.db")cur = con.cursor()
value = 5
for row in cur.execute('SELECT Name FROM fraleon where Relay = value'):
print(row)
Traceback (most recent call last):
File "C:\Users\xxx\AppData\Local\Temp/ipykernel_10292/3072186007.py", line 1, in <module>
for row in cur.execute('SELECT Name FROM fraleon where Relay = (Value)'):
OperationalError: no such column: Value
You may need to concatenate the query string with variable value otherwise it will be considered literally 'value'
for row in cur.execute('SELECT Name FROM fraleon where Relay = ' + str(value)):
You can try to write the query -
SELECT Name FROM fraleon WHERE Relay like '%somevalue%'
That way you will specify a sub-string of the Relay's column values
I am attempting to retrieve data associated given a particular last name. This last name, however, is stored within a listbox along with a first name and separator comma. As such, the name is retrieved from the listbox first by cursor selection, and only the last name is used to search by partitioning the rest:
lastname, sep, firstname = (self.patient_list.get(self.patient_list.curselection())).partition(',')
Once this is done, I am trying to get a printout of the data rows for this chosen last name. However, I am coming across the issue:
TypeError: argument 1 must be a string or unicode object: got tuple instead
I am wondering how to proceed with this issue. I have attempted a few solutions, such as the tuple function within the argument in load_query, as well as str on lastname, but it is a string...
I am also wondering if it is even necessary to select from a PostgreSQL db by this method. Can I try binding a listbox element to a row of data in the db? Or avoid partitioning the element, since that seems to be of trouble perhaps...
Full code:
def load_profile(self, event):
conn = pg.connect(user='postgres',
password='123!',
host='localhost',
port='5430',
database='carepartnerdb')
cur = conn.cursor()
#gets lastname from listbox, removes all else past the comma
#used to associate to DB
lastname, sep, firstname = (self.patient_list.get(self.patient_list.curselection())).partition(',')
load_query = (""" SELECT * FROM profiles_table WHERE patient_lastname=%s """, lastname)
cur.execute(load_query)
conn.commit()
#data = cur.fetchall()
#print(data)
cur.close()
conn.close()
load_query is a tuple, so cur.execute(load_query) will raise the exception as execute() expects a string (the query string) as the first argument.
You should change:
load_query = (""" SELECT * FROM profiles_table WHERE patient_lastname=%s """, lastname)
cur.execute(load_query)
to:
load_query = "SELECT * FROM profiles_table WHERE patient_lastname = %s"
cur.execute(load_query, (lastname,))
Also SELECT statement does not require commit().
I have been using Psycopg2 to read stored procedures from Postgres successfully and getting a nice tuple returned, which has been easy to deal with. For example...
def authenticate(user, password):
conn = psycopg2.connect("dbname=MyDB host=localhost port=5433 user=postgres password=mypwd")
cur = conn.cursor()
retrieved_pwd = None
retrieved_userid = None
retrieved_user = None
retrieved_teamname = None
cur.execute("""
select "email", "password", "userid", "teamname"
from "RegisteredUsers"
where "email" = '%s'
""" % user)
for row in cur:
print row
The row that prints would give me ('user#gmail.com ', '84894531656894hashedpassword5161651165 ', 36, 'test ')
However, when I run the following code to read a row of fixtures with a Stored Procedure, I get (what looks to me like) an unholy mess.
def get_from_sql(userid):
conn = psycopg2.connect("dbname=MyDB host=localhost port=5433 user=postgres password=pwd")
fixture_cursor = conn.cursor()
callproc_params = [userid]
fixture_cursor.execute("select sppresentedfixtures(%s)", callproc_params)
for row in fixture_cursor:
print row
The resulting output:
('(5,"2015-08-28 21:00:00","2015-08-20 08:00:00","2015-08-25 17:00:00","Team ",,"Team ",,"Final ")',)
I have researched the cursor class and cannot understand why it outputs like this for a stored procedure. When executing within Postgres, the output is in a perfect Tuple. Using Psycopg2 adds onto the tuple and I don't understand why?
How do I change this so I get a tidy tuple? What am I not understanding about the request that I am making that gives me this result?
I have tried the callproc function and get an equally unhelpful output. Any thoughts on this would be great.
This is because you're SELECTing the result of the function directly. Your function returns a set of things, and each "thing" happens to be a tuple, so you're getting a list of stringified tuples back. What you want is this:
SELECT * FROM sppresentedfixtures(...)
But this doesn't work, because you'll get the error:
ERROR: a column definition list is required for functions returning "record"
The solution is to return a table instead:
CREATE OR REPLACE FUNCTION sppresentedfixtures(useridentity integer) RETURNS TABLE(
Fixture_No int,
Fixture_Date timestamp,
...
) AS
$BODY$
select
"Fixtures"."Fixture_No",
"Fixtures"."Fixture_Date",
...
from "Fixtures" ...
$BODY$ LANGUAGE sql
I'd like to have returned to me (via cx_oracle in python) the value of the Identity that's created for a row that I'm inserting. I think I can figure out the python bit on my own, if someone could please state how to modify my SQL statement to get the ID of the newly-created row.
I have a table that's created with something like the following:
CREATE TABLE hypervisor
(
id NUMBER GENERATED BY DEFAULT AS IDENTITY (
START WITH 1 NOCACHE ORDER ) NOT NULL ,
name VARCHAR2 (50)
)
LOGGING ;
ALTER TABLE hypervisor ADD CONSTRAINT hypervisor_PK PRIMARY KEY ( id ) ;
And I have SQL that's similar to the following:
insert into hypervisor ( name ) values ('my hypervisor')
Is there an easy way to obtain the id of the newly inserted row? I'm happy to modify my SQL statement to have it returned, if that's possible.
Most of the google hits on this issue were for version 11 and below, which don't support automatically-generated identity columns so hopefully someone here can help out.
Taking what user2502422 said above and adding the python bit:
newest_id_wrapper = cursor.var(cx_Oracle.STRING)
sql_params = { "newest_id_sql_param" : newest_id_wrapper }
sql = "insert into hypervisor ( name ) values ('my hypervisor') " + \
"returning id into :python_var"
cursor.execute(sql, sql_params)
newest_id=newest_id_wrapper.getvalue()
This example taken from learncodeshare.net has helped me grasp the correct syntax.
cur = con.cursor()
new_id = cur.var(cx_Oracle.NUMBER)
statement = 'insert into cx_people(name, age, notes) values (:1, :2, :3) returning id into :4'
cur.execute(statement, ('Sandy', 31, 'I like horses', new_id))
sandy_id = new_id.getvalue()
pet_statement = 'insert into cx_pets (name, owner, type) values (:1, :2, :3)'
cur.execute(pet_statement, ('Big Red', sandy_id, 'horse'))
con.commit()
It's only slightly different from ragerdl's answer, but different enough to be added here I believe!
Notice the absence of sql_params = { "newest_id_sql_param" : newest_id_wrapper }
Use the returning clause of the insert statement.
insert into hypervisor (name ) values ('my hypervisor')
returning id into :python_var
You said you could handle the Python bit ? You should be able to "bind" the return parameter in your program.
I liked the answer by Marco Polo, but it is incomplete.
The answer from FelDev is good too but does not address named parameters.
Here is a more complete example from code I wrote with a simplified table (less fields). I have omitted code on how to set up a cursor since that is well documented elsewhere.
import cx_Oracle
INSERT_A_LOG = '''INSERT INTO A_LOG(A_KEY, REGION, DIR_NAME, FILENAME)
VALUES(A_KEY_Sequence.nextval, :REGION, :DIR_NAME, :FILENAME)
RETURNING A_KEY INTO :A_LOG_ID'''
CURSOR = None
class DataProcessor(Process):
# Other code for setting up connection to DB and storing it in CURSOR
def save_log_entry(self, row):
global CURSOR
# Oracle variable to hold value of last insert
log_var = CURSOR.var(cx_Oracle.NUMBER)
row['A_LOG_ID'] = log_var
row['REGION'] = 'R7' # Other entries set elsewhere
try:
# This will fail unless row.keys() =
# ['REGION', 'DIR_NAME', 'FILE_NAME', 'A_LOG_ID']
CURSOR.execute(INSERT_A_LOG, row)
except Exception as e:
row['REJCTN_CD'] = 'InsertFailed'
raise
# Get last inserted ID from Oracle for update
self.last_log_id = log_var.getvalue()
print('Insert id was {}'.format(self.last_log_id))
Agreeing with the older answers. However, depending on your version of cx_Oracle (7.0 and newer), var.getvalue() might return an array instead of a scalar.
This is to support multiple return values as stated in this comment.
Also note, that cx_Oracle is deprecated and has moved to oracledb now.
Example:
newId = cur.var(oracledb.NUMBER, outconverter=int)
sql = """insert into Locations(latitude, longitude) values (:latitude, :longitude) returning locationId into :newId"""
sqlParam = [latitude, longitude, newId]
cur.execute(sql, sqlParam)
newIdValue = newId.getvalue()
newIdValue would return [1] instead of 1