I am trying to insert into a postgresql database in python 3.6 and currently am trying to execute this line
cur.execute("INSERT INTO "+table_name+"(price, buy, sell, timestamp) VALUES (%s, %s, %s, %s)",(exchange_rate, buy_rate, sell_rate, date))
but every time it tries to run the table name has ' ' around it so it turns out like INSERT INTO table_name('12', ..., ..., ...) ... instead of
INSERT INTO table_name(12, ..., ..., ...) ... how can I make the string formatter leave the quotes out or remove them or something? It is causing a syntax error around the 12 because it doesn't need the single quotes.
Use it with triple quotes. Also you may pass table_name as a element of second parameter, too.
cur.execute("""INSERT INTO %s (price, buy, sell, timestamp) VALUES (%s, %s, %s, %s)""",(table_name, exchange_rate, buy_rate, sell_rate, date))
More detailed approach;
Triple qoutes give developers a change to write SQL query as multi-lines.
Also it allows you to use single and double qoutes without escaping from them. (It is beneficiary for complex SQL Queries but you don't need that on your case)
Use the new string formatting to have a clean representation. %s is explicitly converting to a string, you don't want that. Format chooses the most fitting type for you.
table_name = "myTable"
exchange_rate = 1
buy_rate = 2
sell_rate = 3
date = 123
x = "INSERT INTO {0} (price, buy, sell, timestamp) VALUES ({1}, {2}, {2}, {4})".format(
table_name, exchange_rate, buy_rate, sell_rate, date)
print x
>INSERT INTO myTable (price, buy, sell, timestamp) VALUES (1, 2, 2, 123)
Related
I'm using python's psycopg2 to create a table and the insert SQL string look like this
create_table ='''CREATE TABLE Datetime_Response(
rid BIGINT NOT NULL,
qid BIGINT,
roid BIGINT,
rotext VARCHAR(20)
)'''
INSERT INTO Datetime_Response (rid, qid, roid, rotext)
VALUES
(13020638659, 711799502, 4681912759, 07/21/2021)
But the output is weird. All the datetime string becomes 0. I tried both VARCHAR and TEST in the column rotext. They all show 0. I don't know what goes wrong.
(13020638659, 711799502, 4681912759, '0')
This is what the values looks like
values = list(entry.values())
print(values)
['13020638659', '711799502', '4681912759', '07/21/2021']
And this is what the insert syntax look like
values_str = "(%s)" % (', '.join( values ))
sql_string = "INSERT INTO %s (%s)\nVALUES\n %s" % (
table_name,
', '.join(columns),
values_str
)
print(sql_string)
INSERT INTO GZ_Datetime_Response (rid, qid, roid, rotext)
VALUES
(13020638659, 711799502, 4681912759, 07/21/2021)
Do it like this. Let the connector fill in the properly quoted values:
sql_string = "INSERT INTO %s (%s)\nVALUES (%s)" % (
table_name,
','.join(columns),
','.join(['?']*len(values))
)
cur.execute( sql_string, values )
Because 7 divided by 21 divided by 2021 is close to zero. / is integer division in Python 2 if both operands are an integer. Quote your value: 07/21/2021
Your query is open to SQL injection attacks by malicious users of your script!
values = list(entry.values())
print(values)
['13020638659', '711799502', '4681912759', '07/21/2021']
values_str = "(%s)" % (', '.join( values ))
sql_string = "INSERT INTO %s (%s)\nVALUES\n %s" % (
table_name,
', '.join(columns),
values_str
)
print(sql_string)
When using your string values to format your string, you will end up with:
INSERT INTO GZ_Datetime_Response (rid, qid, roid, rotext)
VALUES (13020638659, 711799502, 4681912759, 07/21/2021)
which does not contain any quotation marks. Use prepared statements to avoid problems caused by wrong types and to stop hackers from misusing your application/database.
If, for whatever reason, you cannot use prepared statements, make sure the string ends up as string in your final statement:
values = ['13020638659', '711799502', '4681912759', "'07/21/2021'"]
or INSERT INTO GZ_Datetime_Response (rid, qid, roid, rotext) VALUES (%s, %s, %s, '%s)
I can iterate through a python object with te following code, however I would like to be able to use placeholders for the schema and table name, normally I do this with {}.{} ad the .format() methods, but how do you combine the two?
cur.executemany("INSERT INTO schema.table_name (x,y,z) "
"values (%s, %s, %s)", top_sample)
Depends on which python you use you can try use f-string
schema = "schema"
table_name = "table_name"
cur.executemany(f"INSERT INTO {schema}.{table_name} (x,y,z) values (%s, %s, %s)", top_sample)
check PEP 498 -- Literal String Interpolation
another option is a simple format
cur.executemany("INSERT INTO {schema}.{table_name} (x,y,z) values (%s, %s, %s)".format(schema=schema, table_name=table_name), top_sample)
but I find the first option shorter and cleaner
I'm not sure what the issue is. You can very well use format like this:
cur.executemany("INSERT INTO {}.{} (x,y,z) values (%s, %s, %s)".format('hello', 'world'), top_sample)
cur.executemany(
"""INSERT INTO schema.{table_name} (x,y,z) values (%s, %s, %s)""".format(table_name=your_table_name),
top_sample
)
place your table name in place of your_table_name
I am inserting data into a Postgres DB using Psycopg2 using:
params = "ts, price, volume"
entry = (1234567, None, 3)
cur.execute(f"INSERT INTO stock_data ({params}) VALUES {entry};")
I have used this pattern of f-string with params and a tuple entry countless times with no issue.
For additional information, the price column does not have any constraints and is a numeric(5).
For this particular instance, it is giving me the error:
psycopg2.ProgrammingError: column "none" does not exist
Everything I tried resulted in this error. What did end up working was not using an f-string. The following entry worked:
cur.execute(f"INSERT INTO stock_data ({params}) VALUES (%s, %s, %s);", entry)
My understanding is that these are identical. Why would the latter work, but including the entry directly into the INSERT statement not work?
Format just inserts the value into a string:
>>> x = None
>>> f"What should I expect here? {x}"
'What should I expect here? None'
You pass the already formatted string to cur.execute(). When you pass a query text with placeholders and arguments, the function knows how to convert them.
>>> cur.mogrify(f"INSERT INTO stock_data ({params}) VALUES {entry};")
b'INSERT INTO stock_data (ts, price, volume) VALUES (1234567, None, 3);'
>>> cur.mogrify(f"INSERT INTO stock_data ({params}) VALUES (%s, %s, %s);", entry)
b'INSERT INTO stock_data (ts, price, volume) VALUES (1234567, NULL, 3);'
It would be best if you always used placeholders and arguments in cursor.execute(), also for your own safety. Read Passing parameters to SQL queries, where you can find this
Warning
Never, never, NEVER use Python string concatenation (+) or string parameters interpolation (%) to pass variables to a SQL query string. Not even at gunpoint.
Please consider me to be a complete novice with psycopg2. My aim is to insert a 1D numpy array of dtype object (where the elements are only strings) into a postgresQL table. My main program saves the fields as strings in the numpy array. I then want to add each separate element to a column in the postgresqL table (or if you prefer, the the 1d Array is one row). Please note, the actual array has 36 elements! I need a method to put them all in.
I am using the cur.execute command although I believe there is some problem with the string conversion.
Array=np.empty(3,dype=object)
Array[0]='Hello'
Array[1]='Tea?'
Array[2]='Bye'
statement= "INSERT INTO testlog (field1,field2,field3) VALUES (%s)" #Etc.
cur.execute(statement,Array)
I get error:
cur.execute(statement,Array)
TypeError: not all arguments converted during string formatting
Also tried:
cur.executemany('INSERT INTO testlog VALUES ( %s )', [[v] for v in Array ]
Thanks
Your statement should contain place holder for all values:
statement= "INSERT INTO testlog (field1,field2,field3) VALUES (%s, %s, %s)"
For Example:
=# create table testlog (field1 varchar(50), field2 varchar(50), field3 varchar(50))`;
then in the python shell (note dtype not dype:
Array=np.empty(3,dtype=object)
Array[0]='Hello'
Array[1]='Tea?'
Array[2]='Bye'
sql = "INSERT INTO testlog (field1, field2, field3) VALUES (%s, %s, %s)"
cur.execute(sql, [f for f in Array])
conn.commit()
And in DB:
select * from testlog;
field1 | field2 | field3
--------+--------+--------
Hello | Tea? | Bye
(1 row)
I have a problem with MySQL and Python's MySQLdb when I try to INSERT more than one variable.
I have a table wordurl with three fields. The first one is an auto_increment ID, second and third should hold the values. Second and third fields are named word_id and url_id.
This is the code.
cursor.execute("INSERT INTO wordurl (word_id, url_id) VALUES (%s, %s)", (word_temp_id, url_temp_id))
When I try to INSERT only one value the code works, two not.
Error message:
(1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '),), (('2',),))' at line 1")
I also tried it with tripple ticks around the statement, the variables with and without bracket, without the field names and with the first id field included. I also tried with C-style-printf-stuff % (which isn't clever!). Nothing worked.
And you, glorious people on stackoverflow, you are my last hope :)
MySQL-Server is 5.5.9 on FreeBSD 8.2. Python version is 2.7:82508 on OSX Lion
Thanks in advance!
Steffen
UPDATE: I use cursor.fetchall() and cursor.fetchone() to get the IDs. Maybe this information is important.
Regarding your update:
AFAIK:
fetchall() returns a tuple with tuples inside. Those tuples inside are what you would get from fetchone(). You can image it like
fetchall() = (fetchone(), fetchtwo(), ...)
And fetchone also returns a tuple which has the actual variable values. This is why you can't simply insert the return value of fetchall().
Just to clarify: the insert statement you pasted, look like this when you fill in the value for max_price:
c.execute("""SELECT spam, eggs, sausage FROM breakfast WHERE price < %s""", (5,))
so the first %s is replaced with the first element of the tuple (5). Your insert statement looks like this:
cursor.execute("INSERT INTO wordurl (word_id, url_id) VALUES (%s, %s)", (((233L,),), ((3L,),)))
I can't make it more clear.
Did you try this:
cursor.execute("INSERT INTO wordurl (word_id, url_id) VALUES ('%s', '%s')" %(word_temp_id, url_temp_id))
It depends on what variables you're trying to insert, but on strings this is needed.
Notice the difference:
>>> word_temp_id = "bla1"
>>> url_temp_id = "bla2"
>>> query = "INSERT INTO wordurl (word_id, url_id) VALUES (%s, %s)" %(word_temp_id, url_temp_id)
>>> print query
INSERT INTO wordurl (word_id, url_id) VALUES (bla1, bla2)
>>> query = "INSERT INTO wordurl (word_id, url_id) VALUES ('%s', '%s')" %(word_temp_id, url_temp_id)
>>> print query
INSERT INTO wordurl (word_id, url_id) VALUES ('bla1', 'bla2')