Update jsonb field in PostgreSQL with json that contains single quote - python

I'm using PostgreSQL v9.4.12 and I'm trying to update a jsonb column. I want to update the whole json object and not a specific key of the object.
I'm using a Python dict to store my object and before using it I'm using json.dumps() to transform it to a json formatted String.
However, a value of the json is having a single quote ' that throws an psycopg2.ProgrammingError: syntax error while trying to update.
So far, I've tried:
"UPDATE table "
"SET jsonb_column='{} ".format(json.dumps(new_data)) + ""
"WHERE id='12345'"
Note that new_data is my dict and jsonb_column is the name of the column holding the json data.
The error I'm getting:
psycopg2.ProgrammingError: syntax error at or near "s"
LINE 1: ...code": "BR3", "short_description": "This property's price
is...
^
I was assuming that json.dumps() escapes the single quote but doesn't seem that to be the case. Is there any solution to overcome this error?
Thanks in advance.

json is very fine with single quote, eg:
t=# select $${"short_description": "This property's price is..."}$$::jsonb;
jsonb
------------------------------------------------------
{"short_description": "This property's price is..."}
(1 row)
so I assume you could try using dollar sign quotes, to avoid statement structuring exception with single quotes

The practice of string concatenation is not a good practice.
Better use the way documented in PsyCoPg2 docs.
cur.execute("UPDATE table SET jsonb_column = %s WHERE id = %s", [json, id])

Related

psycopg2 issues with inserting a prepared JSONB object into postgres

I have a JSONB object that I want to insert into my database that also contains single quote characters. The example below is just trimmed to highlight my issue:
[{"name": "Moody's Analytics BBB Corp Bd ETF"}]
I have tried just dumping that JSON object to a string and inserting it, but the single ' wont allow that to work and when I try to execute (where detail is the JSON object):
cur.execute("INSERT INTO schmea.table (id, detail) VALUES (?,?)", (sec_id, detail)
I get the following syntax error:
psycopg2.errors.SyntaxError: syntax error at or near ","
LINE 1: INSERT INTO schema.table (id, detail) VALUES (?,?)
^
All of the solutions I've found so far suggest turning the JSON object into a string using json.dumps() but that wont work in my situation because of the single ' character in the name.
Any help is appreciated here.
Edit: Was told that the proper bind is %s not ? but that changes the error to:
psycopg2.ProgrammingError: can't adapt type 'dict'
edit 2:
SOLUTION:
Using the correct %s binds and using json.dumps(detail) fixed the issue.
cur.execute("INSERT INTO schmea.table (id, detail) VALUES (%s,%s)", (sec_id, json.dumps(detail))
Thanks for your help!
The way to pass bind variables to psycopg2 is with %s, not ?.

Error in Sqlite while Updating the table using Python

I am using a python to update entries in Sqlite table.
The command I am using is:
handle.execute("UPDATE RECORD set NAME=%s DEVICE=%s PROJECT=%s IP=%s COMMENT=%s where ID = %s"%(arg[2],arg[3],arg[4],arg[5],arg[6],arg[1]))
To this I get am getting an error as:
sqlite3.OperationalError: near "DEVICE": syntax error
I cannot understand what is specifically wrong with Device. Also I have checked the variables are as expected. The data base has a column named device and the database can be opened / accessed and edited using this python file.
There are commas missing between set items.
In addition to that, instead of string formatting, pass parameters to prevent SQL injection:
handle.execute(
"""UPDATE RECORD
SET NAME=%s, DEVICE=%s, PROJECT=%s, IP=%s, COMMENT=%s
WHERE ID = %s""",
(arg[2], arg[3], arg[4], arg[5], arg[6], arg[1]))
UPDATE
If you insist to use string formatting, you should quote %s: '%s'

python- Insert query to store regex expression in mysql database

Insert query for string having \" character in it in mysql db-
How to write insert query for string such as:
This is the string i want to insert into my table,
reg="item-cell\"(.*?)</span></div>"
cur = db.cursor()
query='INSERT into table_name(col_name) values("%s")'%(reg)
cur.execute(query)
cur.close()
Below is the error:
_mysql_exceptions.ProgrammingError: (1064, 'You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near \'(.*?)</span></div>")\' at line 1')
I know its something related to escape character, but don't know how to make that work.
EDIT: This string reg is variable i.e. I am getting this string from some API and I want to insert it into my database. So inserting escape characters in between the string literal will not suffice my case. I want something that can generalize single quote, double quote or one double quote(eg. reg) all these cases.
I hope I made my point clear.
EDIT: This is how i am getting the value of reg(froma json file)
import urllib, json
import MySQLdb
url = "some_url"
response = urllib.urlopen(url)
data = json.loads(response.read())
for item in data["key1"]["key2"]["key3"]["key4"]:
prop=str(item)
reg=str(data["key1"]["key2"]["key3"]["key4"][prop]["regex"])
The problem is in the part where you convert the json object to string. To properly do this without altering the string you need to use json.dumps
for item in data["key1"]["key2"]["key3"]["key4"]:
prop = json.dumps(item)
reg = json.dumps(data["key1"]["key2"]["key3"]["key4"][prop]["regex"])

pymysql query formatting, TypeError: %d format: a number is required, not str

So I was writing a sql query to get data from the database using python pymysql library. The user will enter a year as a text field in HTML form and display the data associated with this year. I'm using this code and I'm keep getting error for it.
sql = "SELECT 'Name' From 'winners' WHERE 'Year'=%d"
c.execute(sql, (textfield,))
Can someone show me how to fix this please ? Thank you
There are two issues:
You are using incorrect syntax. ' single quotes are used to define string values, not to quote table and column names. You probably meant to use backticks, `. See When to use single quotes, double quotes, and backticks?
You are passing in a string where your query parameter states you wanted to receive an integer. SQL parameters only support %s format placeholders, however.
Corrected code would look like this:
sql = "SELECT `Name` From `winners` WHERE `Year`=%s"
c.execute(sql, (int(textfield),))
Note the int() call; you want to select on integers, not strings, if your column type is NUMBER or similar.

Given table and column name, how to test if INSERT needs quotes ('') around the values to be inserted?

I have a dictionary of column name / values, to insert into a table. I have a function that generates the INSERT statement. I'm stuck because the function always puts quotes around the values, and some are integers.
e.g. If column 1 is type integer then the statement should be INSERT INTO myTable (col1) VALUES 5; vs
INSERT INTO myTable (col1) VALUES '5'; second one causes an error saying column 5 does not exist.
EDIT: I found the problem (I think). the value was in double quotes not single, so it was "5".
In Python, given a table and column name, how can I test if the INSERT statement needs to have '' around the VALUES ?
This question was tagged with "psycopg2" -- you can prepare the statement using a format string and have psycopg2 infer types for you in many cases.
cur.execute('INSERT INTO myTable (col1, col2) VALUES (%s, %s);', (5, 'abc'))
psycopg2 will deal with it for you, because Python knows that 5 is an integer and 'abc' is a string.
http://initd.org/psycopg/docs/usage.html#passing-parameters-to-sql-queries
You certainly want to use a library function to decide whether or not to quote values you insert. If you are inserting anything input by a user, writing your own quoting function can lead to SQL Injection attacks.
It appears from your tags that you're using psycopg2 - I've found another response that may be able to answer your question, since I'm not familiar with that library. The main gist seems to be that you should use
cursor.execute("query with params %s %s", ("param1", "pa'ram2"))
Which will automatically handle any quoting needed for param1 and param2.
Although I personally don't like the idea, you can use single quotes around integers when you insert in Postgres.
Perhaps your problem is the lack of parentheses:
INSERT INTO myTable(col1)
VALUES('5');
Here is a SQL Fiddle illustrating this code.
As you note in the comments, double quotes do not work in Postgres.
You can put always the single quote (be careful, if the value contents a quote you must double it: insert into example (value_t) values ('O''Hara');
You can decide checking the value that you want to insert regardles of the type of de destination
You can decide checking the type of the target field
As you can see in http://sqlfiddle.com/#!15/8bfbd/3 theres no mater with inserting integers into a text field or string that represents an integer in a numeric field.
To check the field type you can use the information_schema:
select data_type from information_schema.columns
where table_schema='public'
and table_name='example'
and column_name='value_i';
http://sqlfiddle.com/#!15/8bfbd/7

Categories