I had no problem with SELECTing data in python from postgres database using cursor/execute. Just changed the sql to INSERT a row but nothing is inserted to DB. Can anyone let me know what should be modified? A little confused because everything is the same except for the sql statement.
<!-- language: python -->
#app.route("/addcontact")
def addcontact():
# this connection/cursor setting showed no problem so far
conn = pg.connect(conn_str)
cur = conn.cursor(cursor_factory=psycopg2.extras.DictCursor)
sql = f"INSERT INTO jna (sid, phone, email) VALUES ('123','123','123')"
cur.execute(sql)
return redirect("/contacts")
first look at your table setup and make sure your variables are named right in the right order, format and all that, if your not logging into the specific database on the sql server it won't know where the table is, you might need to send something like 'USE databasename' before you do your insert statement so your computer is in the right place in the server.
I might not be up to date with the language but is that 'f' supposed to be right before the quotes? if thats in ur code that'd probably throw an error unless it has a use im not aware of or its not relevant to the problem.
You have to commit your transaction by adding the line below after execute(sql)
conn.commit()
Ref: Using INSERT with a PostgreSQL Database using Python
Related
I work with the Python mysql.connector for the first time and I am not able to create a working insert statement.
This is the table:
'CREATE TABLE IF NOT EXISTS products (id INT AUTO_INCREMENT PRIMARY KEY, title VARCHAR(255));'
I am trying to insert a variable as title while the id should be auto incremented. I have tried multiple solutions but it simply won't work.
def insert_product(title: str):
insert_product_query = 'INSERT INTO products (title) VALUES (%s);'
cursor.execute(insert_product_query, (title,))
This runs without any error, but the insert is not working. It does nothing. I tried multiple versions of this, with '?' instead of '%s' and without a tuple but it won't work.
Another solution I tried is this:
def insert_product(title: str):
insert_product_query = f'INSERT INTO products (title) VALUES (\'{title}\')'
print(insert_product_query)
cursor.execute(insert_product_query)
I printed the insert statement and when I copy paste it directly into the database it works perfectly, so I don't have any idea why it is not working out of the python code as it is not producing any errors.
I found many similar problems but none of the solution worked for me.
I hope someone can help me as I might overlook something obvious.
Thanks in advance!
Python's connector disables autocommit by default (as a reasonable library would do!). You need to explicitly commit after you perform a DML statement:
con.commit() # Assuming con is the name of the connection variable
I think I'm going mad here... again :). I'm trying to do the most simple thing on the planet and it doesn't work for some reason unknown to me. I have a python script that connects to a mssql database using pypyodbc and does stuff. when I insert data into the database, it works. when I try to extract it, it fails miserably. what am I doing wrong?
import pypyodbc as mssql
msConnErr = None
try:
msconn = mssql.connect('DRIVER={SQL Server};SERVER=server_name;DATABASE=database;TRUSTED_CONNECTION=True')
print('Source server connected')
srcCursor = msconn.cursor()
except:
print('Source server error')
msConnErr = True
srcCursor.execute("SELECT * FROM schema.table")
srcResult = srcCursor.fetchall()
print(srcResult)
the connection works as I'm being given a successful message. I can also see my script using sql server management studio being connected to the correct database, so I know I'm working in the right environment. the error I'm getting is:
UndefinedTable: relation "schema.table" does not exist
LINE 1: SELECT * FROM schema.table
the table exists, I must specify the schema as I have the same table name in different schemas (data lifecycle). I can extract data from it using sql server management studio, yet python fails miserably. it doesn't fail to insert 35 million rows in it using the same driver. no other query works, even SELECT ##VERSION fails, SELECT TOP (10) * FROM schema.table fails etc. ...
any ideas?
basically, I had a piece of code that would rewrite the srcCursor variable with another connection, obviously that relation wouldn't be present on another server. apologies!
I'm trying to populate a couple databases with psycopg2 within a server I am not the root user of (don't know if it's relevant or not). My code looks like
import json
from psycopg2 import connect
cors = connect(user='jungal01', dbname='course')
req = connect(user="jungal01", dbname='requirement')
core = cors.cursor()
reqs = req.cursor()
with open('gened.json') as gens:
geneds = json.load(gens)
for i in range(len(geneds)):
core.execute('''insert into course (number, description, title)
values({0}, {1}, {2});''' .format(geneds[i]["number"], geneds[i]['description'], geneds[i]['title'] ))
reqs.execute('''insert into requirement (fulfills)
values({0});''' .format(geneds[i]['fulfills'] ))
db.commit()
when I execute the code, I get the above pycopg2 error. I know that these particular databases exist, but I just can't figure out why it won't connect to my databases. (side quest, I am also unsure about that commit statement. Should it be in the for loop, or outside of it? It suppose to be database specific?)
First, you have db is not a defined variable, so you code shouldn't run completely anyway.
\list on this server is a bunch of databases full of usernames, of which my username is one
Then the following is how you should connect. To a database, not a table, and the regular pattern is to put the database name, and then the user/pass.
A "schema" is a loose term in relational database. Both tables and databases have schemas, but you seem to be expecting to connect to a table, not a database.
So, try this code with an attempt at fixing your indentation and SQL injection problem -- See this documentation
Note that you first must have created the two tables in the database you are connecting to.
import json
from psycopg2 import connect
username = 'jungal01'
conn = connect(dbname=username, user=username)
cur = conn.cursor()
with open('gened.json') as gens:
geneds = json.load(gens)
for g in geneds:
cur.execute('''insert into course (number, description, title)
values(%(number)s, %(description)s, %(title)s);''', g)
cur.execute('''insert into requirement (fulfills)
values(%(fulfills)s);''', g)
conn.commit()
Allen, you said: "in postgres, tables are databases." That's wrong. Your error message results from this misunderstanding. You want to connect to a database, and insert into a table that exists in that database. You're trying to insert into a database -- a nonsensical operation.
Make sure you are giving the catalog name as database name and not the schema's under catalog.
Catalog is confusing and quite unnecessary. More details below: What's the difference between a catalog and a schema in a relational database?
This isn't a question, so much as a pre-emptive answer. (I have gotten lots of help from this website & wanted to give back.)
I was struggling with a large bit of SQL query that was failing when I tried to run it via python using pymssql, but would run fine when directly through MS SQL. (E.g., in my case, I was using MS SQL Server Management Studio to run it outside of python.)
Then I finally discovered the problem: pymssql cannot handle temporary tables. At least not my version, which is still 1.0.1.
As proof, here is a snippet of my code, slightly altered to protect any IP issues:
conn = pymssql.connect(host=sqlServer, user=sqlID, password=sqlPwd, \
database=sqlDB)
cur = conn.cursor()
cur.execute(testQuery)
The above code FAILS (returns no data, to be specific, and spits the error "pymssql.OperationalError: No data available." if you call cur.fetchone() ) if I call it with testQuery defined as below:
testQuery = """
CREATE TABLE #TEST (
[sample_id] varchar (256)
,[blah] varchar (256) )
INSERT INTO #TEST
SELECT DISTINCT
[sample_id]
,[blah]
FROM [myTableOI]
WHERE [Shipment Type] in ('test')
SELECT * FROM #TEST
"""
However, it works fine if testQuery is defined as below.
testQuery = """
SELECT DISTINCT
[sample_id]
,[blah]
FROM [myTableOI]
WHERE [Shipment Type] in ('test')
"""
I did a Google search as well as a search within Stack Overflow, and couldn't find any information regarding the particular issue. I also looked under the pymssql documentation and FAQ, found at http://code.google.com/p/pymssql/wiki/FAQ, and did not see anything mentioning that temporary tables are not allowed. So I thought I'd add this "question".
Update: July 2016
The previously-accepted answer is no longer valid. The second "will NOT work" example does indeed work with pymssql 2.1.1 under Python 2.7.11 (once conn.autocommit(1) is replaced with conn.autocommit(True) to avoid "TypeError: Cannot convert int to bool").
For those who run across this question and might have similar problems, I thought I'd pass on what I'd learned since the original post. It turns out that you CAN use temporary tables in pymssql, but you have to be very careful in how you handle commits.
I'll first explain by example. The following code WILL work:
testQuery = """
CREATE TABLE #TEST (
[name] varchar(256)
,[age] int )
INSERT INTO #TEST
values ('Mike', 12)
,('someone else', 904)
"""
conn = pymssql.connect(host=sqlServer, user=sqlID, password=sqlPwd, \
database=sqlDB) ## obviously setting up proper variables here...
conn.autocommit(1)
cur = conn.cursor()
cur.execute(testQuery)
cur.execute("SELECT * FROM #TEST")
tmp = cur.fetchone()
tmp
This will then return the first item (a subsequent fetch will return the other):
('Mike', 12)
But the following will NOT work
testQuery = """
CREATE TABLE #TEST (
[name] varchar(256)
,[age] int )
INSERT INTO #TEST
values ('Mike', 12)
,('someone else', 904)
SELECT * FROM #TEST
"""
conn = pymssql.connect(host=sqlServer, user=sqlID, password=sqlPwd, \
database=sqlDB) ## obviously setting up proper variables here...
conn.autocommit(1)
cur = conn.cursor()
cur.execute(testQuery)
tmp = cur.fetchone()
tmp
This will fail saying "pymssql.OperationalError: No data available." The reason, as best I can tell, is that whether you have autocommit on or not, and whether you specifically make a commit yourself or not, all tables must explicitly be created AND COMMITTED before trying to read from them.
In the first case, you'll notice that there are two "cur.execute(...)" calls. The first one creates the temporary table. Upon finishing the "cur.execute()", since autocommit is turned on, the SQL script is committed, the temporary table is made. Then another cur.execute() is called to read from that table. In the second case, I attempt to create & read from the table "simultaneously" (at least in the mind of pymssql... it works fine in MS SQL Server Management Studio). Since the table has not previously been made & committed, I cannot query into it.
Wow... that was a hassle to discover, and it will be a hassle to adjust my code (developed on MS SQL Server Management Studio at first) so that it will work within a script. Oh well...
I'm trying to run a simple insert query to a database. I have it configured correctly, and it should work, but it doesn't. For some reason I get the following error on this query:
Query:
INSERT INTO searches (query) VALUES ('test')
Error:
(1062, "Duplicate entry 'test' for key 'query'")
The query runs without problems in the MySQL console so it must be a problem with Python? Here's my Python code:
def increase_search_count(search_query):
from django.db import connection, transaction
search_query = search_query.strip()
cursor = connection.cursor()
rows = cursor.execute("INSERT INTO searches (query) VALUES ('test')")
I know there are much better ways to handle databases, but I'm new to Python, and I have a deadline. I'd just like to get this to work, I have another SELECT query in another function and that one runs without any problems!
Any ideas what might be wrong?
The way that query is constructed means you will always be inserting 'test' into the database, and seeing the query is likely the primary key in your table, it will be creating duplicate rows.
The query should be something like "INSERT INTO searches (query) VALUES ('" variable "')" so you don't insert the same value over and over.