Unable to INSERT with Pymysql (incremental id changes though) - python

When I'm using pymysql to perform operations on MySQL database, it seems that all the operations are temporary and only visible to the pymysql connection, which means I can only see the changes through cur.execute('select * from qiushi') and once I cur.close() and conn.close() and log back in using pymysql, everything seems unchanged.
However, when I'm looking at the incremental id numbers, it does increased, but I can't see the rows that were inserted from pymysql connection. It seems that they were automatically deleted?!
Some of my code is here:
import pymysql
try:
conn = pymysql.connect(host='127.0.0.1',port=3306,user='pymysql',passwd='pymysql',charset='utf8')
cur = conn.cursor()
#cur.execute('CREATE TABLE qiushi (id INT NOT NULL AUTO_INCREMENT, content_id BIGINT(10) NOT NULL, content VARCHAR(1000), created TIMESTAMP DEFAULT CURRENT_TIMESTAMP, PRIMARY KEY(id));')
#cur.execute('DESCRIBE content')
#cur.fetchall()
cur.execute('USE qiushibaike')
for _ in range(0,len(content_ids)):
cur.execute("INSERT INTO qiushi (content,content_id) VALUES (\"%s\",%d)"%(jokes[_],int(content_ids[_])))
finally:
cur.close()
conn.close()

I solved the problem by myself...
Because the config is automatically committed, so after each SQL sentence we should commit the changes.
Approach 1:
add cur.commit() after the cur.execute()
Approach 2:
edit the connection config, add autocommit=True

Related

Select MySQL Database in Python

I know I can call a database in the mysql.connector.connect method. However, I'm trying to programmatically create the database if it doesn't already exist, then connect too to upload table data.
Currently my line cursor.execute("SELECT DATABASE myDatabase") is generating an error in my "SQL syntax".
How can I check if a database exists and then connect to it?
db = mysql.connector.connect(
host="localhost",
user=os.environ.get('MYSQL_DB_USER'),
password=os.environ.get('MYSQL_DB_USER_PASS')
)
cursor = db.cursor()
cursor.execute("CREATE DATABASE IF NOT EXISTS myDatabase")
cursor.execute("SELECT DATABASE myDatabase")
# create table on database
sql = "CREATE TABLE IF NOT EXISTS `trades` (`id` INT(11) UNSIGNED NOT NULL AUTO_INCREMENT, `objectid` VARCHAR(100), `type` VARCHAR(5), `ownerid` VARCHAR(100), `color` VARCHAR(15), `shape` VARCHAR(15), `size` VARCHAR(15), PRIMARY KEY (`id`) );"
print(sql)
cursor.execute(sql)
Alternatively, the database property can be used of the MySQLConnection object, as per the MySQL Connector documentation (which executes the USE command for you):
db = mysql.connector.connect(
host="localhost",
user=os.environ.get('MYSQL_DB_USER'),
password=os.environ.get('MYSQL_DB_USER_PASS')
)
cursor = db.cursor()
cursor.execute("CREATE DATABASE IF NOT EXISTS myDatabase")
db.database = 'myDatabase'
(I was dealing with the same issue and stumbled upon this post, after which I found the described solution. So thought I'd share this alternative solution for anyone interested.)
Per MySQL documentation, the syntax is:
CREATE DATABASE `db_name`;
Then to select/change the database, use:
USE `db_name`;
To list all available databases, use:
SHOW DATABASES;
Alternatively, you can skip the USE command and reference the database in the statement itself as:
CREATE TABLE `db_name`.`table1` ... ;
Or, in the case of a SELECT statement:
SELECT * FROM `db_name`.`table1`;

MySQL python connector code aborts without error (Code from the MySQL documentation)

I was trying to use the python connector code given in the MySQL documentation and test it on a small database already created, but it aborts. The code is just supposed to connect to the db and add a new email adress.
import mysql.connector
from mysql.connector import errorcode
cnx = mysql.connector.connect(user='root', password='pwd', host='localhost', database='db')
cursor = cnx.cursor()
add_email = ("INSERT INTO employee(MID, Email) VALUES (%s,%s)")
email_details = (NULL, "a#a.de")
cursor.execute(add_email, email_details)
cnx.commit()
input("data entered successfully")
cnx.close()
By setting breakpoints I found out that the problem probably lies in the cursor.execute() statement. (I used Null as the first %s since MID is Auto Incrementing btw)
To solve this problem NULL (for the autoincrementing "MID") needs to be replaced with None.

PostgreSQL relation doesn't exist (Python)

I created a database in psql and in it, created a table called "tweet".
CREATE TABLE tweet
( tid CHARACTER VARYING NOT NULL, DATA json,
CONSTRAINT tid_pkey PRIMARY KEY (tid) );
Then when I use
SELECT * FROM tweet;
in the psql window it works and shows an empty table.
Now I have a python script that takes JSON data and is loading it into this table.
conn_string = "host='localhost' port=5432 dbname='tweetsql' user='tweetsql' password='tweetsql'"
conn = psycopg2.connect(conn_string)
cur = conn.cursor()
That sets up the connection and I don't think it had any issues.
Now I have some logic to read in the JSON file and then to add it in, I say:
cur.execute("INSERT INTO tweet (tid, data) VALUES (%s, %s)", (cur_tweet['id'], json.dumps(cur_tweet, cls=DecimalEncoder), ))
But this always says that the relation tweet doesn't exist. Am I missing something here? Is there an issue with my connection or can my script somehow not see the table? For reference I'm using psycopg2 for the connection.
EDIT: I updated the DDL to include a transaction I could commit but that didn't fix it either. Is it a schema issue?
This is what I did regarding the table creation to commit:
BEGIN;
CREATE TABLE tweet
( tid CHARACTER VARYING NOT NULL, DATA json,
CONSTRAINT tid_pkey PRIMARY KEY (tid) );
COMMIT;
EDIT 2: I'm posting some code here...
import psycopg2
import json
import decimal
import os
import ctypes
conn_string = "host='localhost' port=5432 dbname='tweetsql' user='tweetsql' password='tweetsql'"
conn = psycopg2.connect(conn_string)
cur = conn.cursor()
cur.execute("CREATE TABLE tweet (tid CHARACTER VARYING NOT NULL, DATA json, CONSTRAINT tid_pkey PRIMARY KEY (tid) );")
cur.commit()
for file in os.listdir(path):
if not is_hidden(file):
with open(path+file, encoding='utf-8') as json_file:
tweets = json.load(json_file, parse_float=decimal.Decimal)
for cur_tweet in tweets:
cur.execute("INSERT INTO tweet (tid, data) VALUES (%s, %s)", (cur_tweet['id'], json.dumps(cur_tweet, cls=DecimalEncoder), ))
cur.commit()
cur.close()
conn.close()
You're probably not committing the table creation, and, (I'm assuming; not seeing your complete code) you're starting a new connection via psycopg2 each time. You need to commit right after the table creation, and not in a new connection, as each connection is its own implicit transaction. So, your code flow should be something like this:
connect to the db
create the table using the cursor
fill the table
commit and disconnect from db.
Or, if you must separate creation from filling, just commit and disconnect after (2) and then reconnect before (3).

Python not committing MySQL transaction

I am inserting a couple thousand records into a table via the python code below:
values = ''
for row in cursor:
values = values + "(" + self.quoted_comma_separate(row) + "),"
values = values[:-1]
insert_statement = "INSERT INTO t1 ({0}) VALUES {1};".format(
self.comma_separate(members), values)
db = Database()
conn = db.get_db()
cursor = conn.cursor()
cursor.execute(insert_statement)
conn.commit()
conn.close()
When I check the database after it runs none of the records show up in the database. If I go into an MySQL editor and manually commit the transaction all of the records appear. Why is my conn.commit() not working?
The insert statements were fine. Turns out I had another database connection open and it was getting confused and committing to the wrong connection or something like that. Sorry for the pointless question :)

Database connectivity succeeds but unable to run query

I am able to connect Python 3.4 and Postgres but I am the query is not successfully getting executed. For e.g, the table below is not getting created
import psycopg2
from psycopg2 import connect
try:
conn = psycopg2.connect("dbname='postgres' user='postgres' host='localhost' password='postgres'")
print("Database connected!")
cur = conn.cursor()
cur.execute("""CREATE TABLE DEPARTMENT(
ID INT PRIMARY KEY NOT NULL,
DEPT CHAR(50) NOT NULL,
EMP_ID INT NOT NULL
)""")
except:
print("I am unable to connect to the database")
Just add
conn.commit()
after you've run the execute.
Relational databases have the concept of transaction, which happen (if at all) "atomically" (all-or-none). You need to commit a transaction to actually make it take place; until you've done that, you keep the option to rollback it instead, to have no changes made to the DB if you've found something iffy on the way.

Categories