I know I can call a database in the mysql.connector.connect method. However, I'm trying to programmatically create the database if it doesn't already exist, then connect too to upload table data.
Currently my line cursor.execute("SELECT DATABASE myDatabase") is generating an error in my "SQL syntax".
How can I check if a database exists and then connect to it?
db = mysql.connector.connect(
host="localhost",
user=os.environ.get('MYSQL_DB_USER'),
password=os.environ.get('MYSQL_DB_USER_PASS')
)
cursor = db.cursor()
cursor.execute("CREATE DATABASE IF NOT EXISTS myDatabase")
cursor.execute("SELECT DATABASE myDatabase")
# create table on database
sql = "CREATE TABLE IF NOT EXISTS `trades` (`id` INT(11) UNSIGNED NOT NULL AUTO_INCREMENT, `objectid` VARCHAR(100), `type` VARCHAR(5), `ownerid` VARCHAR(100), `color` VARCHAR(15), `shape` VARCHAR(15), `size` VARCHAR(15), PRIMARY KEY (`id`) );"
print(sql)
cursor.execute(sql)
Alternatively, the database property can be used of the MySQLConnection object, as per the MySQL Connector documentation (which executes the USE command for you):
db = mysql.connector.connect(
host="localhost",
user=os.environ.get('MYSQL_DB_USER'),
password=os.environ.get('MYSQL_DB_USER_PASS')
)
cursor = db.cursor()
cursor.execute("CREATE DATABASE IF NOT EXISTS myDatabase")
db.database = 'myDatabase'
(I was dealing with the same issue and stumbled upon this post, after which I found the described solution. So thought I'd share this alternative solution for anyone interested.)
Per MySQL documentation, the syntax is:
CREATE DATABASE `db_name`;
Then to select/change the database, use:
USE `db_name`;
To list all available databases, use:
SHOW DATABASES;
Alternatively, you can skip the USE command and reference the database in the statement itself as:
CREATE TABLE `db_name`.`table1` ... ;
Or, in the case of a SELECT statement:
SELECT * FROM `db_name`.`table1`;
Related
I am trying to execute some Postgres queries with psycopg2.
import psycopg2
conn = psycopg2.connect(
host="test.postgres.database.azure.com",
database="postgres",
user="test",
password="test")
cur = conn.cursor()
sql = "select site_panel from test.ws_sites"
cur.execute(sql)
rows = cur.fetchall()
The query above works well and returns the data but the following query does not delete the table as intended.
cur.execute ("DROP TABLE IF EXISTS test.ws_sites")
Anything wrong I'm doing here?
A query that modifies table data or schema structure needs to be committed:
cur.execute ("DROP TABLE IF EXISTS test.ws_sites")
conn.commit()
Alternatively, place
conn.autocommit = True
after creating the connection.
I'm currently working on a automated scripting assignment from school... I'm trying to create a table if it does not exists. But I want the table to have the hostname from the machine (it's working on virtual machine's). I have tried searching on the magical interwebs. But I can't find a clear answer to it.
Is it possible to add a hostname like this?
#!/bin/python3
import psutil
import socket
import mysql.connector
machine = socket.gethostname()
memory = psutil.virtual_memory()[2]
disk = psutil.disk_usage('/').percent
cpu = psutil.cpu_percent()
#print (machine, memory, disk, cpu)
def create_table():
try:
connection = mysql.connector.connect(host='192.168.0.2',
database='gegevens',
user='db_user',
password='Welkom01')
if connection.is_connected():
cursor = connection.cursor()
sql = "CREATE TABLE IF NOT EXISTS "+ machine("virtual_mem varchar(255), disk_usage varchar(255),cpu_usage varchar (255);")
cursor.execute(sql)
connection.commit()
finally:
if (connection.is_connected()):
cursor.close()
connection.close()
print("MySQL connection is closed")
create_table()
Or do you guys suggest I'm doing it differently. Because when i'm trying to run it like this, i'm getting a TypeError;
Traceback (most recent call last):
File "./agent.py", line 36, in <module>
create_table()
File "./agent.py", line 23, in create_table
sql = "CREATE TABLE IF NOT EXISTS "+ machine("virtual_mem varchar(255), disk_usage varchar(255),cpu_usage varchar (255);")
TypeError: 'str' object is not callable
With many to quamrana and S3DEV I have figured out how to make it done... I have changed line 23:
sql = "CREATE TABLE IF NOT EXISTS "+ machine("virtual_mem varchar(255), disk_usage varchar(255),cpu_usage varchar (255);")
into:
sql = f"CREATE TABLE IF NOT EXISTS {machine} (virtual_mem varchar(255), disk_usage varchar(255),cpu_usage varchar (255))"
With this is it working.
You seem to just the sql string wrong. You are trying to add 3 strings together but are mixing up the syntax, here are a few options with the correct syntax:
# option 1
sql = f"CREATE TABLE IF NOT EXISTS {machine} (virtual_mem varchar(255), disk_usage varchar(255),cpu_usage varchar (255)(;"
# option 2
sql = "CREATE TABLE IF NOT EXISTS {table_name} (virtual_mem varchar(255), disk_usage varchar(255),cpu_usage varchar (255));".format(
table_name=machine
)
Basically 'machine' in your code is a string and is the sql block you have there, we want this string to be 'injected' into our string.
The way to do this in python is using string format/formatted string or even just plain concatetation (I prefer the first two options for you use-case as they are a bit more clear)
See this article for more information: https://realpython.com/python-f-strings/
You have an error in the way you try to format your sql query.
Please change the below line:
sql = "CREATE TABLE IF NOT EXISTS "+ machine("virtual_mem varchar(255), disk_usage varchar(255),cpu_usage varchar (255);")
To a format string like this:
sql = f"CREATE TABLE IF NOT EXISTS {machine} (virtual_mem varchar(255), disk_usage varchar(255),cpu_usage varchar (255);"
It would be even better if you checked out the formatting options offered by the mysql connector so the query substitution would be executed by the connector engine, which most probably guards against SQL injection issues.
When I'm using pymysql to perform operations on MySQL database, it seems that all the operations are temporary and only visible to the pymysql connection, which means I can only see the changes through cur.execute('select * from qiushi') and once I cur.close() and conn.close() and log back in using pymysql, everything seems unchanged.
However, when I'm looking at the incremental id numbers, it does increased, but I can't see the rows that were inserted from pymysql connection. It seems that they were automatically deleted?!
Some of my code is here:
import pymysql
try:
conn = pymysql.connect(host='127.0.0.1',port=3306,user='pymysql',passwd='pymysql',charset='utf8')
cur = conn.cursor()
#cur.execute('CREATE TABLE qiushi (id INT NOT NULL AUTO_INCREMENT, content_id BIGINT(10) NOT NULL, content VARCHAR(1000), created TIMESTAMP DEFAULT CURRENT_TIMESTAMP, PRIMARY KEY(id));')
#cur.execute('DESCRIBE content')
#cur.fetchall()
cur.execute('USE qiushibaike')
for _ in range(0,len(content_ids)):
cur.execute("INSERT INTO qiushi (content,content_id) VALUES (\"%s\",%d)"%(jokes[_],int(content_ids[_])))
finally:
cur.close()
conn.close()
I solved the problem by myself...
Because the config is automatically committed, so after each SQL sentence we should commit the changes.
Approach 1:
add cur.commit() after the cur.execute()
Approach 2:
edit the connection config, add autocommit=True
I created a database in psql and in it, created a table called "tweet".
CREATE TABLE tweet
( tid CHARACTER VARYING NOT NULL, DATA json,
CONSTRAINT tid_pkey PRIMARY KEY (tid) );
Then when I use
SELECT * FROM tweet;
in the psql window it works and shows an empty table.
Now I have a python script that takes JSON data and is loading it into this table.
conn_string = "host='localhost' port=5432 dbname='tweetsql' user='tweetsql' password='tweetsql'"
conn = psycopg2.connect(conn_string)
cur = conn.cursor()
That sets up the connection and I don't think it had any issues.
Now I have some logic to read in the JSON file and then to add it in, I say:
cur.execute("INSERT INTO tweet (tid, data) VALUES (%s, %s)", (cur_tweet['id'], json.dumps(cur_tweet, cls=DecimalEncoder), ))
But this always says that the relation tweet doesn't exist. Am I missing something here? Is there an issue with my connection or can my script somehow not see the table? For reference I'm using psycopg2 for the connection.
EDIT: I updated the DDL to include a transaction I could commit but that didn't fix it either. Is it a schema issue?
This is what I did regarding the table creation to commit:
BEGIN;
CREATE TABLE tweet
( tid CHARACTER VARYING NOT NULL, DATA json,
CONSTRAINT tid_pkey PRIMARY KEY (tid) );
COMMIT;
EDIT 2: I'm posting some code here...
import psycopg2
import json
import decimal
import os
import ctypes
conn_string = "host='localhost' port=5432 dbname='tweetsql' user='tweetsql' password='tweetsql'"
conn = psycopg2.connect(conn_string)
cur = conn.cursor()
cur.execute("CREATE TABLE tweet (tid CHARACTER VARYING NOT NULL, DATA json, CONSTRAINT tid_pkey PRIMARY KEY (tid) );")
cur.commit()
for file in os.listdir(path):
if not is_hidden(file):
with open(path+file, encoding='utf-8') as json_file:
tweets = json.load(json_file, parse_float=decimal.Decimal)
for cur_tweet in tweets:
cur.execute("INSERT INTO tweet (tid, data) VALUES (%s, %s)", (cur_tweet['id'], json.dumps(cur_tweet, cls=DecimalEncoder), ))
cur.commit()
cur.close()
conn.close()
You're probably not committing the table creation, and, (I'm assuming; not seeing your complete code) you're starting a new connection via psycopg2 each time. You need to commit right after the table creation, and not in a new connection, as each connection is its own implicit transaction. So, your code flow should be something like this:
connect to the db
create the table using the cursor
fill the table
commit and disconnect from db.
Or, if you must separate creation from filling, just commit and disconnect after (2) and then reconnect before (3).
I am able to connect Python 3.4 and Postgres but I am the query is not successfully getting executed. For e.g, the table below is not getting created
import psycopg2
from psycopg2 import connect
try:
conn = psycopg2.connect("dbname='postgres' user='postgres' host='localhost' password='postgres'")
print("Database connected!")
cur = conn.cursor()
cur.execute("""CREATE TABLE DEPARTMENT(
ID INT PRIMARY KEY NOT NULL,
DEPT CHAR(50) NOT NULL,
EMP_ID INT NOT NULL
)""")
except:
print("I am unable to connect to the database")
Just add
conn.commit()
after you've run the execute.
Relational databases have the concept of transaction, which happen (if at all) "atomically" (all-or-none). You need to commit a transaction to actually make it take place; until you've done that, you keep the option to rollback it instead, to have no changes made to the DB if you've found something iffy on the way.