We Keep Coding

Add MySQL Table with python connector fails to add table without throwing error with multi=True

I have run into a very strange issue where my code below was working (I was able to add a table into the DB). After dropping a test table via the mysql workbench I am able to add a database and the code continues to run with no errors but no table is added. I have slimmed the SQL right down and attempted on different dbs. I have also restarted. Am I doing something wrong here? Any help is greatly appreciated.
import mysql.connector
import os
dirname = os.path.abspath('')
sql_filename = dirname + '\SQL_Creation\Test.sql'
class connectionsetup:
def __init__(self):
self.mydb = mysql.connector.connect(
host="localhost",
user="root",
password="")
try:
self.mycursor = self.mydb.cursor()
print('Conneciton Succesful')
self.mycursor.execute("CREATE DATABASE AP_Application_Db_test")
print('Database Created')
self.mycursor.execute('USE ap_application_db_test; Create TABLE test (SERIAL_NUMBER VARCHAR(255),VIOLATION_STATUS VARCHAR(255))', multi=True)
self.mydb.commit()
self.mydb.close()
except mysql.connector.Error as err:
print(err)
print("Error Code:", err.errno)
print("SQLSTATE", err.sqlstate)
print("Message", err.msg )
connectionsetup()

According to the docs, cursor.execute(..., multi=True)
[...] returns an iterator that enables processing the result of each statement
So the code needs to be like this:
for _ in self.mycursor.execute(multiple_sql_statements, multi=True):
pass

psycopg2 why is the schema not created?

im trying to create a schema in postgres database using psycopg2.
For some reason the schema is not created and later on the code crashes because it tries to refer to the missing schema. The connection is set to auto commit mode, which definetly works because i can create a database with this specific connection.
For debugging purposes i have wrapped every step in it's own try/except statement.
Code is below, as it is right there, it does not raise any exceptions, just the follow up crashes because the schema is missing.
def createDB(dbName, connString):
conn = psycopg2.connect(connString)
conn.set_session(autocommit =True) # autocommit must be True sein, else CREATE DATABASE will fail https://www.psycopg.org/docs/usage.html#transactions-control
cursor = conn.cursor()
createDB = sql.SQL('CREATE DATABASE {};').format(
sql.Identifier(dbName)
)
createSchema = sql.SQL('CREATE SCHEMA IF NOT EXISTS schema2;')
searchpath = sql.SQL('ALTER DATABASE {} SET search_path TO public, schema2;').format(
sql.Identifier(dbName)
)
dropDB = sql.SQL('DROP DATABASE IF EXISTS {};').format(
sql.Identifier(dbName)
)
try:
cursor.execute(dropDB)
except Exception as e:
print('drop DB failed')
logging.error(e)
conn.close()
exit()
try:
cursor.execute(createDB)
except Exception as e:
print('create DB failed')
logging.error(e)
conn.close()
exit()
try:
cursor.execute(createSchema)
print('schema created')
except Exception as e:
print('create schema failed')
logging.error(e)
conn.close()
exit()
try:
cursor.execute(searchpath)
except Exception as e:
print('set searchpath failed')
logging.error(e)
conn.close()
exit()
conn.close()
Adding an explicit commit does not do the trick either.
What am i missing?
EDIT
I have added a small screenshot with the console logs. As you can see, the code below gets executed.
EDIT 2
Out of sheer curiosity, i have tried to execute this very SQL statement in pgadmin:
CREATE SCHEMA IF NOT EXISTS schema2
and it works just fine, which shows, that my SQL is not wrong, so back to square one.
EDIT 3 -- Solution
So i have come up with a solution, thank to you #jjanes for pointing me in the right direction. This function does not connect to a specific database, but the server as a whole, since im using it to create new databases, hence the connection string looks something like this :
user=postgres password=12345 host=localhost port=5432
Which allows me to perform server level operations like create and drop database. But schemas are a Database level operation. Moving the exact same logic to the part of the code which is connected to the newly created database works like a charm.

You create the schema in the original database specified by the connect string. Once you create the new database, you need to connect to it in order to work in it. Otherwise, you are just working in the old database.

Connecting to MySQL DB using mysql.connector.connect fails with no error to catch

I'm using python to try and connect to a DB. This code worked and something in my environment changed so that the host in not present/accessible. This is as expected. The thing that I'm trying to work out is, I can't seem to catch the error of this happening. This is my code:
def create_db_connection(self):
try:
message('try...')
DB_HOST = os.environ['DB_HOST']
DB_USERNAME = os.environ['DB_USERNAME']
DB_PASSWORD = os.environ['DB_PASSWORD']
message('connecting...')
db = mysql.connector.connect(
host=DB_HOST,
user=DB_USERNAME,
password=DB_PASSWORD,
auth_plugin='mysql_native_password'
)
message('connected...')
return db
except mysql.connector.Error as err:
log.info('bad stuff happened...')
log.info("Something went wrong: {}".format(err))
message('exception connecting...')
except Exception as ex:
log.info('something bad happened')
message("Exception: {}".format(ex))
message('returning false connection...')
return False
I see up to the message('connecting...') call, but nothing afterwards. Also, I don't see any of the except messages/logs at all.
Is there something else I need to catch/check in order to know that a DB connection attempt has failed?
This is running inside an AWS Lambda and was working until I changed some subnets/etc. The key thing is I want to catch it no longer being able to connect.

The issue is most likely that your lambda function is timing out before the database connection is timing out.
First, modify the lambda function to execute for 60 seconds and test. You should find after about 30 seconds you will see the connection to the database timeout.
To resolve this issue, modify the security group on the database instance to include the security group configured for lambda. Use this entry to open a the correct port 3306

PostgreSQL ALTER SEQUENCE using psycopg2 in django

I am trying to alter sequence of id field in my table using psycopg. This is works fine on my local server, but not working on production. I am not getting exceptions, the sequence is just not restarting.
def alter_sequence(last_id):
try:
dbname = settings.DATABASES['default']['NAME']
user = settings.DATABASES['default']['USER']
host = settings.DATABASES['default']['HOST']
password = settings.DATABASES['default']['PASSWORD']
port = settings.DATABASES['default']['PORT']
connection = psycopg2.connect(
dbname=dbname,
user=user,
password=password,
host=host,
port=port,
)
cursor = connection.cursor()
cursor.execute('ALTER SEQUENCE "gs_requests_id_seq" RESTART WITH {}'.format(last_id))
connection.close()
except Exception as e:
print(e)
pass
I double-checked the database settings - its correct. Other database operations, performed by django ORM with this settings is works fine.
I think, this is not enough information about my project settings but dont know what information I need to specify. I have postgres 9.6 on my local computer and 10.1 on production.

cursor.execute('ALTER SEQUENCE gs_requests_id_seq RESTART WITH {};'.format(last_id))
Try this!! it will worked for me!

I found source of the problem. I didn't commit:
connection.commit()

python-mysql cursor.execute failing with access denied error

I have two machines: local_machine, server_machine. I have mysql server on server_machine and sftp server on local_machine. I am trying to send sritest.csv file (UTF-8) from local_machine to server_machine using python. These are the contents of sritest.csv:
1,2,3
I have the sql query saved in sritest.sql and these are the contents of the file:
LOAD DATA INFILE '{}'
INTO TABLE TESTBED_STAGING.test
COLUMNS TERMINATED BY ','
;
This is the python script I have now:
import MySQLdb
import os
import string
# Open database connection
db = MySQLdb.connect (host="1.2.3.4",port=3306,user="app_1",\
passwd="passwd",db="TESTBED_STAGING")
cursor=db.cursor()
#Query under testing
sql = open('sritest.sql','r').read()
print sql
l = os.listdir(".")
for file_name in l:
if file_name.endswith('sritest.csv'):
print 'the csv file we are reading is: '+file_name
#try:
cursor = db.cursor()
print 'filename is '+sql.format(file_name)
cursor.execute(sql.format(file_name))
db.commit()
'''
except Exception:
# Rollback in case there is any error
db.rollback()
print 'ERROR - So, rollback :( :( '
'''
# disconnect from server
db.close()
In the above script, I commented try,except so I can see the error where it breaks. Currently the code is breaking at cursor.execute(sql.format(file_name)) line with this error:
OperationalError: (1045, "Access denied for user 'app_1'#'%' (using password: YES)")
I have been playing around but not able to fix it. Any suggestions/ideas?

For starters, creating cursor at every loop is not a good idea. You've already created a cursor earlier, so you can remove the cursor declaration in the for loop.
Second, I think your error is due to lack of access on MySQL server at 1.2.3.4 remotely using user app_1. Try this on the server's MySQL console,
GRANT ALL PRIVILEGES ON TESTBED_STAGING.* TO 'app_1'#'%';
Lastly, try and avoid using print "line" notation and start switching to the print("line") notation for compatibility with Python 3.x

I figured out the answer and decided to leave this question open for those who might face the similar problem:
In the MySQL server (server_machine), make sure you do this after you start mysql:
mysql>grant all privileges on *.* to 'app_1'#'%' identified by 'passwd';
change LOAD DATA INFILE '{}' in sritest.sql to LOAD DATA LOCAL INFILE '{}'
In the python code, edit the MySQLdb.connect statement as:
db = MySQLdb.connect (host="1.2.3.4",port=3306,user="app_1",\
passwd="passwd",db="TESTBED_STAGING", local_infile=1)
All errors are eliminated and data is transferred.