I'm coding up a Python file that inserts rows into a MySQL table from a dataframe using mysql.connector. I'm trying to log each successful request and also the exceptions- I'm using Jupyter Notebook to execute the program. However, I'm not able to see any logging on the notebook- I have manually to go into the MySql database and check what has gone in, and I have no idea which entries haven't been successfully entered. Here is my code:
import os
import pandas as pd
import mysql.connector
import logging
import math
def upload_to_db(host, database, user, password,
tbl_name, col_str, dataframe):
try:
conn = mysql.connector.connect(
host=host, database=database, user=user, password=password)
cursor = conn.cursor()
cursor.execute("drop table if exists %s;" % (tbl_name))
cursor.execute("create table %s (%s);" % (
tbl_name, col_str))
data_list = dataframe.to_numpy().tolist()
for i in range(0, len(data_list)-1):
row_values = convert_list(data_list[i])
sql_statement = 'INSERT INTO %s VALUES (%s);' % (
tbl_name, row_values)
cursor.execute(sql_statement)
logging.info("SQL statement [" + sql_statement + "] successful")
conn.commit()
cursor.close()
except mysql.connector.Error as err:
logging.info("Exception: {}".format(err))
Why doesn't the python logging class show the exceptions or successes on the Notebook?
Related
I tried to connect snowflake(SSO Authentication) and get data from table.
But, When i run the code, i can login with my credentials in the pop-up browser window and it connect Snowflake, no response after that(the program neither terminate nor provide result).
Not sure, where am doing mistake, Pls help.
'''
import snowflake.connector
# Connecting to Snowflake using SAML 2.0-compliant IdP federated authentication
conn = snowflake.connector.connect(
user='G*****K',
account='abcdf',
authenticator='externalbrowser',
warehouse='abcdf',
database='abcdf',
schema='abcdf'
)
cur = conn.cursor()
sql = "select * from abcdf.ACCT limit 10"
x=cur.execute(sql)
cur.close()
print(x)
'''
I believe you are closing the cursor before the print;
try:
cur.execute("SELECT col1, col2 FROM test_table ORDER BY col1")
for (col1, col2) in cur:
print('{0}, {1}'.format(col1, col2))
finally:
cur.close()
Details: https://docs.snowflake.com/en/user-guide/python-connector-example.html#using-cursor-to-fetch-values
Results of the query are stored in cursor. The contents of cursor may then be stored in a local variable.
Also, best practice to close connection in the end.
https://www.psycopg.org/docs/cursor.html
import snowflake.connector
# Connecting to Snowflake using SAML 2.0-compliant IdP federated authentication
conn = snowflake.connector.connect(
user='G*****K',
account='abcdf',
authenticator='externalbrowser',
warehouse='abcdf',
database='abcdf',
schema='abcdf'
)
cur = conn.cursor()
sql = "select * from abcdf.ACCT limit 10"
cur.execute(sql)
print(cur.fetchall())
cur.close()
conn.close()
First of all thank you for your help.
I was trying to retrieve some data from a sybase IQ database using python, but I can not make it.
I've tried with the following code( from https://github.com/sqlanywhere/sqlanydb):
import sqlanydb
conn = sqlanydb.connect(uid='dba', pwd='sql', eng='demo', dbn='demo' )
curs = conn.cursor()
curs.execute("select 'Hello, world!'")
print( "SQL Anywhere says: %s" % curs.fetchone() )
curs.close()
conn.close()
Unfotunately it gives me the following error:
InterfaceError: ('Could not load dbcapi. Tried: None,dbcapi.dll,libdbcapi_r.so,libdbcapi_r.dylib', 0)
Does anyone know how to fix it?
Thanks in advance
Jessica
On windows, first you need to add the data source name (DSN).
You do this by searching for 'odbc data source administrator' on windows and creating a DSN for 'SQL Anywhere 12'. Fill in the necessary information like username,password,host,port,server name and database name. Finally test the connection as shown.
Once finished you can call the code as follows:
import sqlanydb
conn = sqlanydb.connect(dsn='SYBASE_IQ')
curs = conn.cursor()
curs.execute("select 'Hello, world!'")
print( "SQL Anywhere says: %s" % curs.fetchone())
curs.close()
conn.close()
Get and install the SYBASE ODBC DRIVER.
Configure the DSN on your PC.
On Windows, search for the Microsoft ODBC Administrator. Then create a DSN.
Python code:
using SQLAchemy
import sqlalchemy as sa
from sqlalchemy import create_engine, event
from sqlalchemy.engine.url import URL
import urllib
params = urllib.parse.quote_plus('DSN=dsn_name;PWD=user_pwd')
engine = sa.create_engine("sybase+pyodbc:///?odbc_connect={}".format(params))
with engine.connect() as cursor:
cursor.execute(""" SELECT * FROM database """)
Using PyODBC
import pyodbc
conn = pyodbc.connect('DSN=dsn_name;PWD=user_pwd')
with conn:
cursor = conn.cursor()
cursor.execute(""" SELECT * FROM database """)
I am running postgressql on a docker container. I am trying to connect to postgres via python and display the tables below is the code that I am using to connect to postgres:
import psycopg2
conn_string = "host='192.168.99.100:15432' dbname='PREDICTIVE_DS_POSTGRESQL'
user='ds_user' password='ds_user'"
print("Connecting to database\n ->%s" % (conn_string))
conn = psycopg2.connect(conn_string)
cursor = conn.cursor()
print("Connected!\n")
Then I use the below Python code to display the existing tables within postgres:
def table_exists(con, table_str):
exists = False
try:
cur = con.cursor()
cur.execute("select exists(select relname from pg_class where relname='"
+ table_str + "')")
exists = cur.fetchone()[0]
print("exists")
cur.close()
except psycopg2.Error as e:
print(e)
return exists
def get_table_col_names(con, table_str):
col_names = []
try:
cur = con.cursor()
cur.execute("select * from " + table_str + " LIMIT 0")
for desc in cur.description:
col_names.append(desc[0])
cur.close()
except psycopg2.Error as e:
print(e)
However, it is not working at all. It says that it cannot connect translate host name "192.168.99.100:15432" to address: Unknown host. However, the container is up and running and that is the host name. Additionally, I don't know whether the rest of the code will work once it connects.
Have your database credentials defined in a separate file.
For example, have a file called database.ini and define it like this:
[creds]
host=192.168.99.100
port=15432
database=PREDICTIVE_DS_POSTGRESQL
user=ds_user
password=ds_user
Have another config parser file to parse this. Call it config.py
#!/usr/bin/python
try:
import configparser
except:
from six.moves import configparser
def config(section,filename='database.ini',):
parser = configparser.ConfigParser()
parser.read(filename)
db = {}
if parser.has_section(section):
params = parser.items(section)
for param in params:
db[param[0]] = param[1]
else:
raise Exception('Section {0} not found in the {1}
file'.format(section, filename))
return db
Now, in your main file, import your config function like this:
from config import config
and connect like this:
dbParams = config("creds")
con = psycopg2.connect(**dbParams)
This question already has answers here:
How do I connect to a MySQL Database in Python?
(26 answers)
Closed 5 years ago.
i am familiar with python and also familiar with mysql and SQL. I am also clear on con, cur and commit, but my problem here is that im trying to make a small program in python (no need for a gui) to insert data into a mysql database, and bring it on my console or file. I'm confused where to start, i tried seeking google but i couldn't find any toturials about my issue, any help? a link or where to start. Also:
i know the python program can be written in an IDE or a text file, but how does it connect to mysql database? if im wrong please correct me.
SQLAlchemy is good: https://www.sqlalchemy.org/
Otherwise, using the conn/cur as you described is easy: https://www.tutorialspoint.com/python/python_database_access.htm
Go though the documentation to get yourself familiar with python, mysql and how to work with them together.
Although, the minimal code would look something like this :
import MySQLdb
query = "insert into DB_NAME values (1,2)"
try :
conn = MySQLdb.connect(host="",
user="",
passwd="",
db="")
cursor = conn.cursor()
cursor.execute(query)
conn.commit()
cursor.close()
conn.close()
except (MySQLdb.Error, Exception) as error :
print error
print "Insert data unsuccessful"
See the code below
import mysql.connector
from mysql.connector import MySQLConnection, Error
class SQL_Connect:
def __init__(self):
#-------------------------------------------------------
# Database Connection Param's
self.host_Address = 'Host Here'
self.database_Name = 'Database Name'
self.userName = 'User Name'
self.db_Password = 'Password'
#-------------------------------------------------------
def insert_IntoDB(self, Manufacturer, partNum, formFactor, socket, chipSet, memSlots, memType, maxMem, raidSup, onboardVid, crosFire_Sup, sli_Sup, sata6GBS, sataExpress, onboard_Ether):
test_Query = 'INSERT INTO motherboards (Manufacturer, modelNum, formFactor, socket, chipset, memSlots, memType, maxMem, raidSup, onboardVid, crosfireSup, sliSup, sata6GBS, sataExpress, onboardEther) VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)'
args = (Manufacturer, partNum, formFactor, socket, chipSet, memSlots, memType, maxMem, raidSup, onboardVid, crosFire_Sup, sli_Sup, sata6GBS, sataExpress, onboard_Ether)
try:
conn = mysql.connector.connect(host = self.host_Address, database = self.database_Name, user = self.userName, password = self.db_Password)
if conn.is_connected():
print 'MySQL Database Connection Established'
cursor = conn.cursor()
cursor.execute(test_Query, args)
conn.commit()
print 'Data Inserted!!!'
except Error as e:
print ('ERROR: ',e)
finally:
cursor.close()
conn.close()
I'm using python-mysql, attacked below is the code snippet I'm using to insert into a database table. For some reasons, the code is not populating any rows in the database. There are no exceptions raised and the SELECT queries work fine. On copying the code inside execute and running in phpmyadmin, the database is populated fine.
import MySQLdb as mdb
try:
con = mdb.connect(host='localhost', user='', passwd='', db='indoor')
cur = con.cursor()
cur.execute("INSERT INTO locationdata VALUES('1','1','1','1','1','1')")
numrows = cur.execute("SELECT * FROM locationdata")
print str(numrows) + " : total Rows"
print cur.fetchone()
if con.open:
print "Hello DB"
except mdb.Error, e:
Print "Error " + e. args [0]
Any ideas what am I missing?