I have a list of SQL scripts I have narrowed down and want to execute. The data in the list follows this pattern more or less:
[DROP TABLE ABC ....;, CREATE TABLE ABC ....;, INSERT INTO TABLE ABC .....;,UPDATE TABLE ABC .....;]
Then it repeats itself for the next table. All of this is confined to a single list below:
dfToList_P
The problem I am having is when a table doesn't exist, I want to ignore the error and execute the CREATE TABLE statement that follows it. Here is my code:
def load_test_table(self):
s = self.connection()
df = self.retrieve_sql()
df_P = df.loc[df['STEP_TYPE'] == 'P']
dfToList_P = df_P['SQL_SCRIPT'].tolist()
try:
for sql_script in dfToList_P:
#print(sql_script)
pd.read_sql(sql_script, s)
except teradata.DatabaseError as ex:
sql_state = ex.args[0]
if sql_state == 3807:
print('Tried to DROP table that did not exist:' + sql_script)
else:
print('DatabaseError in SQL Script: ' + sql_script)
I google'd around and added the try/except condition but I don't think that is actually doing anything in this instance.
Running the script errors out with:
pandas.io.sql.DatabaseError: Execution failed on sql 'DROP TABLE ABC;': (3807, "[42S02] [Teradata][ODBC Teradata Driver][Teradata Database](-3807)Object 'ABC' does not exist.")
Any ideas?
I figured it out, thanks to a combo of comments above. First, my Try/Except was in the incorrect location and also I switched from using pandas read_sql to just using regular session execute and it worked as expected. If table exists, drop it first, if not then create it.
Revised code below:
def load_test_table(self):
s = self.connection()
df = self.retrieve_sql()
df_P = df.loc[df['STEP_TYPE'] == 'P']
dfToList_P = df_P['SQL_SCRIPT'].tolist()
for sql_script in dfToList_P:
try:
s.execute(sql_script)
except teradata.DatabaseError as ex:
sql_state = ex.args[0]
if sql_state == 3807:
print('Tried to DROP table that did not exist:' + sql_script)
else:
print('DatabaseError in SQL Script: ' + sql_script)
continue
Related
I'm getting the mentioned error while I'm using a WHERE condition in cx_oracle.
However, there is no error while I fetch all rows.
Please find my code below. The input - cols is a list of all the columns I want to fetch. Similarly, I want to pass the where condition as variable too, wherein I'm passing on the values in a loop.
For testing, lets keep it static.
Thanks
cols = [
'ID',
'CODE',
'LOGINDEX_CODE',
'IS_ACTIVE',
'IS_PORT_GROUP'
]
table_name = 'PORT'
os.chdir(os.path.dirname(__file__))
main_dir = os.getcwd()
import cx_Oracle
try:
cx_Oracle.init_oracle_client(lib_dir= main_dir + "\\instantclient_21_3\\")
except:
pass
dsn_tns = cx_Oracle.makedsn(r'some_db', 1521, service_name=r'some_service')
conn = cx_Oracle.connect(user='abcdef', password=r'ghijkl', dsn=dsn_tns).cursor()
query_cols = ', '.join(cols)
named_params = {
'varx' : query_cols,
'vary' : 'LEG',
'varz' : 242713
}
sql_query = 'SELECT :varx FROM :vary WHERE START_PORT_ID IN :varz'
conn.prepare(sql_query)
conn.execute(sql_query, named_params)
Bind variables cannot be used to replace parts of the SQL statement itself. They can only be used to supply data that is sent to the database. So you would need to do something like this instead:
sql_query = f"select {', '.join(cols)} from LEG where start_port_id = :varz"
with conn.cursor() as cursor:
for row in cursor.execute(sql_query, varz=242713):
print(row)
import pyodbc as odbc
Driver = 'SQL Server'
server_name = ''
database_name = ''
conn = odbc.connect(f"""
Driver={{{Driver}}};
Server={server_name};
Database={database_name};
UID=;
PWD=;
""")
def sku_search():
SKU_check = '''SELECT [ITEMSKU] FROM
[StilettoXT].[dbo].[PRODDETAIL] WHERE
ITEMSKU = (?)'''
SKU_code = input('Please enter the SKU code: ')
cursor = conn.cursor()
cursor.execute(SKU_check, SKU_code)
if cursor.fetchall() == []:
print('Not Found')
elif cursor.fetchall() != []:
print('Found')
sku_search()
conn.close()
I'm having real difficulty trying to get the following code to work, specially the 'elif' part of the my if statement. If you run the script with an invalid SKU code that doesn't exists in that database, you get the print statement. If you run with a valid SKU code, nothing gets returned even though the cursor.fetchall() is not an empty list. If you swap the if and elif arguments round, you will only get a result for the 'if' part of the function. I can't understand why the 'elif' part returns nothing. If you query an invalid sku code, SQL will return a blank value. I've tried to reference this as None as well but the same issue.
Any suggestions would be greatly appreciated!
I have a SQL database that displays a varbinary (max) like this 0x9406920691068F... I want to import it to python pycharm to get the same exact type of data.
However, it shows something like this instead
[b'\x94\x06\x92\x06\x91\x06\x8f\x06\x8d..
how do I copy the same numbers to python? I am a beginner in python, please help.
I copied the code from previous post and it didn't work
import pyodbc
def hexToString(binaryString):
try:
hashString = ["{0:0>2}".format(hex(b)[2:].upper()) for b in binaryString]
return '0x' + "".join(hashString)
except:
return binaryString
query = """ select P from Access.table """
conn_str = (
**** private database details # I don't copy on the page
)
cnxn = pyodbc.connect(conn_str)
cnxn.add_output_converter(pyodbc.SQL_BINARY, hexToString)
cursor = cnxn.cursor()
try:
cursor.execute(query)
row = cursor.fetchone()
except MySQLdb.error as err:
print(err)
else:
while row is not None:
print(row)
row = cursor.fetchone()
If the column return type is varbinary(max) then you need to add the output converter function to handle SQL_VARBINARY, not SQL_BINARY
cnxn.add_output_converter(pyodbc.SQL_VARBINARY, converter_function_name)
I am trying to write. code that will allow a user to select specific columns from a sqlite database which will then be transformed into a pandas data frame. I am using a test database titled test_database.db with a table titled test. The table has three columns, id, value_one, and value_two. The function I am showing exists within a class that establishes a connection to the database and in this function the user only needs to pass the table name and a list of columns that they would like to extract. For instance in command line sqlite I might type the command select value_one, value_two from test if I wanted only to read in the columns value_one and column_two from the table test. If I type this command into command line the method works. However, in this case I use python to build the text string which is fed into pandas.read_sql_query() and the method does not work. My code is shown below
class ReadSQL:
def __init__(self, database):
self.database = database
self.conn = sqlite3.connect(self.database)
self.cur = self.conn.cursor()
def query_columns_to_dataframe(table, columns):
query = 'select '
for i in range(len(columns)):
query = query + columns[I] + ', '
query = query[:-2] + ' from ' + table
# print(query)
df = pd.read_sql_query(query, self.conn)
return
def close_database()
self.conn.close
return
test = ReadSQL(test_database.db)
df = query_columns_to_dataframe('test', ['value_one', 'value_two'])
I am assuming my problem has something to do with the way that query_columns_to_dataframe() pre-processes the information because if I uncomment the print command in query_columnes_to_dataframe() I get a text string that looks identical to what works if I just type it directly into command line. Any help is appreciated.
I mopped up a few mistakes in your code to produce this, which works. Note that I inadvertently changed the names of the fields in your test db.
import sqlite3
import pandas as pd
class ReadSQL:
def __init__(self, database):
self.database = database
self.conn = sqlite3.connect(self.database)
self.cur = self.conn.cursor()
def query_columns_to_dataframe(self, table, columns):
query = 'select '
for i in range(len(columns)):
query = query + columns[i] + ', '
query = query[:-2] + ' from ' + table
#~ print(query)
df = pd.read_sql_query(query, self.conn)
return df
def close_database():
self.conn.close
return
test = ReadSQL('test_database.db')
df = test.query_columns_to_dataframe('test', ['value_1', 'value_2'])
print (df)
Output:
value_1 value_2
0 2 3
Your code are full of syntax errors and issues
The return in query_columns_to_dataframe should be return df. This is the primary reason why your code does not return anything.
self.cur is not used
Missing self parameter when declaring query_columns_to_dataframe
Missing colon at the end of the line def close_database()
Missing self parameter when declaring close_database
Missing parentheses here: self.conn.close
This df = query_columns_to_dataframe should be df = test.query_columns_to_dataframe
Fixing these errors and your code should work.
I'm quite new to mysql as in manipulating the database itself. I succeeded to store new lines in a table but my next endeavor will be a little more complex.
I'd like to fetch the column names from an existing mysql database and save them to an array in python. I'm using the official mysql connector.
I'm thinking I can achieve this through the information_schema.columns command but I have no idea how to build the query and store the information in an array. It will be around 100-200 columns so performance might become an issue so I don't think its wise just to iterate my way through it for each column.
The base code to inject code into mysql using the connector is:
def insert(data):
query = "INSERT INTO templog(data) " \
"VALUES(%s,%s,%s,%s,%s)"
args = (data)
try:
db_config = read_db_config()
conn = MySQLConnection(db_config)
cursor = conn.cursor()
cursor.execute(query, args)
#if cursor.lastrowid:
# print('last insert id', cursor.lastrowid)
#else:
# print('last insert id not found')
conn.commit()
cursor.close()
conn.close()
except Error as error:
print(error)
As said this above code needs to be modified in order to get data from the sql server. Thanks in advance!
Thanks for the help!
Got this as working code:
def GetNames(web_data, counter):
#get all names from the database
connection = create_engine('mysql+pymysql://user:pwd#server:3306/db').connect()
result = connection.execute('select * from price_usd')
a = 0
sql_matrix = [0 for x in range(counter + 1)]
for v in result:
while a == 0:
for column, value in v.items():
a = a + 1
if a > 1:
sql_matrix[a] = str(('{0}'.format(column)))
This will get all column names from the existing sql database