I'm quite new to mysql as in manipulating the database itself. I succeeded to store new lines in a table but my next endeavor will be a little more complex.
I'd like to fetch the column names from an existing mysql database and save them to an array in python. I'm using the official mysql connector.
I'm thinking I can achieve this through the information_schema.columns command but I have no idea how to build the query and store the information in an array. It will be around 100-200 columns so performance might become an issue so I don't think its wise just to iterate my way through it for each column.
The base code to inject code into mysql using the connector is:
def insert(data):
query = "INSERT INTO templog(data) " \
"VALUES(%s,%s,%s,%s,%s)"
args = (data)
try:
db_config = read_db_config()
conn = MySQLConnection(db_config)
cursor = conn.cursor()
cursor.execute(query, args)
#if cursor.lastrowid:
# print('last insert id', cursor.lastrowid)
#else:
# print('last insert id not found')
conn.commit()
cursor.close()
conn.close()
except Error as error:
print(error)
As said this above code needs to be modified in order to get data from the sql server. Thanks in advance!
Thanks for the help!
Got this as working code:
def GetNames(web_data, counter):
#get all names from the database
connection = create_engine('mysql+pymysql://user:pwd#server:3306/db').connect()
result = connection.execute('select * from price_usd')
a = 0
sql_matrix = [0 for x in range(counter + 1)]
for v in result:
while a == 0:
for column, value in v.items():
a = a + 1
if a > 1:
sql_matrix[a] = str(('{0}'.format(column)))
This will get all column names from the existing sql database
Related
I am trying to clean raw json data by parsing and inserting it into a table of an sqlite db.
I have 22 columns in my table and want to find a way of looping through them so I don't need to write 22 loops which insert the data or a single column.
I have simplified the approach I am trying with the following:
import sqlite3
conn = sqlite3.connect('cdata.sqlite')
cur = conn.cursor()
column = 'name'
value = 'test'
cur.execute('''INSERT INTO COMPANY (?)
VALUES (?)''',(column,),(value,))
conn.commit()
conn.close()
This doesn't work at the moment and return the error TypeError: function takes at most 2 arguments (3 given).
Does anyone know if it is possible to write an SQLite insert statement using 2 parameters like this or another way I might be able to iterate through the columns?
Sample as below:
import sqlite3
conn = sqlite3.connect("cdata.sqlite")
cur = conn.cursor()
column = ("name", "age")
table = f"CREATE TABLE IF NOT EXISTS COMPANY ({column[0]} text, {column[1]} text);"
cur.execute(table)
name = "hello"
age = "1"
sql_stmt = f"INSERT INTO COMPANY({column[0]},{column[1]}) VALUES ('{name}', '{age}')"
cur.execute(sql_stmt)
with conn:
cur.execute("SELECT * FROM COMPANY")
print(cur.fetchall())
conn.commit()
conn.close()
I'm trying to write a python script to get a count of some tables for monitoring which looks a bit like the code below. I'm trying to get an output such as below and have tried using python multi-dimensional arrays but not having any luck.
Expected Output:
('oltptransactions:', [(12L,)])
('oltpcases:', [(24L,)])
Script:
import psycopg2
# Connection with the DataBase
conn = psycopg2.connect(user = "appuser", database = "onedb", host = "192.168.1.1", port = "5432")
cursor = conn.cursor()
sql = """SELECT COUNT(id) FROM appuser.oltptransactions"""
sql2 = """SELECT count(id) FROM appuser.oltpcases"""
sqls = [sql,sql2]
for i in sqls:
cursor.execute(i)
result = cursor.fetchall()
print('Counts:',result)
conn.close()
Current output:
[root#pgenc python_scripts]# python multi_getrcount.py
('Counts:', [(12L,)])
('Counts:', [(24L,)])
Any help is appreciated.
Thanks!
I am a bit reluctant to show this way, because best practices recommend to never build a dynamic SQL string but always use a constant string and parameters, but this is one use case where computing the string is legit:
a table name cannot be a parameter in SQL
the input only comes from the program itself and is fully mastered
Possible code:
sql = """SELECT count(*) from appuser.{}"""
tables = ['oltptransactions', 'oltpcases']
for t in tables:
cursor.execute(sql.format(t))
result = cursor.fetchall()
print("('", t, "':,", result, ")")
I believe something as below, Unable to test code because of certificate issue.
sql = """SELECT 'oltptransactions', COUNT(id) FROM appuser.oltptransactions"""
sql2 = """SELECT 'oltpcases', COUNT(id) FROM appuser.oltpcases"""
sqls = [sql,sql2]
for i in sqls:
cursor.execute(i)
for name, count in cursor:
print ("")
Or
sql = """SELECT 'oltptransactions :'||COUNT(id) FROM appuser.oltptransactions"""
sql2 = """SELECT 'oltpcases :'||COUNT(id) FROM appuser.oltpcases"""
sqls = [sql,sql2]
for i in sqls:
cursor.execute(i)
result = cursor.fetchall()
print(result)
def view_empdetails(): #this is my function: it works great
conn = mysql.connector.connect(host="localhost",user="root",passwd="#####",database="#DB")
cursor = conn.cursor() # this is database connection
viw = """select * from employees"""
cursor.execute(viw)
for emp_no,first_name,last_name,gender,DOB,street,city,state,zipcode,email,phone,hire_date in cursor.fetchall(): # fetch all data from employee table in DB
print('-'*50)
print(emp_no)
print(first_name)
print(last_name)
print(gender)
print(DOB)
print(street)
print(city)
print(state) # I need all these output be in a table or organize format
print(zipcode) #not only list of records
print(email)
print(phone)
print(hire_date)
print('-'*50)
conn.commit()
conn.close()
return menu2()
I need all records in one table|| codes bring data from Database as line by line without formatting> I need them in table
I'm not sure if you are familiar with the pandas library, but I believe it is helpful here. I have never used it with mysql, but I have used it with psycopg2 and pyodbc, so I think the basic idea should work:
data = pd.DataFrame(cur.fetchall(),columns = colnames)
creates a dataFrame (think python spreadsheet or python table) that uses the column names from the table you're querying.
I am trying to calculate the mode value of each row and store the value in the judge = judge column, however it updates only the first record and leaves the loop
ps: Analisador is my table and resultado_2 is my db
import sqlite3
import statistics
conn = sqlite3.connect("resultado_2.db")
cursor = conn.cursor()
data = cursor.execute("SELECT Bow, FastText, Glove, Wordvec, Python, juiz, id FROM Analisador")
for x in data:
list = [x[0],x[1],x[2],x[3],x[4],x[5],x[6]]
mode = statistics.mode(list)
try:
cursor.execute(f"UPDATE Analisador SET juiz={mode} where id={row[6]}") #row[6] == id
conn.commit()
except:
print("Error")
conn.close()
You have to fetch your records after SQL is executed:
cursor.execute("SELECT Bow, FastText, Glove, Wordvec, Python, juiz, id FROM Analisador")
data = cursor.fetchall()
That type of SQL query is different from UPDATE (that you're using in your code too) which doesn't need additional step after SQL is executed.
I have a SQL database file which contains a multitude of columns, two of which are 'GEO_ID' and 'MED_INCOME'. I am trying to retrieve just the 'MED_INCOME' column data using the associated 'GEO_ID'. Here is what I thought would work:
import sqlite3 as db
def getIncome(censusID):
conn = db.connect('census.db')
c = conn.cursor()
c.execute("SELECT 'MED_INCOME' FROM censusDbTable WHERE GEO_ID = %s" % (censusID)
response = c.fetchall()
c.close()
conn.close()
return response
id = 60014001001
incomeValue = getIncome(id)
print("incomeValue: ", incomeValue)
Which results in:
incomeValue: [('MED_INCOME',)]
I thought that I had used this method before when attempting to retrieve the data from just one column, but this method does not appear to work. If I were to instead write:
c.execute("SELECT * FROM censusDbTable WHERE GEO_ID = %s" % (censusID)
I get the full row's data, so I know the ID is in the database file.
Is there something about my syntax that is causing this request to result in an empty set?
Per #Ernxst comment, I adjusted the request to:
c.execute("SELECT MED_INCOME FROM censusDbTable WHERE GEO_ID = %s" % (censusID)
Removing the quotes around the column ID, which solved the problem.