I have a very weird problem with mysqldb (mysql module for python).
I have a file with queries for inserting records in tables. If I call the functions from the file, it works just fine; but when trying to call one of the functions from another file it throws me a
_mysql_exception.InterfaceError: (0, '')
I really don't get what I'm doing wrong here..
I call the function from buildDB.py :
import create
create.newFormat("HD", 0,0,0)
The function newFormat(..) is in create.py (imported) :
from Database import Database
db = Database()
def newFormat(name, width=0, height=0, fps=0):
format_query = "INSERT INTO Format (form_name, form_width, form_height, form_fps) VALUES ('"+name+"',"+str(width)+","+str(height)+","+str(fps)+");"
db.execute(format_query)
And the class Database is the following :
import MySQLdb
from MySQLdb.constants import FIELD_TYPE
class Database():
def __init__(self):
server = "localhost"
login = "seq"
password = "seqmanager"
database = "Sequence"
my_conv = { FIELD_TYPE.LONG: int }
self.conn = MySQLdb.connection(host=server, user=login, passwd=password, db=database, conv=my_conv)
# self.cursor = self.conn.cursor()
def close(self):
self.conn.close()
def execute(self, query):
self.conn.query(query)
(I put only relevant code)
Traceback :
Z:\sequenceManager\mysql>python buildDB.py
D:\ProgramFiles\Python26\lib\site-packages\MySQLdb\__init__.py:34: DeprecationWa
rning: the sets module is deprecated
from sets import ImmutableSet
INSERT INTO Format (form_name, form_width, form_height, form_fps) VALUES ('HD',0
,0,0);
Traceback (most recent call last):
File "buildDB.py", line 182, in <module>
create.newFormat("HD")
File "Z:\sequenceManager\mysql\create.py", line 52, in newFormat
db.execute(format_query)
File "Z:\sequenceManager\mysql\Database.py", line 19, in execute
self.conn.query(query)
_mysql_exceptions.InterfaceError: (0, '')
The warning has never been a problem before so I don't think it's related.
I got this error when I was trying to use a closed connection.
Problem resolved.. I was initializing the database twice.. Sorry if you lost your time reading this !
I couldn't get your setup to work. I gives me the same error all the time. However the way you connect to and make queries to the db with the query seems to be "non-standard".
I had better luck with this setup:
conn = MySQLdb.Connection(user="user", passwd="******",
db="somedb", host="localhost")
cur = conn.cursor()
cur.execute("insert into Format values (%s,%s,%s,%s);", ("hd",0,0,0))
This way you can take advantage of the db modules input escaping which is a must to mitigate sql injection attacks.
Related
I have an aiohttp app where when ran the first time it will attempt to create an sqlite db if it doesnt exist on a try_make_db() function. I also don't think an exception error is a good way to do this either...
Could someone give me a tip on how to make this better?
When run the file I get a traceback:
Traceback (most recent call last):
File "app.py", line 280, in <module>
try_make_db()
File "app.py", line 250, in try_make_db
with sqlite3.connect(my_path) as conn:
sqlite3.OperationalError: unable to open database file
I dont think this has anything to do with aiohttp or sqlite but more of a PATH question.
This isn't the entire code but only near the bottom, here's a gist of the entire script:
def try_make_db() -> None:
my_path = './form_data.db'
with sqlite3.connect(my_path) as conn:
cur = conn.cursor()
print("Trying to connect to the db!")
try:
cur.execute("SELECT 1 FROM posts LIMIT 1;")
cur.close()
print("Good enough DB should be Ok")
except Exception as e:
print("Table 'posts' does not exist")
cur.execute(
"""CREATE TABLE posts (
Date TEXT PRIMARY KEY,
starttime TEXT,
endtime TEXT,
Weekends NUMBER,
Weekdays NUMBER,
setpoint NUMBER)
"""
)
print("DB TABLE CREATED")
conn.commit()
async def init_db(app: web.Application) -> AsyncIterator[None]:
sqlite_db = 'form_data.db'
db = await aiosqlite.connect(sqlite_db)
db.row_factory = aiosqlite.Row
app["DB"] = db
yield
await db.close()
try_make_db()
web.run_app(init_app() , host='0.0.0.0', port=8080)
Any tips greatly appreciated not a lot of wisdom here. Would anyone have a tip on how I could modify the try_to_make_db function maybe something like this:
import os.path
if os.path.exists(PATH_TO_DB): # replace the path here
print("Path exists")
check write permissions on form_data.db also you don't need to use ./ to explicitly state your current working directory, just using the name form_data.db will create the file in same dir as your main program.
I'm trying to get an admin account to edit a 'rank' (basically access level) for one of the profiles in my data-base. The error is:
Traceback (most recent call last):
File "U:/A-level Computor Science/Y12-13/SQL/sqlite/Databases/ork task/Python for SQL V_2.py", line 154, in <module>
main()
File "U:/A-level Computor Science/Y12-13/SQL/sqlite/Databases/ork task/Python for SQL V_2.py", line 9, in main
start_menu()
File "U:/A-level Computor Science/Y12-13/SQL/sqlite/Databases/ork task/Python for SQL V_2.py", line 22, in start_menu
login()
File "U:/A-level Computor Science/Y12-13/SQL/sqlite/Databases/ork task/Python for SQL V_2.py", line 72, in login
Mek_menu()
File "U:/A-level Computor Science/Y12-13/SQL/sqlite/Databases/ork task/Python for SQL V_2.py", line 108, in Mek_menu
where Uzaname = %s""" % (NewRank, Findaname))
sqlite3.OperationalError: unrecognized token: "0rk_D4T4B453"`
The code that seems to be the problem is:
cursor.execute(""" update 0rk_D4T4B453.Da_Boyz
set Rank = %s
where Uzaname = %s""" % (NewRank, Findaname))
Originally, it was all on one line and it didn't work, and now I've tried it on multiple lines and it still doesn't work. So I checked here to see if anyone could help.
EDIT1: Thanks for the suggestions. None of them have fixed the code, but I've narrowed the problem code to: where Uzaname = %s""" % (NewRank, Findaname))
Unless you use ATTACH, SQLite (a file-level database) does not recognize other databases. Usually server-level databases (Oracle, Postgres, SQL Server, etc.) use the database.schema.table reference. However, in SQLite the very database file you connect to is the main database in scope. But ATTACH allows you to connect to other SQLite databases and then recognizes database.table referencing.
Additionally, for best practices:
In sqlite3 and any other Python DB-APIs, use parameterization for literal values and do not format values to SQL statement.
In general Python, stop using the de-emphasized (not deprecated yet) string modulo operator, %. Use str.format or more recent F-string for string formatting. But neither is needed here.
Altogether, if you connect to the 0rk_D4T4B453 database, simply query without database reference:
conn = sqlite3.connect('/path/to/0rk_D4T4B453.db')
cursor = conn.cursor()
# PREPARED STATEMENT WITH QMARK PLACEHOLDERS
sql = """UPDATE Da_Boyz
SET Rank = ?
WHERE Uzaname = ?"""
# BIND WITH TUPLE OF PARAMS IN SECOND ARG
cursor.execute(sql, (NewRank, Findaname))
conn.commit()
If you do connect to a different database, call ATTACH. Here also, you can alias other database with better naming instead of number leading identifier.
cursor.execute("ATTACH '/path/to/0rk_D4T4B453.db' AS other_db")
sql = """UPDATE other_db.Da_Boyz
SET Rank = ?
WHERE Uzaname = ?"""
cursor.execute(sql, (NewRank, Findaname))
conn.commit()
cur.execute("DETACH other_db")
So I am trying to retrieve data from database with MySQLdb in pandas dataframe.
import MySQLdb as mysqldb
import MySQLdb.cursors
import pandas as pd
def connection():
db = mysqldb.connect(
host="123.456.7.890",
user="user",
passwd="password",
db="database",
port=12345,
cursorclass=MySQLdb.cursors.DictCursor
)
return db
mysql = connection()
def testing():
cur = mysql.cursor()
query = cur.execute("select * from table1")
result = cur.fetchall()
cur.close()
result_df = pd.DataFrame(result)
return result_df
When I print the 'testing' function, I get an error:
Traceback (most recent call last):
File "C:/Users/xx/PycharmProjects/practice/python.py", line 97, in <module>
print(testing())
File "C:/Users/xx/PycharmProjects/practice/python.py", line 94, in testing
result_df = pd.DataFrame(result)
File "C:\Users\xx\PycharmProjects\practice\venv\lib\site-packages\pandas\core\frame.py", line 422, in __init__
raise ValueError('DataFrame constructor not properly called!')
ValueError: DataFrame constructor not properly called!
I put cursorclass to MySQLdb.cursors.DictCursor to get the data in dictionary form but instead it seems like I'm getting them in tuple. Currently using python 3.7.
It works when I use pymysql but seems quite slow.
I am trying to connect with db using the following code:
import MySQLdb
db = MySQLdb.connect(host="localhost", # your host, usually localhost
user="root", # your username
passwd="root", # your password
db="test101") # name of the data base
# you must create a Cursor object. It will let
# you execute all the queries you need
cur = db.cursor()
# Use all the SQL you like
cur.execute("SELECT * FROM test1")
# print all the first cell of all the rows
for row in cur.fetchall():
print row[0]
db.close()
However, I am getting the following error message on the console:
Traceback (most recent call last):
File "C:\Users\JRambo\workspace\DBConnection\src\DBConnection.py", line 6, in <module>
import MySQLdb
File "C:\Python27\lib\site-packages\MySQLdb\__init__.py", line 19, in <module>
import _mysql
ImportError: DLL load failed: %1 is not a valid Win32 application.
I have followed the steps meticulously.
How do I connect to a MySQL Database in Python?
You might want to verify that you have the correct bit Python and correct bit MySQLdb. If you have 32 bit Python and 64 bit MySQLdb it won't work. I had a similar problem with the same Traceback error and when I installed the correct bit type of each application, bingo! Hope this helps!
With reference to Import MySQL dump to PostgreSQL database.
An unknown developer has offered there to use the following script to import MySQL database to PostgreSQL
import MySQLdb
#from magic import Connect #Private mysql connect information - I COMMENTED THIS LINE to use direct connection
db = MySQLdb.connect(host="localhost", # your host, usually localhost
user="USER", # your username
passwd="PASS", # your password
db="w3i") # name of the data base
import psycopg2
dbx=Connect()
DB=psycopg2.connect("dbname='w3i'")
DC=DB.cursor()
mysql='''show tables from w3i'''
dbx.execute(mysql); ts=dbx.fetchall(); tables=[]
for table in ts: tables.append(table[0])
for table in tables:
mysql='''describe w3i.%s'''%(table)
dbx.execute(mysql); rows=dbx.fetchall()
psql='drop table %s'%(table)
DC.execute(psql); DB.commit()
psql='create table %s ('%(table)
for row in rows:
name=row[0]; type=row[1]
if 'int' in type: type='int8'
if 'blob' in type: type='bytea'
if 'datetime' in type: type='timestamptz'
psql+='%s %s,'%(name,type)
psql=psql.strip(',')+')'
print psql
try: DC.execute(psql); DB.commit()
except: pass
msql='''select * from w3i.%s'''%(table)
dbx.execute(msql); rows=dbx.fetchall()
n=len(rows); print n; t=n
if n==0: continue #skip if no data
cols=len(rows[0])
for row in rows:
ps=', '.join(['%s']*cols)
psql='''insert into %s values(%s)'''%(table, ps)
DC.execute(psql,(row))
n=n-1
if n%1000==1: DB.commit(); print n,t,t-n
DB.commit()
As you can see - I changed line 2 to direct connection with MySQL
But now I have the following error
python postgres.py
Traceback (most recent call last):
File "postgres.py", line 9, in <module>
dbx=Connect()
NameError: name 'Connect' is not defined
Thanks in advance for a hint how to solve it !
EDIT : I forgot the cursor ...
EDIT2 : original script did not correctly process fields of TINYTEXT, MEDIUMTEXT or LONGTEXT type => added a conversion to PostgreSQL TEXT type
EDIT3 : the original script did not process ENUM fields, choked on non 7 bits characters, and had a wrong error management
You commented out line 2 where Connect was defined, but you left line 9 where Connect() is used untouched, so the error.
As you now explicitely connect to MySQL, you should replace dbx = Connect() with :
dbx = db.cursor()
It should now give (with the conversion of TEXT types line 28):
import MySQLdb
#from magic import Connect #Private mysql connect information - I COMMENTED THIS LINE to use direct connection
db = MySQLdb.connect(host="localhost", # your host, usually localhost
user="USER", # your username
passwd="PASS", # your password
db="w3i") # name of the data base
import psycopg2
# set client_encoding if different that PostgreSQL database default
encoding = 'Latin1'
dbx=db.cursor()
DB=psycopg2.connect("dbname='w3i'")
DC=DB.cursor()
DC.execute("set client_encoding = " + encoding)
mysql='''show tables from w3i'''
dbx.execute(mysql); ts=dbx.fetchall(); tables=[]
for table in ts: tables.append(table[0])
for table in tables:
mysql='''describe w3i.%s'''%(table)
dbx.execute(mysql); rows=dbx.fetchall()
psql='drop table %s'%(table)
DC.execute(psql); DB.commit()
psql='create table %s ('%(table)
for row in rows:
name=row[0]; type=row[1]
if 'int' in type: type='int8'
if 'blob' in type: type='bytea'
if 'datetime' in type: type='timestamptz'
if 'text' in type: type='text'
if 'enum' in type:
type = 'varchar'
print ("warning : conversion of enum to varchar %s(%s)" % (table, name))
psql+='%s %s,'%(name,type)
psql=psql.strip(',')+')'
print psql
try: DC.execute(psql); DB.commit()
except Exception as e:
print e
DB.rollback()
Above script convert enum to VARCHAR. If you have only one enum type you can try to create it PostgreSQL side :
DC.execute("DROP TYPE IF EXISTS enumtyp CASCADE")
DC.execute("CREATE TYPE enumtyp AS ENUM( ... )"
where enumtyp is the name of the type and ... is the list of (textual) values (don't forget to add an empty value if the field can be empty in MySQL)
Then you replace enum with enumtyp by replacing line type = 'varchar' with :
if 'enum' in type:
type = 'enumtyp'
as reported in the answer you cite:
from magic import Connect #Private mysql connect information
Connect() is (I assume) a method feeding parameters to connect to a specific db.
You thus have either to implement on your own this magic module, with references to your specific parameters, or to specify which connection you want to setup, namely MySQLdb.connect(...) or psycopg2.connect(...)