I want to create an Access database (*.accdb) from within a Python script.
Using win32com and Dispatch I can call the application. However, I cant find anything on how to create a new database.
access = win32com.client.Dispatch('Access.Application')
At that point I have no need to put data into the database and I would do this using pyodbc - I simply need to create an empty database.
Does somebody has an example on how to do this?
Cheers Thomas
You have an Access application object. Use its DBEngine.CreateDatabase method to create your db file.
This sample worked from Python 2.7 to create an MDB format database file. To create an ACCDB, use 128 (dbVersion120) for dbVersion.
import win32com.client
oAccess = win32com.client.Dispatch('Access.Application')
DbFile = r'C:\Users\hans\Documents\NewDb.mdb'
dbLangGeneral = ';LANGID=0x0409;CP=1252;COUNTRY=0'
# dbVersion40 64
dbVersion = 64
oAccess.DBEngine.CreateDatabase(DbFile, dbLangGeneral, dbVersion)
oAccess.Quit()
del oAccess
To create a new, empty .accdb file, the following Python code should do the trick:
import win32com.client
f = 'C:\\Users\\Gord\\Desktop\\pyTest.accdb'
c = win32com.client.Dispatch('ADOX.Catalog')
c.Create('Provider=Microsoft.ACE.OLEDB.12.0;Data Source=' + f + ';')
c = None
print '"' + f + '" created.'
[Edit 1]
A comment to a blog posting here suggests that if the .Create call generates a "Class not registered" error you may need to use regsvr32.exe to re-register msadox.dll. Be aware of "bitness" when you attempt this: There are 32-bit and 64-bit versions of both of those files:
64-bit
C:\Windows\System32\regsvr32.exe
C:\Program Files\Common Files\System\ado\msadox.dll
32-bit
C:\Windows\SysWOW64\regsvr32.exe
C:\Program Files (x86)\Common Files\System\ado\msadox.dll
Also, be aware that you could be running 32-bit Python on a 64-bit machine.
[Edit 2]
I've done a few tests and have reached the conclusion that this approach did not work in this particular case because the Python script was running as 64-bit, but the 64-bit Access Database Engine was not installed. (32-bit Office only installs the 32-bit version of ACE.)
The error message was perhaps a bit misleading. It wasn't the ADOX component that was missing (not registered), it was the 64-bit version of the ACE engine itself that couldn't be found.
Furthermore, on a 64-bit machine with 32-bit Access installed, the 64-bit version of ACE will never be available because it cannot be installed
This could very well have implications when you try to manipulate data within the .accdb file from a 64-bit Python script. I didn't have Python available on my "32-bit Office on 64-bit Windows" test machine, but when I tried the following VBScript...
Option Explicit
Dim con, rst
Set con = CreateObject("ADODB.Connection")
con.Open "Provider=Microsoft.ACE.OLEDB.12.0;Data source=C:\Users\Gord\Desktop\adoTest.accdb;"
Set rst = CreateObject("ADODB.Recordset")
rst.Open "SELECT Field1 FROM Table1", con
Wscript.Echo rst(0).Value
rst.Close
Set rst = Nothing
con.Close
Set con = Nothing
...the results were as follows:
C:\__tmp>C:\Windows\System32\cscript.exe /nologo dataAccessTest.vbs
C:\__tmp\dataAccessTest.vbs(4, 1) ADODB.Connection: Provider cannot be found.
It may not be properly installed.
C:\__tmp>C:\Windows\SysWOW64\cscript.exe /nologo dataAccessTest.vbs
This is Table1 data in Access.
The script failed when run as 64-bit, but it worked when run as 32-bit.
Recommendation: If your machine has 32-bit Access installed you'll probably be better off running your Python scripts as 32-bit too.
import win32com.client
ConFileName = r'c:\mydb\myaccess.mdb'
try:
Catalog = win32com.client.Dispatch('ADOX.Catalog')
Catalog.Create('Provider=Microsoft.ACE.OLEDB.12.0;Data Source=' + ConFileName + ';')
Catalog = None
except:
Exception as e:
print("Database generation failed, Error="+str(e))
print("NewAccessDB.mdb created successfully")
Then you can connect to access database :
ConFileName=(r'c:\mydb\myaccess.mdb')
conn = pyodbc.connect(r'Driver={Microsoft Access Driver (*.mdb, *.accdb)};DBQ=' + ConFileName + ';')
cursor = conn.cursor()
To insert in to the access table :
ConFileName=(r'c:\mydb\myaccess.mdb')
conn = pyodbc.connect(r'Driver={Microsoft Access Driver (*.mdb, *.accdb)};DBQ=' +ConFileName+ ';')
cursor = conn.cursor()
for ta in TableArray:
Sql_insert_query = "INSERT INTO Table1(ID, Value1,Value2,Value3,Value4,Value5,Value6) " \
"VALUES ('{a}','{b}','{c}','{d}','{e}','{f}','{g}')".format(a=str(ta[0]),b=str(ta[1]),c=str(ta[2]),d=str(ta[3]),e=str(ta[4]),f=str(ta[5]),g=str(ta[6]))
cursor.execute(Sql_insert_query)
conn.commit()
cursor.close()
Please follow this link for more information :
https://elvand.com/python-and-ms-access/
Related
I am trying to load a .csv file through mysql.connector in python. Here is the relevant portion of the script:
import os
print(os.listdir(r'C:\ProgramData\MySQL\MySQL Server 8.0\Uploads')) #returns test.csv
import_file = 'C:\\ProgramData\\MySQL\\MySQL Server 8.0\\Uploads\\test.csv'
cnx = mysql.connector.connect(user=usr,
password=passwd,
host='127.0.0.1',
database='ctr')
cursor = cnx.cursor()
load_data = (
"LOAD DATA "
"INFILE "
"'{}' "
"INTO TABLE import "
"FIELDS TERMINATED BY ',' "
"""OPTIONALLY ENCLOSED BY '"' """
"LINES TERMINATED BY '\\n' "
"IGNORE 1 LINES".format(import_file))
securefile_query = ("SHOW VARIABLES LIKE 'secure_file_priv'")
cursor.execute(securefile_query)
directory = cursor.fetchall()
print(directory) #returns C:\ProgramData\MySQL\MySQL Server 8.0\Uploads
print(import_file)
print(load_data)
cursor.execute(load_data)
Traceback of the error that follows points to cursor.execute(load_data) in my script, and the last line of the traceback is "mysql.connector.errors.DatabaseError: 1290 (HY000): The MySQL server is running with the --secure-file-priv option so it cannot execute this statement".
According to the secure_file_priv documentation:
If empty, the variable has no effect. This is not a secure setting.
If set to the name of a directory, the server limits import and export operations to work only with files in that directory. The directory must exist; the server does not create it.
If set to NULL, the server disables import and export operations.
I have confirmed multiple times that the file is in the correct directory. I have also confirmed the value of secure_file_priv using "SHOW VARIABLES". Doesn't this mean MySQL ought to be able to read test.csv from the Uploads folder? How can I read the file without disabling secure_file_priv?
Edit 17Jan2022:
Since I'm running the server locally and have the files locally, I ended up using LOAD DATA LOCAL INFILE as a workaround by adding "allow_local_infile=True" as a parameter inside cnx and then moving the source file to the same directory as the script. Setting secure-file-priv to an empty string (C:\ProgramData\MySQL\MySQL Server 8.0\my.ini) also worked.
I still don't know why I was receiving "server is running with the --secure-file-priv option" given the previous location of my files.
I'm trying to export a table, contained within an Oracle 12c database, to csv format - using Python 2.7. The code I have written is shown below:
import os
import cx_Oracle
import csv
SQL = 'SELECT * FROM ORACLE_TABLE'
filename = 'C:\Temp\Python\Output.csv'
file = open(filename, 'w')
output = csv.writer(file, dialect='excel')
connection = cx_Oracle.connect('username/password#connection_name')
cursor = connection.cursor()
cursor.execute(SQL)
for i in cursor:
output.writerow(i)
cursor.close()
connection.close()
file.close()
This code yields an error in the line where I define 'connection':
ORA-12557: TNS:protocol adapter not loadable
How can I remedy this? Any help would be appreciated.
Please note: I have already encountered StackOverflow responses to very similar problems to this. However, they often suggest changing the path within environment variables - I cannot do this since I don't have appropriate administer privileges. Thanks again for your assistance.
ORA-12557 is caused by problems with the %ORACLE_HOME% on Windows. That's the usual suggestion is to change the PATH setting.
"I cannot do this since I don't have appropriate administer privileges."
In which case you don't have too many options. Perhaps you could navigate to the ORACLE_HOME directory and run your script from there. Otherwise look to see what other tools you have available: Oracle SQL Developer? TOAD? SQL*Plus?
We found that by navigating to config -> Oracle and editing the file 'tnsnames.ora' the problem can be solved. The tnsnames file appears as follows:
connection_name =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS= ... )
)
(CONNECT_DATA =
(SERVICE_NAME= ...)
)
)
By changing the first instance of connection_name to connection_name.WORLD, then typing
set ORACLE_HOME=
into the command line before executing the Python script, the above script now runs with no error.
I use ini-file to store DB connection parameters. Hope it helps.
self.mydsn = cx_Oracle.makedsn(self.parser.get('oracle', 'db'),self.parser.get('oracle', 'port'),self.parser.get('oracle', 'service_name'))
try:
self.connpool = cx_Oracle.SessionPool(user=self.parser.get('oracle', 'username'),password=self.parser.get('oracle', 'userpass'),dsn=self.mydsn,min=1,max=5,increment=1)
except Exception as e:
print e
You can use this python script for oracle csv export:
https://github.com/teopost/csv_exp
I have about 40 MS Access Databases and have some troubles if need to create or transfer one of MS Access Query (like object) from one db to other dbs.
So I tried to solve this problem with pyodbc but.. as I saw pyodbc doesn't support to create new, permanent MS Access Query (object).
I can connect to db, create or delete tables/rows but can't to create and save new query.
import pyodbc
odbc_driver = r"{Microsoft Access Driver (*.mdb, *.accdb)}"
db_test1 = r'''..\Test #1.accdb'''
db_test2 = r'''..\Test #2.accdb'''
db_test3 = r'''..\Test #3.accdb'''
db_test4 = r'''..\Test #4.accdb'''
db_test_objects = [db_test1, db_test2, db_test3, db_test4]
odbc_conn_str = "Driver=%s;DBQ=%s;" % (odbc_driver, db_file)
print (odbc_conn_str)
conn = pyodbc.connect(odbc_conn_str)
odbc_cursor = conn.cursor()
NewQuery = "CREATE TABLE TestTable(symbol varchar(15), leverage double)"
odbc_cursor.execute(NewQuery)
conn.commit()
conn.close()
SO, How to create and save MS Access Query like objects from python?
I tried to search info in Google, but the answers were related with Run SQL code.
On VBA this code looks like:
Public Sub CreateQueryDefX()
Dim base(1 To 4) As String
base(1) = "..\Test #1.accdb"
base(2) = "..\Test #2.accdb"
base(3) = "..\Test #3.accdb"
base(4) = "..\Test #4.accdb"
For i = LBound(base) To UBound(base)
CurrentBase = base(i)
Set dbo = OpenDatabase(CurrentBase)
With dbo
Set QueryNew = .CreateQueryDef("TestQuery", _
"SELECT * FROM TestTable")
RefreshDatabaseWindow
.Close
End With
Next i
RefreshDatabaseWindow
End Sub
Sorry for my English, it's not my native :)
By the way, I know how to solve this by VBA, but I'm interested in solve this by python.
Thank you.
You can use a CREATE VIEW statement to create a saved Select Query in Access. The pyodbc equivalent to your VBA example would be
crsr = conn.cursor()
sql = """\
CREATE VIEW TestQuery AS
SELECT * FROM TestTable
"""
crsr.execute(sql)
To delete that saved query you could simply execute a DROP VIEW statement.
For more information on DDL in Access see
Data Definition Language
Consider the Python equivalent of the VBA running exactly what VBA uses: a COM interface to the Access Object library. With Python's win32com third-party module, you can call the CreateQueryDef method. Do note: this COM interfacing can be applied in other languages such as PHP and R!
Below uses a try/except/finally block to ensure the Access application process closes regardless of error or success of code (similar to VBA's On Error handling):
import win32com.client
# OPEN ACCESS APP AND DATABASE
dbases = ["..\Test #1.accdb", "..\Test #2.accdb", "..\Test #3.accdb", "..\Test #4.accdb"]
try:
oApp = win32com.client.Dispatch("Access.Application")
# CREATE QUERYDEF
for db in dbases:
oApp.OpenCurrentDatabase(db)
currentdb = oApp.CurrentDb()
currentdb.CreateQueryDef("TestQuery", "SELECT * FROM TestTable")
currentdb = None
oApp.DoCmd.CloseDatabase
except Exception as e:
print(e)
finally:
currentdb = None
oApp.Quit
oApp = None
Also, if you need to run DML statements via pyodbc and not a COM interface, consider distributed queries as Access can query other databases directly in SQL. Below should work in Python (be sure to escape the backslash):
SELECT t.* FROM [C:\Path\To\Other\Database.accdb].TestTable t
This has been a long running issue for me. I have a proprietary database that I cannot change and many of the tables have fields which are defined as e.g. decimal(12, 4).
When I try to pull data from such a table on ubuntu 12.04 using pyodbc/freeTDS like this...
import pyodbc
connection_string = 'DRIVER={FreeTDS};DSN=<myDSN>;UID=<my_user>;PWD=<my_password>;'
conn = pyodbc.connect(connection_string)
cur = conn.cursor()
cur.execute('SELECT myfield FROM mytable')
for row in cur.fetchall():
print row[0]
...I get a really unhelpful message.
Traceback (most recent call last): File
"/path/to/testing_pyodbc.py", line 6, in <module>
for row in cur.fetchall(): pyodbc.Error: ('HY000', 'The driver did not supply an error!')
Whereas if I cast the result to a float the query runs with no problem.
import pyodbc
connection_string = 'DRIVER={FreeTDS};DSN=<myDSN>;UID=<my_user>;PWD=<my_password>;'
conn = pyodbc.connect(connection_string)
cur = conn.cursor()
cur.execute('SELECT CAST(myfield AS FLOAT) FROM mytable')
for row in cur.fetchall():
print row[0]
My first question is can I fix this problem without changing the table structure? The database is not mine so I have no access to change it.
I would like to use SQLAlachemy to grab this data from the database. I am doing so happily on Windows like this.
class MyTable(Base):
__tablename__ = u'table'
...
myfield = Column(DECIMAL(12, 4), nullable=True)
another_field = Column(DECIMAL(12, 4), nullable=True)
...
My second question (if the first is not solvable) is can I define my sqlAlchemy class to automatically cast the data to a float under the hood so the code that uses the class needn't worry about it?
I am running ubuntu 12.04 so the installed version of freetds is 0.91:
$ dpkg -s freetds-common
Package: freetds-common
Status: install ok installed
Multi-Arch: foreign
Priority: optional
Section: libs
Installed-Size: 91
Maintainer: Ubuntu Developers <ubuntu-devel-discuss#lists.ubuntu.com>
Architecture: all
Source: freetds
Version: 0.91-1
Replaces: libct3, libct4 (<< 0.82-1)
Description: configuration files for FreeTDS SQL client libraries
FreeTDS is an implementation of the Tabular DataStream protocol, used for
connecting to MS SQL and Sybase servers over TCP/IP.
.
This package manages the configuration files that are common to all of
the TDS client library implementations (CT-Lib, DB-Lib, and ODBC),
stored in /etc/freetds/.
Original-Maintainer: Steve Langasek <vorlon#debian.org>
Homepage: http://www.freetds.org/
But when I ask tsql, it tells me v0.64:
$ tsql -C
Compile-time settings (established with the "configure" script):
Version: freetds v0.64
MS db-lib source compatibility: no
Sybase binary compatibility: unknown
Thread safety: yes
iconv library: yes
TDS version: 5.0
iODBC: no
unixodbc: yes
Also note that when I use tsql or isql on the command line, they are happy to give me the data without the CAST() operation.
So this is my requirement. I've some use python module to execute .sql files on different databases such as oracle, MSSQL, db2, etc. on different platforms such linux and windows. In my requirement i can't open the sql file and run each command. I've to run .sql file as whole. I was reading about sql alchemy but it seems as it has to execute each statement. So is there is any module to run the complete .sql file
PS: - I've .sql file with respect to each database, i.e. if I've abc.sql for oracle then every statement is compatible to run on oracle database and if this abc.sql file is not supposed to run on MSSQL or DB2, my program will not execute it on these database.
No. There is standard SQL language, but this standard says only how SELECT, INSERT etc should work. There is nothing about .sql files. This way each DBMS vendor has his own ways of working with such files. In PostgreSQL you can easily run psql command line programm with your file as input. In Oracle world you can try to do the same using sqlplus but of course such .sql files will vary. In PostgreSQL you should set encoding of input file using non-standard SQL command that other vendors will report as error. For such things Oracle uses environment settings. With Oracle your .sql file for sqlplus must end with COMMIT; EXIT; etc. Even datetime string literals are different for each vendor. MS SQL uses {ts '...'} which will not work with Informix, PostgreSQL nor Oracle.
Shortly: it seems impossible to do such program for each database.
All you can do is to invite additional layer that will change your standard input file (add some header and footer, convert datetime literals etc). Then such prepared file can be run against command line tools given by database vendor, or by your specialized program able to execute such converted file.
EDIT:
It seems that you have different files for different databases, and even for different database versions. You can also use native programs that are able to run .sql file. So the only problem is to detect database and database version and execute proper file using proper native client. This is code in Jython (I often use JDBC, but you can use DB-API and Python db drivers):
def get_db_version(db):
dbname = ''
ver = ''
c = db.createStatement()
try:
rs = c.executeQuery("SELECT FIRST 1 DBINFO('version','full') FROM systables")
dbname = 'informix'
# IBM Informix Dynamic Server Version 11.50.FC4
except:
try:
rs = c.executeQuery("SELECT * FROM v$version WHERE banner LIKE 'Oracle%'")
dbname = 'oracle'
# Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
except:
try:
rs = c.executeQuery("SELECT version()")
dbname = 'postgresql'
# PostgreSQL 9.2.4 on ..., 64-bit
# PostgreSQL 9.3.0 on ..., 64-bit
except:
raise
if dbname:
while (rs.next()):
ver = rs.getString(1)
return dbname, ver
def select_sql_app_and_file():
import_app = None
ver_postfix = ''
dbname, ver = get_db_version(create_db_connection())
if dbname == 'postgresql':
import_app = 'psql'
if 'PostgreSQL 9.2' in ver:
import_app = 'psql92'
ver_postfix = '92'
elif dbname == 'oracle':
import_app = 'sqlplus'
if 'Release 11.' in ver:
import_app = 'sqlplus11'
ver_postfix = '11'
# ...
sql_file_name = 'import_' + dbname + ver_postfix + '.sql'
return import_app, sql_file_name
def run_sql_file():
import_app, sql_file_name = select_sql_app_and_file()
if import_app:
execute_import_app(import_app, sql_file_name)