i want to move from SQLite database M, MySQL database. if i understood correctly, so i can do it with changing the engine:
engine = create_engine('sqlite:///foo.db', echo = False)
to:
engine = create_engine('mysql://username:password#localhost/ db_name (the database name) ')
engine = create_engine("mysql+pymysql://user:pass#some_mariadb/dbname?charset=utf8mb4")
or
engine = create_engine("mariadb+pymysql://user:pass#some_mariadb/dbname?charset=utf8mb4")
sqlalchemy-mysql
pip install MySQL-python
On Ubuntu based systems:
apt-get install python-mysqldb
On RHEL/CentOS based systems:
yum install MySQL-python
On Windows:
Unofficial Windows Binaries for Python Extension Packages
I am trying to retrieve data from Kudu. But I am not able to install kudu-python package in anaconda or my server. Can I get some help with it? The documentation on the internet is not really clear.
#Karthik, did you encounter any errors? I just installed kudu-python client on Anaconda on Centos 6.9. There was one gotcha with versioning, but otherwise it was straightforward. The only error I ran into was
kudu/client.cpp:589:30: fatal error: kudu/util/int128.h: No such file or directory
there is a solution for it here: https://community.cloudera.com/t5/Data-Ingestion-Integration/can-not-install-kudu-python/td-p/67496
Otherwise, the steps are:
1. Install kudu client libraries as described on Kudu website (https://kudu.apache.org/docs/installation.html#_install_on_rhel_or_centos_hosts):
wget http://archive.cloudera.com/kudu/redhat/6/x86_64/kudu/cloudera-kudu.repo
sudo mv cloudera-kudu.repo /etc/yum.repos.d/
sudo yum update
sudo yum install kudu kudu-client0 kudu-client-devel
install a bunch of dev dependencies if you don't have them already:
sudo yum install autoconf automake libtool make gcc gcc-c++
install Cython and kudu-python
pip install Cython kudu-python==1.2.0
Once you have this installed, you can find examples in https://github.com/apache/kudu/tree/master/examples/python
i had no ability to install kudu-client (windows os is not supported) so i used the cluster's Impala to get Kudu's tables:
from impala.dbapi import connect
conn = connect('<Impala Daemon>', port=21050)
cursor = conn.cursor()
cursor.execute('SELECT * FROM mytable LIMIT 100')
print(cursor.description) # prints the result set's schema
results = cursor.fetchall()
https://github.com/cloudera/impyla
I am using python MySQL 5.7.19 on Ubuntu 16.04.1 and I am
trying to run the following code
import MySQLdb
db= MySQLdb.connect("localhost","root",YES,"testDB")
cursor=db.cursor()
cursor.execute("DROP TABLE IF EXISTS USER")
sql="""CREATE TABLE USER(
ID INT(12) NOT NULL AUTO_INCREMENT,
USERNAME VARCHAR(20) NOT NULL,
PASSWORD VARCHAR(20) NOT NULL,
PRIMARY KEY (ID)
)"""
cursor.execute(sql)
db.close()
When I save this in a .py file and execute it as Python3 I get the following error:
$ python3 try.py
Traceback (most recent call last):
File "try.py", line 1, in
import MySQLdb
ImportError: No module named 'MySQLdb'
How do I solve this?
install the package
pip install MySQL-python
or on Ubuntu
sudo apt-get install python-pip python-dev libmysqlclient-dev
Install the package using this command
sudo apt-get install python-mysqldb
and then use these thing to avoid non ascii character issue and persistent mysql db connection issue.
import MySQLdb
db = MySQLdb.connect("localhost", "root", "db_pswd", "mysql_db", charset='utf8', use_unicode=True)
db.ping(True)
use sudo apt-get install python-mysqldb don't specify python version, when system try to install, it will find the current python version in this system.
I'm trying to write a python app (python 3.5 with pyCharm IDE) that put multiple queries and do math etc... It needs to run on both mac and windows. On windows side because of pymssql won't work i tried pyodbc which worked flawlessly (it's the opposite on mac side). But on the mac side whenever i try to connect with this code:
cnxn = pyodbc.connect('DRIVER={SQL Server};SERVER=myServerIp;DATABASE=myDatabaseName;UID=myUserName;PWD=myPassword')
cursor = cnxn.cursor()
it gives the error of: pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib 'odbc.ini' : file not found (0) (SQLDriverConnect)")
Normally i can connect MS SQL from excel or any other query software on mac. I'm using Actual SQL Server driver on system wide.
So please guide me to fix this problem either using Actual SQL Server drivers (which is paid version) or properly configure python or unixodbc (not sure which is the problem). Thank you and regards
PS: I installed both pyodbc, unixodbc and freetds via pip as i followed documentation
Update1:
$ cat $(odbc_config --odbcinstini)
[ODBC]
DEBUG=1
TraceFile=/home/ftp/sqltrace.log
Trace=Yes
[FreeTDS]
Description=MSSQL Driver
Driver=/usr/local/lib/libtdsodbc.so
$ odbc_config --ulen
-DSIZEOF_SQLULEN=8
$ odbc_config --libs
-L/usr/local/Cellar/unixodbc/2.3.2_1/lib -lodbc
$ odbc_config --prefix
/usr/local/Cellar/unixodbc/2.3.2_1
$ odbc_config --odbcinstini
/usr/local/Cellar/unixodbc/2.3.2_1/etc/odbcinst.ini
$ odbc_config --odbcini
/usr/local/Cellar/unixodbc/2.3.2_1/etc/odbc.ini
$ odbc_config --version
2.3.2
uninstalling and installing again with this command fixed the issue:
brew install freetds --with-unixodbc
I'm new to the linux world and I want to query a Microsoft SQL Server from Python. I used it on Windows and it was perfectly fine but in Linux it's quite painful.
After some hours, I finally succeed to install the Microsoft ODBC driver on Linux Mint with unixODBC.
Then, I set up an anaconda with python 3 environment.
I then do this :
import pyodbc as odbc
sql_PIM = odbc.connect("Driver={ODBC Driver 13 for SQL Server};Server=XXX;Database=YYY;Trusted_Connection=Yes")
It returns :
('01000', "[01000] [unixODBC][Driver Manager]Can't open lib '/opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.0.so.0.0' : file not found (0) (SQLDriverConnect)")
The thing I do not undertsand is that PyODBC seems to read the right filepath from odbcinst.ini and still does not work.
I went to "/opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.0.so.0.0" and the file actually exists !
So why does it tell me that it does not exist ?
Here are some possible clues :
I'm on a virtual environment
I need to have "read" rights because it's a root filepath
I do not know how to solve either of these problems.
Thanks !
I also had the same problem on Ubuntu 14 after following the microsoft tutorial for SQL Server Linux ODBC Driver.
The file exists and after running an ldd, it showed there were dependencies missing:
/opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.0.so.0.0: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version GLIBCXX_3.4.20' not found (required by /opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.0.so.0.0)
/opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.0.so.0.0: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: versionCXXABI_1.3.8' not found (required by
after searching for a while I found its because Ubuntu's repo didnt have GLIBCXX on version 3.4.20, it was at 3.4.19.
I then added a repo to Ubuntu, updated it and forced it to upgrade libstdc++6
sudo add-apt-repository ppa:ubuntu-toolchain-r/test
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install libstdc++6
Problem solved, tested with isql:
+---------------------------------------+
| Connected! |
| |
| sql-statement |
| help [tablename] |
| quit |
| |
+---------------------------------------+
SQL>
After that I tried testing using pdo_odbc (PHP), it then gave me the same driver not found error.
To solve this I had to create a symbolic link to fix libodbcinst.so.2:
sudo ln -s /usr/lib64/libodbcinst.so.2 /lib/x86_64-linux-gnu/libodbcinst.so.2
I had the same problem 'file not found (0) (SQLDriverConnect)' on MAC OS with the following code
cnxn = pyodbc.connect('DRIVER={ODBC Driver 13 for SQL Server};SERVER=myServerIP,1433;DATABASE=myDBName;UID=sa;PWD=dbPassword')
after googling for two days, I cannot fix the issue even modify the freetds.conf, odbcinst.ini and odbc.ini
finally, I found the solution via replacing DRIVER value
cnxn =
pyodbc.connect('DRIVER={/usr/local/lib/libmsodbcsql.13.dylib};SERVER=myServerIP,1433;DATABASE=myDBName;UID=sa;PWD=dbPassword')
My dev environment
MAC OS El Capitan
python 3.6.1 in Anaconda
I found an answer that works for me here. This is for python 2.7 (so may not work for those who are looking for a solution for python 3.x).
The suggested solution is to update libgcc: 4.8.5-2 --> 5.2.0-0
For updating libgcc, use this command
conda update libgcc
The following suggestions may help to solve the problem:
Make sure the drive configuration INI file exists odbcinst -j (check odbcinst.ini).
Make sure the filepath to configured driver from your INI file (run: odbcinst -j) exist and has read and executable permission flags (O_RDONLY|O_CLOEXEC).
If you still got file not found error, despite the file exists, the problem could be related to libgcc version mismatch as per nehaljwani's GitHub comment. The solution is to update your libgcc by running conda update libgcc command.
Related: ODBC Driver 13 for SQL Server can't open lib on pyodbc while connecting on AWS E2 ubuntu instance.
For macOS, see: Installing ODBC via HomeBrew.
Maybe it is a bit late, but I leave this scripts that worked for me.
My problem was the same as yours and I tested all the options such as changing the driver location, making a symbolic link, modify /etc/*.ini files, etc... nothing worked.
My problem, running python 3.6, pyodbc package in a docker container from alpine was the library libssl1.0.0
Here you will find my installation script for pyodbc Debian 8 (alpine) docker image using the driver v13
DRIVER={ODBC Driver 13 for SQL Server}
The command I run for database connection was:
import pyodbc
connection_string = 'DRIVER={ODBC Driver 13 for SQL Server};'
connection_string += 'SERVER={0};DATABASE={1};UID={2};PWD={3};'.format(host,dbname,user,pwd)
connection = pyodbc.connect(connection_string)
I solve this problem after installing libssl1.0.0.
First, I setup my connection string in this way:
cnxn = pyodbc.connect('DRIVER={/usr/local/lib/libmsodbcsql.13.dylib};
SERVER=myServerIP,1433;DATABASE=myDBName;UID=sa;PWD=dbPassword')
Then, I installed libssl1.0.0:
echo "deb http://security.debian.org/debian-security jessie/updates main" >> /etc/apt/sources.list
apt-get install libssl1.0.0
Finnaly, I setup the locales:
apt-get -y install locales
echo "en_US.UTF-8 UTF-8" > /etc/locale.gen
After doing these steps, my python module was able to find and connect to database.
had the same issue once..
1.try conda update libgcc(this is because pyodbc installed through pip and conda look for different versions of the file..)..this might have been fixed .....
link:https://github.com/ContinuumIO/anaconda-issues/issues/1639
look for nehaljwani answer .
2.also check the version number of the odbc file correctly in /etc/odbcinst.ini and /etc/odbc.ini ...names should match and also the driver path.
Had the same problem on a Mac mini M1 with macOS Mavericks. After installing the driver 18 from Microsoft which supports ARM it still did not work.
brew tap microsoft/mssql-release https://github.com/Microsoft/homebrew-mssql-release
brew update
HOMEBREW_NO_ENV_FILTERING=1 ACCEPT_EULA=Y brew install msodbcsql18 mssql-tools18
However isql on the commandline was able to connect to the database.
isql -v -k "DRIVER={ODBC Driver 18 for SQL Server};SERVER=<MYSERVERNAME>;PORT=<MYPORT>;DATABASE=<MYDATABASE>;UID=<USERNAME>;PWD=<PASSWORD>"
Finally what did the trick was to uninstall pyodbc and install it again.
python -m pip uninstall pyodbc
python -m pip install pyodbc