Is there a module or a connector that allows Python directly access Hive tables? Any help would be greatly appreciated.
Python has a Pyhive library that you can use to connect to hive database and run query against them.
it allows you to connect through JDBC and Kerberos Authentication. However, it's bit buggy and JDBC over SSL seems not supported.
This is how you connect
connection = hive.connect(host='HIVE_HOST',
port=10000,
database='temp',
username='HIVE_USERNAME',
password='HIVE_PASSWORD',
auth='CUSTOM')
You can find more details here.
http://allabouthadoop.net/how-to-access-hive-with-python-script/
And these are the two issues I am talking about
https://github.com/dropbox/PyHive/issues/258
https://github.com/dropbox/PyHive/issues/257
Related
I need to connect ElasticSearch to Superset for visualization. When I checked in Superset in Sources> Databases, it mentioned to use SQLAlchemy URI and Database for testing the connection.
In our case, ElasticSearch is connected with Python library and not using SQLAlchemy.
Is there any way to connect Elastic Search with Superset using Python library and if so could you please help by mentioning the way to connect?
you can connect by the way mentioned in the docs.You need to add the dbapi python package mentioned in superset installer document and that will in turn help you connect to the elk using the url elasticsearch+http://{user}:{password}#{host}:9200/
pip install elasticsearch-dbapi
How do I find the SQL azure driver for my pypyodbc application.
[gte 2.0] -->
Content relevant to Named Framework versions 2.0 and greater.
There was an existing SO thread pypyodbc: [Microsoft][ODBC SQL Server Driver][DBNETLIB]ConnectionWrite (send()), which got some issue when using pypyodbc. Based on your short & unclear description, I don't know why you want to use it to connect SQL Azure, and whether you will also get the similar issue. So I suggest you can consider for using Azure offical recommended Python driver pyodbc or pymssql, please see the document for them to knwo how to install & use these driver.
Is there any Python API that will allow me to interact with a remote MySQL db (running on RDS) in a transactional manner (read, write)? I believe this can be done using Boto for DynamoDB but I couldn't find anything similar for MySQL.
Any suggestions?
Thanks,
Bucho
There is no such thing in Boto,
The easiest and simplest library to interact with MySQL using python in my opinion is MySQLdb.
I am trying to build a tool that one step of it is connecting to MySQL database.
Just I am so confused about ODBC. If I want to build a cross platform connector by python, should I use python connector or ODBC connector?
I know JDBC, but ODBC stands for Open Database Connectivity. It looks like more compatible.
Could anyone help me clarify that? Thank you very much.
Python has its own DB API v2.0 abstraction layer to connect to databases (it serves the same purpose of ODBC or JDBC but for Python). You should use one of the DB API v2.0 compliant oursql, MySQLdb or PyMySQL packages to connect to MySQL from Python. All these packages are cross platform and will work on Linux, Windows and MacOS X. oursql, and MySQLdb are wrappers for libmysql and PyMySQL is a pure Python implementation.
Note that there are DB API v2.0 implementations (like pyodbc) that provide ODBC connectivity, so conceptually you could connect to MySQL via ODBC, but this would have inferior performance than the above mentioned "native" drivers because of the extra abstraction layers.
I have a Firebird DB set up on my computer and I want to be able to retrieve data from a different computer. What is the best way to go about this?
I am running windows and using python.
Install firebird client to the client pc
To connect firebird programmaticaly from python you should install a python Firebird driver.
For Python 2.x, you can use kinterbasdb. This is the legacy driver and I think it is not actively developed but only maintained.
To connect windows based server database from kinterbasdb you can use
Import kinterbasdb as k
k.init(type_conv = 300) #
con = k.connect(dsn='127.0.0.1:c:\\db\\test.fdb', user='sysdba', password='masterkey', charset='YOUR_CHARSET', dialect=3)
Of course you should adjust connection parameters according to your system. Kinterbasdb documentation is here
If you want use an ORM, you can use SqlAlchemy which uses kinterbasdb for Firebird Support
For Python 3k you can use pyfirebirdsql which also supports Python 2.5+ and under active development, but not supported by SqlAlchemy yet.
Run Firebird server on the computer with database file and connect to it from remote computer. You will need in Firebird client library installed on remote computer.
I think we need a bit more info.
Do you want database access - as in "I want to be able to edit table layout and define new tables, views, procedures and so on" ?
Or do you only need to get data from the database using python ?
The latter could be achieved by installing a Firebird client (in essence its a dll (fbclient.dll)) and then use a connect string from python to connect to your database.