We Keep Coding

Connecting R to Oracle DB without admin

I need an R script that allows me to connect to an Oracle DB without having to install anything needing admin powers, and preferrably nothing at all apart from package downloads. In python the following code works, I believe because it uses the cx_Oracle module as a portable driver. What would be a good R alternative?
import pandas as pd
import sqlalchemy
import sys
host = "xxx.intra"
database = "mydb"
user = "usr"
password = "pw"
def get_oracle_engine(host, database, user, password):
return sqlalchemy.create_engine("oracle+cx_oracle://{user}:{password}#{host}:1521/?service_name={database}".format(host=host, database=database, user=user, password=password))
engine=get_oracle_engine(host, database, user, password)
pd.read_sql_table("mytable", engine, schema= mydb,index.cols="id1")
I managed to install ROracle using the CRAN instructions but I keep getting the ORA-12154 TNS: cound not resolve the connect identifier specified when using:
library(ROracle)
con= DBI::dbconnect(dbDriver("Oracle"), user= user, password=password, host=host, dbname=database, port="1521")
By the way dbDriver("Oracle") returns
Driver name : Oracle (OCI)
Driver version: 1.3-1
Client version: 12.1.0.2.0

Try code like:
library(DBI)
library(ROracle)
drv <- Oracle()
con <- dbConnect(drv, 'cj', 'welcome', 'localhost:1521/orclpdb1')
dbGetQuery(con,"select count(*) from dual")
The connect string components are related to the {host}:1521/?service_name values you used with SQLAlchemy. Use a TNS alias or Easy Connect String, the same as other C based Oracle drivers, e.g. https://cx-oracle.readthedocs.io/en/latest/user_guide/connection_handling.html#connection-strings
The current ROracle code is at https://www.oracle.com/database/technologies/roracle-downloads.html There are some packaging glitches with uploading to CRAN and the CRAN maintainers haven't been responsive about resolving them.
ROracle still needs Oracle Client libraries such as from Oracle Instant Client.

Python - MySQL SSL Connections

I have a MySQL Server set up to use SSL and I also have the CA Certificate.
When I connect to the server using MySQL Workbench, I do not need the certificate. I can also connect to the server using Python and MySQLdb on a Mac without the CA-certificate.
But when I try to connect using the exact same setup of Python and MySQLdb on a windows machine, I get access denied. It appears that I need the CA. And when I enter the CA, I get the following error
_mysql_exceptions.OperationalError: (2026, 'SSL connection error')
My code to open the connection is below:
db = MySQLdb.connect(host="host.name",
port=3306,
user="user",
passwd="secret_password",
db="database",
ssl={'ca': '/path/to/ca/cert'})
Could anyone point out what the problem is on a windows?

I just got the following to work with Python 2.7 and MySQLdb (1.2.4):
database = MySQLdb.connect(host='hostname', user='username', db='db_name',
passwd='PASSWORD', ssl={'ca': '/path/to/ca-file'})
This is what you had so there must be something else going on here. I wonder if you have something either incorrect with the your local CA file or possibly the cert on the server? Can you get a copy of the CA file from the server?

Try this
import ssl
from databases import Database
sslctx = ssl.create_default_context(ssl.Purpose.SERVER_AUTH,
cafile='ca.pem')
sslctx.verify_mode = ssl.CERT_REQUIRED
sslctx.check_hostname = True
sslctx.load_cert_chain(certfile='client.crt', keyfile='pkey.key')
database = Database(DATABASE_URL, ssl=sslctx)
databases library provides support for PostgreSQL, MySQL, and SQLite.
Also useful for async frameworks.

Im using pymysql and I had some problems to connect using SSL keys and certs: for the "ssl" attribute I set up as a dictionary inside. Try as below:
db = MySQLdb.connect(host="host.name",
port=3306,
user="user",
passwd="secret_password",
db="database",
ssl={'ssl':
{'ca': '/path/to/ca/ca',
'key': '/path/to/ca/key',
'cert': '/path/to/ca/cert'
}
}
)

I know this is a bit old but I found a way to get this to work. Use pymysql instead of MySQLdb and write the connection as:
import pymysql
conn = pymysql.connect(user = 'user', password = 'passwd'
, database = 'db', host = 'hst', ssl = {'ssl' : {'ca': 'pathtosll/something.pem'}})
The point people miss (including myself) is that ssl needs to be a dictionary containing
a key 'ssl' which has another dictionary as a value with a key 'ca'. This should work for you.

import pymysql
conn = pymysql.connect(host= # your host, usually localhost,
user = # your username,
passwd = # your password,
db = #your database name ,
ssl ={'ssl': r'path of your pem file'})

Connect to Impala using impyla client with Kerberos auth

I'm on a W8 machine, where I use Python (Anaconda distribution) to connect to Impala in our Hadoop cluster using the Impyla package. Our hadoop cluster is secured via Kerberos. I have followed the API REFERENCE how to configure the connection.
from impala.dbapi import connect
conn = connect( host='localhost', port=21050, auth_mechanism='GSSAPI',
kerberos_service_name='impala')
We are using Kerberos GSSAPI with SASL
auth_mechanism='GSSAPI'
I have managed to install python-sasl library for WIN8 but still I encounter this error.
Could not start SASL: Error in sasl_client_start (-4) SASL(-4): no mechanism available: No worthy mechs found (code THRIFTTRANSPORT): TTransportException('Could not start SASL: Error in sasl_client_start (-4) SASL(-4): no mechanism available: No worthy mechs found',)
I wonder if I am still missing some dependencies.

Install the kerberos Python package, it will fix your issue.

I ran into the same issue but i fixed it by installing the right version of required libraries.
Install below python libraries using pip:
six==1.12.0
bit_array==0.1.0
thrift==0.9.3
thrift_sasl==0.2.1
sasl==0.2.1
impyla==0.13.8
Below code is working fine with the python version 2.7 and 3.4.
import ssl
from impala.dbapi import connect
import os
os.system("kinit")
conn = connect(host='hostname.io', port=21050, use_ssl=True, database='default', user='urusername', kerberos_service_name='impala', auth_mechanism = 'GSSAPI')
cur = conn.cursor()
cur.execute('SHOW DATABASES;')
result=cur.fetchall()
for data in result:
print (data)

Try this to get tables for kerberized cluster. In my case CDH-5.14.2-1.
Make sure you have a valid ticket before running this code.
with python 2.7 having below packages.
thrift-0.9.3
thriftpy-0.3.8
thrift_sasl-0.3.0
impyla==0.14.2.2
Working Code
from impala.dbapi import connect
from impala.util import as_pandas
# 21000 is impala daemon port.
conn = connect(host='yourHost', port=21050, auth_mechanism='GSSAPI')
cursor = conn.cursor()
cursor.execute("SHOW TABLES")
# After running .execute(), Impala will store the result sets on the server
# until it is fetched. Use the method .fetchall() to pull the entire result
# set over the network (you should only do it if you know dataset is small)
tables = cursor.fetchall()
print("Displaying list of tables")
# the result is a list of tuples
for t in tables:
# we know that each row in SHOW TABLES result
# should only contains one table name
print(t[0])
# exit() enable for only one table
print("eol >>>")

For me, installing this package fixed it: libsasl2-modules-gssapi-mit

For me, the following connection parameters worked. I did not have to install any additional packages in python.
connect(host="your_host", port=21050, auth_mechanism='GSSAPI', timeout=100000, use_ssl=False, ca_cert=None, ldap_user=None, ldap_password=None, kerberos_service_name='impala')

To connection Impala using python you can follow below steps,
Install Coludera ODBC Driver for Impala.
Create DSN using 64-bit ODBC driver, put your server details, below is sample screen shot for same
Use below code snippet for connectivity
import pyodbc
with pyodbc.connect("DSN=impala_con", autocommit=True) as conn:
... df = pd.read_sql("", conn)

python cannot connect hiveserver2
make sure you install cyrus-sasl-devel and cyrus-sasl-gssapi

Trouble connecting to SQL server with python

I have recently installed Microsoft SQL Server 2014 on my PC as I want to create a database for a web application I am building (I have been learning Python for a year and have very basic experience with SQLite).
After installing SQL Server 2014 and creating a database called Users, I am just trying to run some very basic commands to my database but I am falling at the first hurdle over and over!
I have installed pymssql and pyodbc and tried running commands directly with these but have failed. (e.g. pymssql gives me a TypeError: argument of type 'NoneType' is not iterable when I set the variable conn = pymssql.connect(server, user, password, "tempdb")
My latest attempt is to use SQLalchemy to achieve my long awaited connection with SQL database. However, after installing this, it is failing on the following error:
"sqlalchemy.exc.OperationalError: (pymssql.OperationalError) (20009, 'DB-Lib error message 20009, severity 9:\nUnable to connect: Adaptive Server is unavailable or does not exist\nNet-Lib error during Unknown error (10035)\n')"
The question I need answering is, how do I start talking to my database using SQLalchemy?
The code I am using is as follows:
from sqlalchemy import *
engine = create_engine('mssql+pymssql://Han & Lew:#SlugarPlum:1433/Users')
m = MetaData()
t = Table('t', m,
Column('id', Integer, primary_key=True),
Column('x', Integer))
m.create_all(engine)
Yes, my PC is called SlugarPlum. User is Han & Lew. And my server is called THELROYSERVER. DSN = 1433. No password. (I don't know if it is wise that I am giving this information online but the data I have is not sensitive so I guess it's worth a shot.)
Also, if anyone can direct me to an ultra-beginners resource for Python-SQL server that would be awesome as I am getting beaten up by how complex this seems to be!

Here's a connect function and example for connecting via pyodbc. Connecting via pymssql should be as easy as formatting the connecting string for pymssql. I've provided Windows and Linux options, but only tested on Linux. I hope it helps.
def sqlalchemy_connect(connect_string):
""" Connect to the database via ODBC, start SQL Alchemy engine. """
def connect():
return pyodbc.connect(connect_string, autocommit=True)
db = create_engine('mssql://', creator=connect)
db.echo = False
return db
def main():
global DBCONN
# Linux with FreeTDS
connect_string = "DRIVER={FreeTDS};SERVER=<server name>;PORT=<port num>;DATABASE=<db>;UID=<user>;PWD=<password>;TDS_Version=<version num>;"
# Windows with SQL Server
connect_string = "DRIVER={SQL Server};SERVER=<server name>;PORT=<port num>;DATABASE=<db>;UID=<user>;PWD=<password>;"
DBCONN = sqlalchemy_connect(connect_string)
if __name__ == "__main__":
main()

Connecting to SQL Server 2012 using sqlalchemy and pyodbc

I'm trying to connect to a SQL Server 2012 database using SQLAlchemy (with pyodbc) on Python 3.3 (Windows 7-64-bit). I am able to connect using straight pyodbc but have been unsuccessful at connecting using SQLAlchemy. I have dsn file setup for the database access.
I successfully connect using straight pyodbc like this:
con = pyodbc.connect('FILEDSN=c:\\users\\me\\mydbserver.dsn')
For sqlalchemy I have tried:
import sqlalchemy as sa
engine = sa.create_engine('mssql+pyodbc://c/users/me/mydbserver.dsn/mydbname')
The create_engine method doesn't actually set up the connection and succeeds, but
iIf I try something that causes sqlalchemy to actually setup the connection (like engine.table_names()), it takes a while but then returns this error:
DBAPIError: (Error) ('08001', '[08001] [Microsoft][ODBC SQL Server Driver][DBNETLIB]SQL Server does not exist or access denied. (17) (SQLDriverConnect)') None None
I'm not sure where thing are going wrong are how to see what connection string is actually being passed to pyodbc by sqlalchemy. I have successfully using the same sqlalchemy classes with SQLite and MySQL.

The file-based DSN string is being interpreted by SQLAlchemy as server name = c, database name = users.
I prefer connecting without using DSNs, it's one less configuration task to deal with during code migrations.
This syntax works using Windows Authentication:
engine = sa.create_engine('mssql+pyodbc://server/database')
Or with SQL Authentication:
engine = sa.create_engine('mssql+pyodbc://user:password#server/database')
SQLAlchemy has a thorough explanation of the different connection string options here.

In Python 3 you can use function quote_plus from module urllib.parse to create parameters for connection:
import urllib
params = urllib.parse.quote_plus("DRIVER={SQL Server Native Client 11.0};"
"SERVER=dagger;"
"DATABASE=test;"
"UID=user;"
"PWD=password")
engine = sa.create_engine("mssql+pyodbc:///?odbc_connect={}".format(params))
In order to use Windows Authentication, you want to use Trusted_Connection as parameter:
params = urllib.parse.quote_plus("DRIVER={SQL Server Native Client 11.0};"
"SERVER=dagger;"
"DATABASE=test;"
"Trusted_Connection=yes")
In Python 2 you should use function quote_plus from library urllib instead:
params = urllib.quote_plus("DRIVER={SQL Server Native Client 11.0};"
"SERVER=dagger;"
"DATABASE=test;"
"UID=user;"
"PWD=password")

I have an update info about the connection to MSSQL Server without using DSNs and using Windows Authentication. In my example I have next options:
My local server name is "(localdb)\ProjectsV12". Local server name I see from database properties (I am using Windows 10 / Visual Studio 2015).
My db name is "MainTest1"
engine = create_engine('mssql+pyodbc://(localdb)\ProjectsV12/MainTest1?driver=SQL+Server+Native+Client+11.0', echo=True)
It is needed to specify driver in connection.
You may find your client version in:
control panel>Systems and Security>Administrative Tools.>ODBC Data
Sources>System DSN tab>Add
Look on SQL Native client version from the list.

Just want to add some latest information here:
If you are connecting using DSN connections:
engine = create_engine("mssql+pyodbc://USERNAME:PASSWORD#SOME_DSN")
If you are connecting using Hostname connections:
engine = create_engine("mssql+pyodbc://USERNAME:PASSWORD#HOST_IP:PORT/DATABASENAME?driver=SQL+Server+Native+Client+11.0")
For more details, please refer to the "Official Document"

import pyodbc
import sqlalchemy as sa
engine = sa.create_engine('mssql+pyodbc://ServerName/DatabaseName?driver=SQL+Server+Native+Client+11.0',echo = True)
This works with Windows Authentication.

I did different and worked like a charm.
First you import the library:
import pandas as pd
from sqlalchemy import create_engine
import pyodbc
Create a function to create the engine
def mssql_engine(user = os.getenv('user'), password = os.getenv('password')
,host = os.getenv('SERVER_ADDRESS'),db = os.getenv('DATABASE')):
engine = create_engine(f'mssql+pyodbc://{user}:{password}#{host}/{db}?driver=SQL+Server')
return engine
Create a variable with your query
query = 'SELECT * FROM [Orders]'
Execute the Pandas command to create a Dataframe from a MSSQL Table
df = pd.read_sql(query, mssql_engine())