We Keep Coding

Connecting to jTDS Microsoft server with SQLalchemy and Presto

I'm trying to connect to an oldschool jTDS ms server for a variety of different analysis tasks. Firstly just using Python with SQL alchemy, as well as using Tableau and Presto.
Focusing on SQL Alchemy first at the moment I'm getting an error of:
Data source name not found and no default driver specified
With this, based on this thread here Connecting to SQL Server 2012 using sqlalchemy and pyodbc
i.e,
import urllib
params = urllib.parse.quote_plus("DRIVER={FreeTDS};"
"SERVER=x-y.x.com;"
"DATABASE=;"
"UID=user;"
"PWD=password")
engine = sa.create_engine("mssql+pyodbc:///?odbc_connect={FreeTDS}".format(params))
Connecting works fine through Dbeaver, using a jTDS SQL Server (MSSQL) driver (which is labelled as legacy).
Curious as to how to resolve this issue, I'll keep researching away, but would appreciate any help.
I imagine there is an old drive on the internet I need to integrate into SQL Alchemy to begin with, and then perhaps migrating this data to something newer.
Appreciate your time

why should we set the local_infile=1 in sqlalchemy to load local file? Load file not allowed issue in sqlalchemy

I am using sqlalchemy to connect to MySQL database and found a strange behavior.
If I query
LOAD DATA LOCAL INFILE
'C:\\\\Temp\\\\JaydenW\\\\iata_processing\\\\icer\\\\rename\\\\ICER_2017-10-
12T09033
7Z023870.csv
It pops an error:
sqlalchemy.exc.InternalError: (pymysql.err.InternalError) (1148, u'The used
command is not allowed with this MySQL versi
on') [SQL: u"LOAD DATA LOCAL INFILE
'C:\\\\Temp\\\\JaydenW\\\\iata_processing\\\\icer\\\\rename\\\\ICER_2017-10-
12T090337Z023870.csv' INTO TABLE genie_etl.iata_icer_etl LINES TERMINATED BY
'\\n'
IGNORE 1 Lines (rtxt);"] (Background on this error at:
http://sqlalche.me/e/2j85)
And I find the reason is that:
I need to set the parameter as
args = "mysql+pymysql://"+username+":"+password+"#"+hostname+"/"+database+"?
local_infile=1"
If I use MySQL official connection library. I do not need to do so.
myConnection = MySQLdb.connect(host=hostname, user=username, passwd=password, db=database)
Can anyone help me to understand the difference between the two mechanisms?

The reason is that the mechanisms use different drivers.
In SQLAlchemy you appear to be using the pymysql engine, which uses the PyMySQL Connection class to create the DB connection. That one requires the user to explicitly pass the local_infile parameter if they want to use the LOAD DATA LOCAL command.
The other example uses MySQLdb, which is basically a wrapper around the MySQL C API (and to my knowledge not the official connection library; that would be MySQL Connector Python, which is also available on SQLAlchemy as mysqlconnector). This one apparently creates the connection in a way that the LOAD DATA LOCAL is enabled by default.

pyodbc always connects to master database

I am using pyodbc to connect to an azure sql database. My source code looks like this:
import pyodbc
server = 'sqlserver.database.windows.net'
database = 'database'
username = 'username'
password = 'password'
conn= pyodbc.connect('DRIVER={ODBC Driver 13 for SQL Server}'+';SERVER='+server+';PORT=1443;DATABASE='+database+';UID='+username+';PWD='+ password ';Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;')
cursor = conn.cursor()
cursor.execute("query")
I am able to connect to a sql database. The only thing which is not working properly is that pyodbc does not connect to the database I have specified in the database variable. It always connects to the master database.
What I have tried so far is to print the name of the databases on the target sql server using SELECT * FROM sys.databases while being connected to the master database. I was able to see the database I am trying to connect to. Anyone got an idea what goes wrong in my source code?

In general, I would assume that the connection string needs to be different in your case.
As per pyodbc docs:
[...]the most important thing to remember is that pyodbc does not even look at the connection string. It is passed directly to the database driver unmodified (through SQLDriverConnect). Connection strings are therefore driver-specific and all ODBC connection string documentation should be valid.
https://github.com/mkleehammer/pyodbc/wiki/Connecting-to-databases
However, since you are connecting to db OK and your connection string seems to be ignored, I would say that if you are using Windows then the connection parameters seem likely to be defined in the ODBC DSN, which can be changed in Control panel. If that is the case, and you have ODBC parameters defined in DSN, most likely your connection string is ignored, except for the choice of the DSN.

SQLAlchemy orm styles, how to make special drive to your connection string

I use pypy, pypyodbc and SQLAlchemy.
I have problem of odbc connections.
I use:
engine = create_engine('mssql+pyodbc://dbuser:dbpasswd#localhost/dbname', echo = False)
Session = sessionmaker(bind=engine)
style try to connect the database.
The error is:
C:\pypy\site-packages\sqlalchemy\connectors\pypyodbc.py:82: SAWarning: No driver
name specified; this is expected by PyODBC when using DSN-less connections
"No driver name specified; "
The reason of this error, I find the connect parameter
DRIVER={SQL Server Native Client}
is not transmit to the engine, in other word, I want to know how to set DRIVER string for this connections style for SQLALchemy.

I've run into the same issue with Sybase ASE and after looking at the pyodbc.py source code you can pass GET-like parameters in your url. As an example (working for me):
sybase+pyodbc://username:password#hostname:5000/dbname?driver=Adaptive Server Enterprise
It's also sort of documented here with the connection string syntax being dialect://user:password#host/dbname[?key=value..]
Hope that helps

You may have also be been suffering from updating your SqlAlchemy version. As of the latest release (v1.0) you need to explicitly define your driver in the connection string for Microsoft SQL Server.
See:
Changed in version 1.0.0: Hostname-based PyODBC connections now require the SQL Server driver name specified explicitly. SQLAlchemy cannot choose an optimal default here as it varies based on platform and installed drivers.
http://docs.sqlalchemy.org/en/latest/dialects/mssql.html#hostname-connections
See:
Connecting to database using SQLAlchemy

I had this error message and fixed it by adding this to the end of the connection string:
"?driver=SQL+Server+Native+Client+10.0"

DateTime import using Pandas/SQLAlchemy

I'm having problems importing datetimes from a SQL Server database into Pandas.
I'm using the following code:
data = pd.read_sql('select top 10 timestamp from mytable',db)
'MyTable' contains a column 'Timestamp', which is of type DateTime2.
If db is a pyodbc database connection this works fine, and my timestamps are returned as data type 'datetime64[ns]'. However if db an SQL Alchemy engine created using create_engine('mssql+pyodbc://...') then the timestamps returned in data are of type 'object' and cause problems later on in my code.
Any idea why this happens? I'm using pandas version 0.14.1, pyodbc version 3.0.7 and SQL alchemy version 0.9.4. How best can I force the data into datetime64[ns]?

Turns out the problem originates from how SQL Alchemy calls PyODBC. By default it will use the 'SQL Server' driver, which doesn't support DateTime2. When I was using PyODBC directly, I was using the 'SQL Server Native Client 10.0' driver.
To get the correct behaviour, i.e. return python datetime objects, I needed to create the SQL Alchemy engine as follows:
import sqlalchemy as sql
connectionString = 'mssql+pyodbc://username:password#my_server/my_database_name?driver=SQL Server Native Client 10.0'
engine = sql.create_engine(connectionString)
The ?driver=... part forces SQL Alchemy to use the right driver.