Teradata-sqlalchemy not using database when given - python

I'm trying to connect to our internal Teradata database, using flask and sqlAlchemy along with a custom engine called sqlalchemy teradata. I put the database into the create_engine function likes so.
engine = sqlalchemy.create_engine('teradata://username:pw#server_name/database')
I've setup my dialect just like in the tests
registry.register("tdalchemy", "sqlalchemy_teradata.dialect", "TeradataDialect")
I'm getting a.
DatabaseError: (teradata.api.DatabaseError) (3807, u"[42S02] [Teradata][ODBC Teradata Driver][Teradata Database] Object 'table_name' does not exist
I can make raw sql queries just fine, I can also have alchemy do a query I construct and it pulls the data. I'm not sure what all is preventing things from working properly at all. When I test a similar call but looking at a database in an psql server it works just fine and pulls from that db without issue.
Also the pypi page says there is supposed to be an test/orm_test.py but it doesn't seem to have it.

Related

Invalid object name gets returned for certain databases in MS SQL Server Management Studio

I can connect to databases in MS SQL Server Management Studio using my python script without issues (using pyodbc).
I then created a database called tempdb - see the db explorer pic referred to below. I did this by running a direct query in MS SQL Management Studio, and created a table (DepartmentTest)
Now, in my script if I do:
cursor.execute("SELECT * FROM DepartmentTest")
I get:
[Microsoft][ODBC SQL Server Driver][SQL Server]Invalid object name
Also tried: a few options for the above query such as:
dbo.DepartmentTest
[dbo].DepartmentTest
(instead of just DepartmentTest as above.)
I don't have this issue when connecting to the master database and accessing the tables in the master database
e.g. I can execute:
cursor.execute("SELECT * FROM MSreplication_options")
and I get back the contents. I.e. anything under System Tables works fine with the script.
In the explorer pic referred to below: I can access the tables circled in green. I can't access my table, circled in red.
I assume I am not correctly pointing to my table with the syntax I am using, but I'm not sure how to modify my query. (it's as though anything under System Tables is fine to access with my code.
(I did connect to the correct database name with my code)
Thanks and
regards
You should double check you’re connected to the right database, which is suppose to be ‘tempdb’. If you do that and try running the query again, it should work.
It seems as though the trusted connection to the server was the problem. Once I did a connection with user and password credentials, I was able to access that database.

Is SQLAlchemy a database itself?

I am new to python but I have worked with ORM frameworks before.
It is confusing for me that when I create database connection I can specify a database URL to some remote DB.
Or I can create DB in a file without remote DB. How does SQLAlchemy work with a file then?
It doesn`t have an RDBMS to connect to. Only file. So this is the reason for my question:
Is SQLAlchemy an RDBMS itself?
It's not a database, but a library for handling databases, or as they put it:
SQLAlchemy is the Python SQL toolkit and Object Relational Mapper that gives application developers the full power and flexibility of SQL.
The file based dialect is SQLite.

why should we set the local_infile=1 in sqlalchemy to load local file? Load file not allowed issue in sqlalchemy

I am using sqlalchemy to connect to MySQL database and found a strange behavior.
If I query
LOAD DATA LOCAL INFILE
'C:\\\\Temp\\\\JaydenW\\\\iata_processing\\\\icer\\\\rename\\\\ICER_2017-10-
12T09033
7Z023870.csv
It pops an error:
sqlalchemy.exc.InternalError: (pymysql.err.InternalError) (1148, u'The used
command is not allowed with this MySQL versi
on') [SQL: u"LOAD DATA LOCAL INFILE
'C:\\\\Temp\\\\JaydenW\\\\iata_processing\\\\icer\\\\rename\\\\ICER_2017-10-
12T090337Z023870.csv' INTO TABLE genie_etl.iata_icer_etl LINES TERMINATED BY
'\\n'
IGNORE 1 Lines (rtxt);"] (Background on this error at:
http://sqlalche.me/e/2j85)
And I find the reason is that:
I need to set the parameter as
args = "mysql+pymysql://"+username+":"+password+"#"+hostname+"/"+database+"?
local_infile=1"
If I use MySQL official connection library. I do not need to do so.
myConnection = MySQLdb.connect(host=hostname, user=username, passwd=password, db=database)
Can anyone help me to understand the difference between the two mechanisms?
The reason is that the mechanisms use different drivers.
In SQLAlchemy you appear to be using the pymysql engine, which uses the PyMySQL Connection class to create the DB connection. That one requires the user to explicitly pass the local_infile parameter if they want to use the LOAD DATA LOCAL command.
The other example uses MySQLdb, which is basically a wrapper around the MySQL C API (and to my knowledge not the official connection library; that would be MySQL Connector Python, which is also available on SQLAlchemy as mysqlconnector). This one apparently creates the connection in a way that the LOAD DATA LOCAL is enabled by default.

Using a base dialect with pyodbc in SQLAlchemy

I can connect via pyODBC to an unsupported database over ODBC. Queries appear to execute correctly. If I try to connect using mssql+pyodbc, I can't connect properly (image not found).
I've tried "base:///", "base+pyodbc:///", or "pyodbc:///".
Do I need to write my own "dialect" that doesn't make any changes to base, and are there any useful (up to date) guides on how to do this?
EDIT:
import pyodbc
conn = pyodbc.connect(DSN = "ODBCCONNECTIONNAME", UID = "ODBCUSER", PWD="PASSWORD")
cursor = conn.cursor()
Also works with this line replacing the connection above:
conn = pyodbc.connect("DSN=fmp_production;UID=odbc_user;PWD=Pwd222")
I can then run selects, and modifications fine, using standard SQL.
EDIT:
Okay, so I'm getting an error:
"Abort trap: 6" basically my python process is crashing out. I've tried testing with a functioning ODBC connection to a MySQL database, using:
engine = create_engine('mysql+pyodbc://root:rootpwd#testenvironment')
If I include the name of the database, then I get an image not found error instead. But I think the actual problem is whatever is crashing my python.

Using two databases with Pyramid SqlAlchemy

I have a SQLite database configured with Pyramid Alchemy scaffold in my Linux machine. I have another remote Mysql database which resides in a Windows machine, with loads of data.Now, I have to connect to the remote Mysql database to pull data and populate them into my Sqlite database with the help of sqlAlchemy.
What I had been doing: I used mysql workbench to query data from the mysql database, export results to a csv file, load the csv file into my pyramid initializedb.py through python's default csv module, and then finally insert the retrieved rows into my Sqlite database.
What I want to do: I want to connect to the remote Mysql database from my initializedb.py itself, fetch results and insert them into my sqlite database.
How do I go on about with this? Any help is appreciated.
You can create a connection into your mysql db within the initializedb.py, extract the data and store them into your local db. Now, you will find the basics about sqlalchemy here:
http://docs.sqlalchemy.org/en/rel_0_9/core/tutorial.html
If you need this configurable, then you can put the url of your database into your config and access it through settings.
Also, you do not have to define the structure of your tables exactly, you can use something like:
users = Table('users', metadata, autoload=True)
On the other hand, if your MySQL db structure is the same as the sqlite db structure, it might be possible to reuse your model, but that would probably be hard to explain here without seeing any of your code.

Categories