Is there a way to return the aliased column names from a sql query returned from JayDeBeApi?
For example, I have the following query:
sql = """ SELECT visitorid AS id_alias FROM table LIMIT 1 """
I then run the following (connect_to_vdm() establishes a connection to my DB):
curs = connect_to_vdm().cursor()
curs.execute(sql)
vals = curs.fetchall()
I normally retrieve column names like so:
desc = curs.description
column_names = [col[0] for col in desc]
This returns the original column name "visitorid" and not the alias specified in the query "id_alias".
I know I could swap the names for the value in Python, but hoping to be able to have this done within the query since it is already defined in the Select statement. This behaves as expected in a SQL client, but I cannot seem to get the Aliases to return when using python/JayDeBeApi. Is there a way to do this using JayDeBeApi?
EDIT:
I have discovered that structuring my query with a CTE seems to help fix the problem, but still wondering if there is a more straightforward solution out there. Here is how I rewrote the same query:
sql = """ WITH cte (id_alias) AS (SELECT visitorid AS id_alias FROM table LIMIT 1) SELECT id_alias from cte"""
I was able to fix this using a CTE (Common Table Expression)
sql = """ WITH cte (id_alias) AS (SELECT visitorid AS id_alias FROM table LIMIT 1) SELECT id_alias from cte"""
Hat tip to pybokeh on Github, but this worked for me.
According to IBM (here and here), the behavior of JDBC drivers changed at some point. Bizarrely, the column aliases display just fine when using a tool like DBVisualizer, but not by querying through jaydebeapi.
To fix, add the following to the end of your DB URL:
:useJDBC4ColumnNameAndLabelSemantics=false;
Example:
jdbc:db2://[DBSERVER]:[PORT]/[DBNAME]:useJDBC4ColumnNameAndLabelSemantics=false;
Related
Using Python looping through a number of SQL Server databases creating tables using select into, but when I run the script nothing happens i.e. no error messages and the tables have not been created. Below is an extract example of what I am doing. Can anyone advise?
df = [] # dataframe of database names as example
for i, x in df.iterrows():
SQL = """
Drop table if exists {x}..table
Select
Name
Into
{y}..table
From
MainDatabase..Details
""".format(x=x['Database'],y=x['Database'])
cursor.execute(SQL)
conn.commit()
Looks like your DB driver doesn't support multiple statements behavior, try to split your query to 2 single statements one with drop and other with select:
for i, x in df.iterrows():
drop_sql = """
Drop table if exists {x}..table
""".format(x=x['Database'])
select_sql = """
Select
Name
Into
{y}..table
From
MainDatabase..Details
""".format(x=x['Database'], y=x['Database'])
cursor.execute(drop_sql)
cursor.execute(select_sql)
cursor.commit()
And second tip, your x=x['Database'] and y=x['Database'] are the same, is this correct?
I'm trying to do something extremely simple that works, but not the way I expect it to. I have a database with various tables and for each of those tables, I'm trying to extract the column names from the information schema. I'm using the code below and everything works like a charm (python):
import psycopg2 as pgsql
# code to connect and generate cursor
table = 'some_table_name'
query = 'SELECT column_name FROM information_schema.columns WHERE table_name = %s'
cursor.execute(query, (table,))
result = pd.DataFrame(cursor.fetchall())
print(result)
So far, so good. The problem arises when I replace the query variable with the following:
import psycopg2 as pgsql
# code to connect and generate cursor
table = 'some_table_name'
**query = 'SELECT column_name FROM information_schema.columns WHERE table_name='+table
cursor.execute(query)**
result = pd.DataFrame(cursor.fetchall())
print(result)
If I print the statement, it's correct:
SELECT column_name FROM information_schema.columns WHERE table_name=some_table_name
However, when I run the query, I'm getting this error message:
UndefinedColumn: column "some_table_name" does not exist
LINE 1: ... FROM information_schema.columns WHERE table_name=some_tabl...
some_table_name is a table name as a parameter to the WHERE clause, not a column name. How is this even possible?
Thanks!
Your problem is that you haven't put some_table_name in quotes so it is treated as a column name, not a string literal. Why not stick with the first method which both worked and is in line with the psycopg documentation?
I'm trying to run SQL query with LIKE operator through Python to find any values that have "test" in any position. The problem seems to be with the formatting of what comes after the LIKE operator. There's no error messages, queries are just empty.
The SQL query that I'm trying to mimic is as follows, and works on when executed on Access.
SELECT Areas.ID, Areas.Name
FROM Areas
WHERE Name LIKE '*test*'
Here's how the connection and test data is made. No issue in there.
import pyodbc
# Connect to database
conn_str = (
r'DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};'
r'DBQ=C:\Temp\TestDB.accdb;'
r'Uid=;'
r'Pwd=;'
)
# Make cursor
connection = pyodbc.connect(conn_str)
connection.setencoding('utf-8')
cursor = connection.cursor()
# Create test table
cursor.execute("CREATE TABLE Areas (ID integer, Name varchar(255))")
connection.commit()
# Create test data
cursor.execute("INSERT INTO Areas (ID, Name) VALUES (1,'Example_1');")
cursor.execute("INSERT INTO Areas (ID, Name) VALUES (2,'Example_test_2');")
cursor.execute("INSERT INTO Areas (ID, Name) VALUES (3,'Example_3');")
connection.commit()
# Query filter
Filter = "'*test*'"
Attempt 01
query_01 = cursor.execute(r"""
SELECT Areas.ID, Areas.Name
FROM Areas
WHERE Name LIKE {Filter}
""".format(Filter=Filter)).fetchall()
for row in query_01:
print(row)
Attempt 02
query_02 = cursor.execute(r"""
SELECT Areas.ID, Areas.Name
FROM Areas
WHERE Name LIKE ?
""",("%{}%".format(filter),)).fetchall()
for row in query_02:
print(row)
Attempt 03, I would like the filter to be variable but even "hard coded" does not work.
query_03 = cursor.execute(r"""
SELECT Areas.ID, Areas.Name
FROM Areas
WHERE Name LIKE '*test*'
""").fetchall()
for row in query_03:
print(row)
To be sure something is working, I ran this and it prints the row.
query_04 = cursor.execute(r"""
SELECT Areas.ID, Areas.Name
FROM Areas
WHERE Name = 'Example_test_2'
""").fetchall()
for row in query_04:
print(row)
The ideal solution would be that the filter variable could be just a string, without the wildcards. How should I format the filter variable and the query?
For historical reasons, LIKE queries run from within the Access UI default to using * and ? as the wildcard characters. However, external applications using ODBC to query an Access database must use the more common % and _ wildcard characters.
Also, the parameter value must contain the wildcard character(s). (A LIKE condition without wildcard characters is just the same as an = condition.) The parameter placeholder in the SQL command text must be a bare question mark ?.
Finally, do not use connection.setencoding('utf-8'). Access stores text values as Unicode, but it does not use UTF-8 encoding. The default pyodbc encoding (UTF-16) works just fine.
So what you're looking for is
filter = 'test'
sql = "SELECT Areas.ID, Areas.Name FROM Areas WHERE Areas.Name LIKE ?"
param = f'%{filter}%'
rows = cursor.execute(sql, param).fetchall()
I'd like to retrieve the fully referenced column name from a PyOdbc Cursor. For example, say I have 2 simple tables:
Table_1(Id, < some other fields >)
Table_2(Id, < some other fields >)
and I want to retrieve the joined data
select * from Table_1 t1, Table2 t2 where t1.Id = t2.Id
using pyodbc, like this:
query = 'select * from Table_1 t1, Table2 t2 where t1.Id = t2.Id'
import pyodbc
conn_string = '<removed>'
connection = pyodbc.connect(conn_string)
cursor = connection.cursor()cursor.execute(query)
I then want to get the column names:
for row in cursor.description:
print row[0]
BUT if I do this I'll get Id twice which I don't want. Ideally I could get t1.Id and t2.Id in the output.
Some of the solutions I've thought of (and why I don't really want to implement them):
re-name the columns in the query - in my real-world use case there are dozens of tables, some with dozens of rows that are changed far too often
parse my query and automate my SQL query generation (basically checking the query for tables, using the cursor.tables function to get the columns and then replacing the select * with a set of named columns) - If I have too I'll do this, but it seems like overkill for a testing harness
Is there a better way? Any advice would be appreciated.
The PyOdbc docs offer
# columns in table x
for row in cursor.columns(table='x'):
print(row.column_name)
www.PyOdbc wiki The API docs are useful
Here's how I do it.
import pyodbc
connection = pyodbc.connect('DSN=vertica_standby', UID='my_user', PWD='my_password', ansi=True)
cursor = connection.cursor()
for row in cursor.columns(table='table_name_in_your_database'):
print(row.column_name)
You have to have your DSN (data source name) set up via two files. odbc.ini and odbcinst.ini
It doesn't seem to be possible to do what I want without writing a decent amount of code to wrap it up. None of the other answers actually answered the question of returning different column names by the table they originate from in some relatively automatic fashion.
So My problem is this, I have a query that uses Mysql User-defined variable like:
#x:=0 SELECT #X:=#X+1 from some_table and this code returns a column from 1-1000.
However, this query doesn't work if I sent it through mySQLdb in Python.
connection =MySQLdb.Connect(host='xxx',user='xxx',passwd='xxx',db = 'xxx')
cursor = connection.cursor
cursor.execute("""SET #X:=0;SELECT #X:=#X+1 FROM some_table""")
rows = cursor.fetchall()
print rows
It prints a empty tuple.
How can I solve this?
Thanks
Try to execute one query at a time:
cursor.execute("SET #X:=0;");
cursor.execute("SELECT #X:=#X+1 FROM some_table");
Try it as two queries.
If you want it to be one query, the examples in the comments to the MySQL User Variables documentation look like this:
SELECT #rownum:=#rownum+1 rownum, t.* FROM (SELECT #rownum:=1) r, mytable t;
or
SELECT if(#a, #a:=#a+1, #a:=1) as rownum
See http://dev.mysql.com/doc/refman/5.1/en/user-variables.html