I'm trying to do something extremely simple that works, but not the way I expect it to. I have a database with various tables and for each of those tables, I'm trying to extract the column names from the information schema. I'm using the code below and everything works like a charm (python):
import psycopg2 as pgsql
# code to connect and generate cursor
table = 'some_table_name'
query = 'SELECT column_name FROM information_schema.columns WHERE table_name = %s'
cursor.execute(query, (table,))
result = pd.DataFrame(cursor.fetchall())
print(result)
So far, so good. The problem arises when I replace the query variable with the following:
import psycopg2 as pgsql
# code to connect and generate cursor
table = 'some_table_name'
**query = 'SELECT column_name FROM information_schema.columns WHERE table_name='+table
cursor.execute(query)**
result = pd.DataFrame(cursor.fetchall())
print(result)
If I print the statement, it's correct:
SELECT column_name FROM information_schema.columns WHERE table_name=some_table_name
However, when I run the query, I'm getting this error message:
UndefinedColumn: column "some_table_name" does not exist
LINE 1: ... FROM information_schema.columns WHERE table_name=some_tabl...
some_table_name is a table name as a parameter to the WHERE clause, not a column name. How is this even possible?
Thanks!
Your problem is that you haven't put some_table_name in quotes so it is treated as a column name, not a string literal. Why not stick with the first method which both worked and is in line with the psycopg documentation?
Related
I'm trying to transfer a user input from a python code to a table in postgresql
What I want to do is place an input() in this code and make it's value go to the comment (#) in the code.
conn = psycopg2.connect(
host="localhost",
database="Twitterzuil",
user="postgres",
password="")
cur = conn.cursor()
cur.execute("INSERT INTO Bericht2 (name) VALUES (#THIS IS WHERE I WANT THE INPUT TO GO)");
conn.commit()
I have no idea how, I'm really a beginner in all this so any help is appreciated
I believe what you are asking about is called string interpolation. Using f-style format, this might look like
new_name = "'bob'" # need single quotes for SQL strings
sql = f"INSERT INTO Bericht2 (name) VALUES ({new_name})" # => sql == "INSERT INTO Bericht2 (name) VALUES ('bob')"
cur.execute(sql)
Note the f at the start of the string, when you do this expressions inside {} pairs get replaced with their python values (tutorial). There are also string formatting approaches involving % substitution and the .format method on strings.
If you are doing anything beyond the basics you should look into using the SQLAlchemy package; here's the link to their insert api. Using SQLAlchemy will help reduce the risks that can come with manually constructing SQL queries.
Example from "Inserting Rows with SQLAlchemy"
from sqlalchemy import insert
stmt = insert(user_table).values(name='spongebob', fullname="Spongebob Squarepants")
with engine.connect() as conn:
result = conn.execute(stmt)
conn.commit()
I'm trying to run SQL query with LIKE operator through Python to find any values that have "test" in any position. The problem seems to be with the formatting of what comes after the LIKE operator. There's no error messages, queries are just empty.
The SQL query that I'm trying to mimic is as follows, and works on when executed on Access.
SELECT Areas.ID, Areas.Name
FROM Areas
WHERE Name LIKE '*test*'
Here's how the connection and test data is made. No issue in there.
import pyodbc
# Connect to database
conn_str = (
r'DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};'
r'DBQ=C:\Temp\TestDB.accdb;'
r'Uid=;'
r'Pwd=;'
)
# Make cursor
connection = pyodbc.connect(conn_str)
connection.setencoding('utf-8')
cursor = connection.cursor()
# Create test table
cursor.execute("CREATE TABLE Areas (ID integer, Name varchar(255))")
connection.commit()
# Create test data
cursor.execute("INSERT INTO Areas (ID, Name) VALUES (1,'Example_1');")
cursor.execute("INSERT INTO Areas (ID, Name) VALUES (2,'Example_test_2');")
cursor.execute("INSERT INTO Areas (ID, Name) VALUES (3,'Example_3');")
connection.commit()
# Query filter
Filter = "'*test*'"
Attempt 01
query_01 = cursor.execute(r"""
SELECT Areas.ID, Areas.Name
FROM Areas
WHERE Name LIKE {Filter}
""".format(Filter=Filter)).fetchall()
for row in query_01:
print(row)
Attempt 02
query_02 = cursor.execute(r"""
SELECT Areas.ID, Areas.Name
FROM Areas
WHERE Name LIKE ?
""",("%{}%".format(filter),)).fetchall()
for row in query_02:
print(row)
Attempt 03, I would like the filter to be variable but even "hard coded" does not work.
query_03 = cursor.execute(r"""
SELECT Areas.ID, Areas.Name
FROM Areas
WHERE Name LIKE '*test*'
""").fetchall()
for row in query_03:
print(row)
To be sure something is working, I ran this and it prints the row.
query_04 = cursor.execute(r"""
SELECT Areas.ID, Areas.Name
FROM Areas
WHERE Name = 'Example_test_2'
""").fetchall()
for row in query_04:
print(row)
The ideal solution would be that the filter variable could be just a string, without the wildcards. How should I format the filter variable and the query?
For historical reasons, LIKE queries run from within the Access UI default to using * and ? as the wildcard characters. However, external applications using ODBC to query an Access database must use the more common % and _ wildcard characters.
Also, the parameter value must contain the wildcard character(s). (A LIKE condition without wildcard characters is just the same as an = condition.) The parameter placeholder in the SQL command text must be a bare question mark ?.
Finally, do not use connection.setencoding('utf-8'). Access stores text values as Unicode, but it does not use UTF-8 encoding. The default pyodbc encoding (UTF-16) works just fine.
So what you're looking for is
filter = 'test'
sql = "SELECT Areas.ID, Areas.Name FROM Areas WHERE Areas.Name LIKE ?"
param = f'%{filter}%'
rows = cursor.execute(sql, param).fetchall()
I want to check whether the table exists or not before inserting the data.
This is what i have tried:
def checkTables(tablename):
stmt = "SHOW TABLES LIKE %s"%tablename
cursor.execute(stmt)
result = cursor.fetchone()
return result
But it gives me error saying:
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'ctg_payload3' at line 1
Maybe it is not the best way.
As for my opinion, I will show tables;, then search the result.
And, I you cannot execute show tables like tablename;, no syntax like that.
edit 1
If you must do it in sql, use
show table status like 'table_name';
' is needed for this sql.
Try this query string :SHOW TABLES WHERE Tables_in_mydb LIKE '%tablename%' or
this one
SELECT table_name
FROM information_schema.tables
WHERE table_schema = 'my_database_name'
AND table_name LIKE '%tablename%'
Good luck
Try this.
def checkTables(tablename):
stmt = "SHOW TABLES LIKE '%s' "% ('%'+str(tablename)+'%')
cursor.execute(stmt)
result = cursor.fetchone()
return result
I would like to check if a database table exists or not, but I don't know how to do.
I wrote (for example with SQLite, although I use MySQL mainly),
import sqlite3
table_name = "some_table"
connection = sqlite3.connect(db)
cursor = connection.cursor()
table_check = "SELECT name FROM sqlite_master WHERE type='table' AND name={};".format(table_name)
if not cursor.execute(table_check).fetchone(): # if the table doesn't exist
# OR if cursor.execute(table_check).fetchone() == "":
create_table()
else:
update_table()
But, an Error occured and I cannot proceed.
sqlite3.OperationalError: no such column: some_table
I read several Q&A here, but I couldn't get those.
Any advice can help me.
Thank you.
Python 3.5.1
The answer is depending on what rdbms product (mysql, sqlite, ms sql, etc.) you use.
You are getting this particular error in your above query because you do not enclose the value of table_name variable in single quotes.
In mysql you can use information_schema.tables table to query if a table exists.
So My problem is this, I have a query that uses Mysql User-defined variable like:
#x:=0 SELECT #X:=#X+1 from some_table and this code returns a column from 1-1000.
However, this query doesn't work if I sent it through mySQLdb in Python.
connection =MySQLdb.Connect(host='xxx',user='xxx',passwd='xxx',db = 'xxx')
cursor = connection.cursor
cursor.execute("""SET #X:=0;SELECT #X:=#X+1 FROM some_table""")
rows = cursor.fetchall()
print rows
It prints a empty tuple.
How can I solve this?
Thanks
Try to execute one query at a time:
cursor.execute("SET #X:=0;");
cursor.execute("SELECT #X:=#X+1 FROM some_table");
Try it as two queries.
If you want it to be one query, the examples in the comments to the MySQL User Variables documentation look like this:
SELECT #rownum:=#rownum+1 rownum, t.* FROM (SELECT #rownum:=1) r, mytable t;
or
SELECT if(#a, #a:=#a+1, #a:=1) as rownum
See http://dev.mysql.com/doc/refman/5.1/en/user-variables.html