Because to deal with static sql into dynamic sql, so need to use python conversion format, for example
DROP TABLE TABLE_NAME PURE;
CREATE TABLE TABLE_NAME NOLOGGING AS
SELECT DUMMY USER_ID FROM DUAL;
CREATE INDEX I_TABLE_NAME_USER_ID ON TABLE_NAME(USER_ID) NOLOGGING;
I want to convert the format to:
First determine whether there is a “drop table” and then remove the data, replaced
V_TAB_NAME: = 'TABLE_NAME';
IF (F_DROP_TAB (V_TAB_NAME) = 1) THEN
V_SQL: = '
The final result is shown as
V_TAB_NAME := 'TABLE_NAME ';
IF (F_DROP_TAB(V_TAB_NAME) = 1) THEN
SJ_SQL := '
CREATE TABLE TABLE_NAME NOLOGGING AS
SELECT DUMMY USER_ID FROM DUAL';
EXECUTE IMMEDIATE SJ_SQL;
COMMIT;
END IF;
SJ_SQL := '
CREATE INDEX I_TABLE_NAME_USER_ID ON TABLE_NAME (USER_ID) NOLOGGING';
EXECUTE IMMEDIATE SJ_SQL;
It may be a bit difficult to understand, but I hope someone can give me a little string to iterate over and judge the modified example or idea. Thanks.
Here's a simple way to insert variables into SQL statements:
table = 'mytable'
col1 = 'name'
col2 = 'age'
statement = 'select {}, {} from {};'.format(col1, col2, table)
Is this what you're looking for? If not, i don't really understand the question.
Related
Im having trouble with a postgresql query using SQLAlchemy.
I created some large tables using this line of code:
frame.to_sql('Table1', con=engine, method='multi', if_exists='append')
It worked fine. Now, when I want to query data out of it, my first problem is that I have to use quotation marks for each table and column name and I dont really know why, maybe somebody can help me out there.
That is not my main problem though. My main problem is, that when querying the data, all numerical WHERE conditions work fine, but not the ones with Strings in the column data. I get an error that the column does not exist. Im using:
df = pd.read_sql_query('SELECT "variable1", "variable2" FROM "Table1" WHERE "variable1" = 123 AND "variable2" = "abc" ', engine)
I think it might be a problem that I use "abc" instead of 'abc', but I cant change it because of the ' signs in the argument of the query. If I change those ' to " then the Column names and Table names are not detected correctly (because of the problem before that they have to be in quotation marks).
This is the error message:
ProgrammingError: (psycopg2.errors.UndefinedColumn) ERROR: COLUMN »abc« does not exist
LINE 1: ...er" FROM "Table1" WHERE "variable2" = "abc"
And there is an arrow pointing to the first quotation mark of the "abc".
Im new to SQL and I would really appreciate if someone could point me in the right direction.
"Most" SQL dialects (notable exceptions being MS SQL Server and MS Access) strictly differentiate between
single quotes: for string literals, e.g., WHERE thing = 'foo'
double quotes: for object (table, column) names, e.g., WHERE "some col" = 123
PostgreSQL throws in the added wrinkle that table/column names are forced to lower case if they are not (double-)quoted and then uses case-sensitive matching, so if your table is named Table1 then
SELECT * FROM Table1 will fail because PostgreSQL will look for table1, but
SELECT * FROM "Table1" will succeed.
The way to avoid confusion in your query is to use query parameters instead of string literals:
# set up test environment
with engine.begin() as conn:
conn.exec_driver_sql('DROP TABLE IF EXISTS "Table1"')
conn.exec_driver_sql('CREATE TABLE "Table1" (variable1 int, variable2 varchar(50))')
df1 = pd.DataFrame([(123, "abc"), (456, "def")], columns=["variable1", "variable2"])
df1.to_sql("Table1", engine, index=False, if_exists="append")
# test .read_sql_query() with parameters
import sqlalchemy as sa
sql = sa.text('SELECT * FROM "Table1" WHERE variable1 = :v1 AND variable2 = :v2')
param_dict = {"v1": 123, "v2": "abc"}
df2 = pd.read_sql_query(sql, engine, params=param_dict)
print(df2)
"""
variable1 variable2
0 123 abc
"""
It should be: AND "variable2" = 'abc'.
You cannot quote strings/literals with ", as PostgreSQL will interpret it as a database object. Btw. you do not need to wrap table names and and columns with double quotes unless it is extremely necessary, e.g. case sensitive object names, names containing spaces, etc. Imho it is a bad practice and on the long run only leads to confusion. So your query could be perfectly written as follows:
SELECT variable1, variable2
FROM table1
WHERE variable1 = 123 AND variable2 = 'abc';
Keep in mind that it also applies for other objects, like tables or indexes.
CREATE TABLE Table1 (id int) - nice.
CREATE TABLE "Table1" (id int) - not nice.
CREATE TABLE "Table1" ("id" int) - definitely not nice ;)
In case you want to remove the unnecessary double quotes from your table name:
ALTER TABLE "Table1" RENAME TO table1;
Demo: db<>fiddle
I'm trying to make a query to a SQLite database from a python script. However, whenever I use parameterization it just returns the first parameter, which is column2. The desired result is for it to return the value held in column2 on the row where column1 is equal to row1.
conn = sqlite3.connect('path/to/database')
c = conn.cursor()
c.execute('SELECT ? from table WHERE column1 = ? ;', ("column2","row1"))
result = c.fetchone()[0]
print(result)
It prints
>>column2
Whenever I run this using concatenated strings, it works fine.
conn = sqlite3.connect('path/to/database')
c = conn.cursor()
c.execute('SELECT ' + column2 + ' from table WHERE column1 = ' + row1 + ';')
result = c.fetchone()[0]
print(result)
And it prints:
>>desired data
Any idea why this is happening?
This behaves as designed.
The mechanism that parameterized queries provide is meant to pass literal values to the query, not meta information such as column names.
One thing to keep in mind is that the database must be able to parse the parameterized query string without having the parameter at hand: obviously, a column name cannot be used as parameter under such assumption.
For your use case, the only possible solution is to concatenate the column name into the query string, as shown in your second example. If the parameter comes from outside your code, be sure to properly validate it before that (for example, by checking it against a fixed list of values).
My issue came when I decided to make a method that could handle a variation of queries, instead of coding 3 methods. I wanted to do so as to recycle code.
I have this table:
(I created it in the purpose of this question. You can do it by:
create table example (id int(1), ex1 varchar(15), ex2 varchar(15), ex3 varchar(15));
insert into example values(1, 'whatever11', 'whatever12', 'whatever13');
insert into example values(2, 'whatever21', 'whatever22', 'whatever23');
insert into example values(3, 'whatever31', 'whatever32', 'whatever33');
SO: I was trying to parameterize column names. I have done it in the whereclause all the time, but as I mention earlier, I thought it would be cleaner and more optimal to do just one method ( select %s from example where id=%s), instead of 3 different: (select ex1 from etc, select ex2 from etc.
So I tried this:
So the normal method is this:
def getex1(id):
myCursor=mydb.cursor()
query='select ex1 from example where id=%s'
myCursor.execute(query, (id,))
result=myCursor.fetchone()[0]
print(result) #prints 'whatever11 if id=1'
When I searched how to do parameterized queries, I saw that to do various parameters, you can just do something like input=(param1, param2, and then execute by (query, input), so I tried to do so but with the column name:
here, info is 'ex1', 'ex2' or 'ex3':
def getAllFromExample(info, id):
myCursor = mydb.cursor()
query= 'select %s from example where id=%s'
input = (info, id)
myCursor.execute(query, input)
result = myCursor.fetchone()[0]
print(result) #in this case, prints the column names 'ex1', 'ex2', not the content
My guess is that you can't just do the param by columns, because you are not assigning a value (like in a whereor in a group by, you have an assignment: whatever=value).
Any insights on this? I did quite the research but did not find anything. here it is mentioned this.
Anything you see wrong with the question, ask me and I'll make it clearer!
You cannot parametrizied the table names, you only can do it with the column values, so you would have to do:
def getAllFromExample(info, DNI):
myCursor = mydb.cursor()
query= 'select '+info+' from example where id=%s'
input = (id,)
myCursor.execute(query, input)
result = myCursor.fetchone()[0]
print(result) #in this case, prints the column name
i have a written the below function to filter a column in a sql query, the function takes a string argument which will be inputted in the 'where clause'
def summaryTable(machineid):
df=pd.read_sql(""" SELECT fld_ATM FROM [003_tbl_ATM_Tables]
WHERE (LINK <> 1) AND (fld_ATM =('machineid')) ;
""",connection)
connection.close()
return df
the function returns an empty Dataframe. i know the query itself is correct 'cause i get the expected data when i 'hardcode' the machine id
Use params to pass a tuple of parameters including machineid to read_sql. pyodbc replaces the ? character in your query with parameters from the tuple, in order. Their values will be safely substituted at runtime. This avoids dangerous string formatting issues which may result in SQL injection.
df = pd.read_sql(""" SELECT fld_ATM FROM [003_tbl_ATM_Tables]
WHERE (LINK <> 1) AND (fld_ATM = ?) ;
""", connection, params=(machineid,))
You need to add machineid to query using params.
# ? is the placeholder style used by pyodbc. Some use %s, for example.
query = """ SELECT fld_ATM FROM [003_tbl_ATM_Tables]
WHERE (LINK <> 1) AND (fld_ATM = ?) ;
"""
data_df = pd.read_sql_query(query, engine, params=(machineid, ))
I have a list of tuples of which i'm inserting into a Table.
Each tuple has 50 values. How do i insert without having to specify the column names and how many ? there is?
col1 is an auto increment column so my insert stmt starts in col2 and ends in col51.
current code:
l = [(1,2,3,.....),(2,4,6,.....),(4,6,7,.....)...]
for tup in l:
cur.execute(
"""insert into TABLENAME(col2,col3,col4.........col50,col51)) VALUES(?,?,?,.............)
""")
want:
insert into TABLENAME(col*) VALUES(*)
MySQL's syntax for INSERT is documented here: http://dev.mysql.com/doc/refman/5.7/en/insert.html
There is no wildcard syntax like you show. The closest thing is to omit the column names:
INSERT INTO MyTable VALUES (...);
But I don't recommend doing that. It works only if you are certain you're going to specify a value for every column in the table (even the auto-increment column), and your values are guaranteed to be in the same order as the columns of the table.
You should learn to use code to build the SQL query based on arrays of values in your application. Here's a Python example the way I do it. Suppose you have a dict of column: value pairs called data_values.
placeholders = ['%s'] * len(data_values)
sql_template = """
INSERT INTO MyTable ({columns}) VALUES ({placeholders})
"""
sql = sql_template.format(
columns=','.join(keys(data_values)),
placeholders=','.join(placeholders)
)
cur = db.cursor()
cur.execute(sql, data_values)
example code to put before your code:
cols = "("
for x in xrange(2, 52):
cols = cols + "col" + str(x) + ","
test = test[:-1]+")"
Inside your loop
for tup in l:
cur.execute(
"""insert into TABLENAME " + cols " VALUES {0}".format(tup)
""")
This is off the top of my head with no error checking