I am new to Sqlite. In the following line of code c is a cursor:
c.execute(
"SELECT rowid,* from statements "
"WHERE [row1] = (?*) AND [row2] = (?) AND [row3] = 'text3'",
(text1,text2,)
)
items = c.fetchall()
In the above I was trying to find rows where the text in row1 is anything that begins with text1, e.g. I would like to select rows where
Row 1 is "the cow", "the horse" or thermometer" where text1="the"
and
Row 2 is "elephant" where text2="elephant"
and
Row 3 is "text3"
I was using the (?) operator because this is part of a function that will be called with different parameters.
You must use the operator LIKE instead of = for the 1st condition and for this to work, you must concatenate the placeholder ? with the wildcard '%':
c.execute("SELECT rowid,* from statements WHERE [row1] LIKE ? || '%' AND [row2] = ? AND [row3] = 'text3'", (text1,text2,))
Related
I would like to use a string as column names for pandas DataFrame.
The problem arised is that pandas DataFrame interpret the string var as single column instead of multiple ones. An thus the error:
ValueError: 1 columns passed, passed data had 11 columns
The first part of my code is intended to get the column names from the Mysql database I am about to query:
cursor1.execute ("SELECT GROUP_CONCAT(COLUMN_NAME) AS cols FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = 'or_red' AND TABLE_NAME = 'nomen_prefix'")
for colsTableMysql in cursor1.fetchall() :
colsTable = colsTableMysql[0]
colsTable="'"+colsTable.replace(",", "','")+"'"
The second part uses the created variable "colsTable" :
cursor = connection.cursor()
cursor.execute("SELECT * FROM or_red.nomen_prefix WHERE C_emp IN ("+emplazamientos+")")
tabla = pd.DataFrame(cursor.fetchall(),columns=[colsTable])
#tabla = exec("pd.DataFrame(cursor.fetchall(),columns=["+colsTable+"])")
#tabla = pd.DataFrame(cursor.fetchall())
I have tried ather aproaches like the use of exec(). In that case, there is no error but there is no response with information either, and the result of print(tabla) is None.
¿Is there any direct way of passing the columns dynamically as string to a python pandas DataFrame?
Thanks in advance
I am going to answer my question since I've already found the way.
The first part of my code is intended to get the column names from the Mysql database table I am about to query:
cursor1.execute ("SELECT GROUP_CONCAT(COLUMN_NAME) AS cols FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = 'or_red' AND TABLE_NAME = 'nomen_prefix'")
for colsTableMysql in cursor1.fetchall() :
colsTable = colsTableMysql[0]
colsTable="'"+colsTable.replace(",", "','")+"'"
The second part uses the created variable "colsTable" as input in the statement to define the columns.
cursor = connection.cursor()
cursor.execute("SELECT * FROM or_red.nomen_prefix WHERE C_emp IN ("+emplazamientos+")")
tabla = eval("pd.DataFrame(cursor.fetchall(),columns=["+colsTable+"])")
Using eval the string is parsed and evaluated as a Python expression.
I have as an input a string that is a SQL query. I need to get all tables that the query uses (like FROM table or table1 INNER JOIN table2). But the query does not respect any standard. So my question is if there is any method to format the query so that searching for these table names is easier.
My method right now is to search for the keywords from and join and take whatever line is after the keyword (or before in the case of the join), but there are exceptions in the queries where the from does not have a newline after it and I have to treat every exception like this. I don't think regex works because while the table name is {schema_name.table_name} there are also columns like that.
for row in text:
to_append = None
split_row = row.strip('\r').strip(' ').strip('\r').split(' ')
if split_row[-1].lower() == "from" and len(split_row) > 1:
from_indexes.append(text.index(row))
if ("join" in split_row or "JOIN" in split_row) and (split_row[-1] != "join" and split_row[-1]
!= "JOIN"):
for ind in range(len(split_row)):
if split_row[ind].lower() == "join":
to_append = split_row[ind + 1:]
row = split_row[:ind + 1]
row = ' '.join(row)
rows.append(row.strip('\r').strip(' ').strip('\t'))
if to_append is not None:
rows.append(' '.join(to_append))
So I am looking for some method that can standardize the sql query or for another method to extract the table names from the query.
I think a more straightforward approach would be to use regular expressions:
import re
sql = """select t1.*, t2.y, sq.z, table3.q from table1 t1 join
table2 t2 on t1.x = t2.x left join
(select 5 as x, 9 as z) sq JOIN
table3 on sq.x = table3.x
;"""
matches = re.findall(r'(\s+(from|join)\s+)(\w+)', sql, re.DOTALL|re.IGNORECASE)
for match in matches:
print(match[2])
Note that it will not consider (select 5 as x, 9 as z) as a table.
You should use an ORM tool in order to make cleaner queries (see https://en.wikipedia.org/wiki/Object-relational_mapping). Or at least some query builder modules.
I recently found a remake of laravels "eloquent" orm here https://pypi.org/project/eloquent/.
Other ORMs like PeeWee are pretty common to use, too.
I'm trying to create a SELECT statement that selects rows where NAME is max. 5 characters and the . is in the NAME.
I only want the first, so I'm including a LIMIT 1 to the statement.
I have worked with the following
searchstring = "."
sql = "SELECT * FROM Table WHERE NAME LIKE %s LIMIT 1"
val = (("%"+searchstring+"%"),)
cursor.execute(sql, val)
But I'm not sure how to incorporate the length of NAME in my statement.
My "Table" is as follows:
ID NAME
1 Jim
2 J.
3 Jonathan
4 Jack M.
5 M.S.
So based on the table above, I would expect row 2 and 5 to be selected.
I could select all, and loop through them. But as I only want the first, I'm thinking I would prefer a SQL statement?
Thanks in advance.
You can use CHAR_LENGTH function along with LIKE:
SELECT * FROM Table WHERE name LIKE '%.%' AND CHAR_LENGTH(name) <= 5 LIMIT 1
Try LEN()
Select LEN(result string);
This will return the length of string. but this will count spaces also. Try removing it with LTRIM().
Oracle SQL
SELECT * FROM Table WHERE name LIKE '%.%' AND LENGTH(name) < 6 and rownum < 2
Base on the sql language(oracle, mysql, sql server, etc) use
length() or char_length()
rownum or limit
I have a list of tuples of which i'm inserting into a Table.
Each tuple has 50 values. How do i insert without having to specify the column names and how many ? there is?
col1 is an auto increment column so my insert stmt starts in col2 and ends in col51.
current code:
l = [(1,2,3,.....),(2,4,6,.....),(4,6,7,.....)...]
for tup in l:
cur.execute(
"""insert into TABLENAME(col2,col3,col4.........col50,col51)) VALUES(?,?,?,.............)
""")
want:
insert into TABLENAME(col*) VALUES(*)
MySQL's syntax for INSERT is documented here: http://dev.mysql.com/doc/refman/5.7/en/insert.html
There is no wildcard syntax like you show. The closest thing is to omit the column names:
INSERT INTO MyTable VALUES (...);
But I don't recommend doing that. It works only if you are certain you're going to specify a value for every column in the table (even the auto-increment column), and your values are guaranteed to be in the same order as the columns of the table.
You should learn to use code to build the SQL query based on arrays of values in your application. Here's a Python example the way I do it. Suppose you have a dict of column: value pairs called data_values.
placeholders = ['%s'] * len(data_values)
sql_template = """
INSERT INTO MyTable ({columns}) VALUES ({placeholders})
"""
sql = sql_template.format(
columns=','.join(keys(data_values)),
placeholders=','.join(placeholders)
)
cur = db.cursor()
cur.execute(sql, data_values)
example code to put before your code:
cols = "("
for x in xrange(2, 52):
cols = cols + "col" + str(x) + ","
test = test[:-1]+")"
Inside your loop
for tup in l:
cur.execute(
"""insert into TABLENAME " + cols " VALUES {0}".format(tup)
""")
This is off the top of my head with no error checking
if count == 1:
cursor.execute("SELECT * FROM PacketManager WHERE ? = ?", filters[0], parameters[0])
all_rows = cursor.fetchall()
elif count == 2:
cursor.execute("SELECT * FROM PacketManager WHERE ? = ? AND ? = ?", filters[0], parameters[0], filters[1], parameters[1])
all_rows = cursor.fetchall()
elif count == 3 :
cursor.execute("SELECT * FROM PacketManager WHERE ? = ? AND ? = ? AND ? = ?", filters[0], parameters[0], filters[1], parameters[1], filters[2], parameters[2])
all_rows = cursor.fetchall()
This is a code snippet in my program. What I'm planning to do is pass the column name and the parameter in the query.
The filters array contains the columnnames, the parameter array contains the parameters. The count is the number of filters set by the user. The filters and paramters array are already ready and have no problem. I just need to pass it to the query for it to execute. This give me an error of "TypeError: function takes at most 2 arguments"
You cannot use SQL parameters to interpolate column names. You'll have to use classic string formatting for those parts. That's the point of SQL parameters; they quote values so they cannot possibly be interpreted as SQL statements or object names.
The following, using string formatting for the column name works, but be 100% certain that the filters[0] value doesn't come from user input:
cursor.execute("SELECT * FROM PacketManager WHERE {} = ?".format(filters[0]), (parameters[0],))
You probably want to validate the column name against a set of permissible column names, to ensure no injection can take place.
You can only set parameters using ?, not table or column names.
You could build a dict with predefined queries.
queries = {
"foo": "SELECT * FROM PacketManager WHERE foo = ?",
"bar": "SELECT * FROM PacketManager WHERE bar = ?",
"foo_bar": "SELECT * FROM PacketManager WHERE foo = ? AND bar = ?",
}
# count == 1
cursor.execute(queries[filters[0], parameters[0])
# count == 2
cursor.execute(queries[filters[0] + "_" + queries[filters[1], parameters[0])
This approach will make you save from SQL injection in filters[0].