We Keep Coding

How to use a list in a raw django SQL query that uses `WHERE ... IN (...)`?

How do you inject a list parameter into a Django raw query?
Given a list of UUIDs i'd like to be able to inject them into the () in a WHERE ... IN (...)
query.
list_of_uuids = [
"<UUID_1>",
"<UUID_2>
]
Output for SQL:
SELECT *
FROM some_model_table
WHERE some_model_table.uuid IN ( "<UUID_1>", "<UUID_2>" )
So I tried doing the following using raw django queries:
query_set = SomeModel.objects.raw("SELECT * FROM some_model_table where some_model_table.uuid IN %s", [list_of_uuids])
Unfortunately, the above will give the following error:
django.db.utils.ProgrammingError: syntax error at or near "ARRAY":
LINE X: WHERE uuid IN ARRAY['00123...
Which means the list is interpreted as an array and passed to the SQL query as an array, which cannot be used in an IN query.
Enclosing the injected array in () also doesn't work:
LINE X: WHERE uuid IN (ARRAY['00123...
^
HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
Note: I'm aware this particular example can be done with Django ORM, but my actual query is more complex than this and has to be done with raw queries, and contains this WHERE ... IN (...) syntax. I just kept the irrelevant code out of the question to make it easier to parse. For those who want to use ORM, here is the code you need:
query_set = SomeModel.objects.filter(uuid__in=list_of_uuids)
I have tried building the delimited list of UUID strings myself and injecting it, unfortunately this is injected in the format WHERE uuid IN ('"some_uuid", "other_uuid"')
You could use .format(...) on the query string before handing it over to Django ORM, but this opens us up to SQL injection.
I've also looked into a way of getting SQL to interpret the Array as the input for the WHERE IN query but I haven't had much luck there.
Is there another way? Could we somehow parse the array given to the SQL into valid syntax for the WHERE ... IN (...) query?

You can use ANY(%s), for example in your case:
query_set = SomeModel.objects.raw("SELECT * FROM some_model_table where some_model_table.uuid = ANY(%s)", [list_of_uuids])

Python/Pyodbc/SQL - Updating a table and setting a field to a CSV File

I am trying to use pyodbc to update an existing MS Access database table with a very long multiline string. The string is actually a csv that has been turned into a string.
The query I am trying to use to update the table is as follows:
query = """
UPDATE Stuff
SET Results = '{}'
WHERE AnalyteName =
'{}'
""".format(df, analytename)
The full printed statement looks as follows:
UPDATE Stuff
SET Results =
'col a,col b,col c,...,col z,
Row 1,a1,b1,c1,
...,...,...,...,
Row 3000,a3000,b3000,c3000'
WHERE AnalyteName = 'Serotonin'
However this does not seem to be working, and I keep getting the following error:
pyodbc.ProgrammingError: ('42000', '[42000] [Microsoft][ODBC Microsoft Access Driver] Syntax error in UPDATE statement. (-3503) (SQLExecDirectW)')
Which I assume is due to the format of the csv string I am trying to use to update the table with.
I have tried using INSERT and inserting a new row with the csv string and other relevant information and that seems to work. However, I need to use UPDATE as I will eventually be adding other csv strings to these columns. This leads me to believe that there is A) Something is wrong with the syntax of my UPDATE query (I am new to SQL syntax) or B) I am missing something from the documentation regarding UPDATE queries.
Is executing an UPDATE query like this possible? If so, where am I going wrong?

It would be determined by the table's field type.
For large amounts of text you'd need a blob field in your database table.
A blob field will store binary info so using blob will not 'see' illegal characters.

Answering my own question in case anyone else wants to use this.
It turns out what I was missing was brackets around the table column fields from my UPDATE statement. My final code looked something like this.
csv = df.to_csv(index=False)
name = 'some_name'
query = """
UPDATE Stuff
SET
[Results] = ?
WHERE
[AnalyteName] = ?
"""
self.cursor.execute(query, (csv, name))
I've seen several other posts here where brackets were not around the column names. However, since this is MS Access, I believe they were required for this query, or rather this specific query since it included a very long strong in the SET statement.
I welcome anyone else here to provide a more efficient method of performing this task or someone else who can provide more insight into why this is what worked for me.

Getting error when running a sql select statement in python

I am new to this and trying to learn python. I wrote a select statement in python where I used a parameter
Select """cln.customer_uid = """[(num_cuid_number)])
TypeError: string indices must be integers

Agree with the others, this doesn't look really like Python by itself.
I will see even without seeing the rest of that code I'll guess the [(num_cuid_number)] value(s) being returned is a string, so you'll want to convert it to integer for the select statement to process.

num_cuid_number is most likely a string in your code; the string indices are the ones in the square brackets. So please first check your data variable to see what you received there. Also, I think that num_cuid_number is a string, while it should be in an integer value.
Let me give you an example for the python code to execute: (Just for the reference: I have used SQLAlchemy with flask)
#app.route('/get_data/')
def get_data():
base_sql="""
SELECT cln.customer_uid='%s' from cln
""" % (num_cuid_number)
data = db.session.execute(base_sql).fetchall()

Pretty sure you are trying to create a select statement with a "where" clause here. There are many ways to do this, for example using raw sql, the query should look similar to this:
query = "SELECT * FROM cln WHERE customer_uid = %s"
parameters = (num_cuid_number,)
separating the parameters from the query is secure. You can then take these 2 variables and execute them with your db engine like
results = db.execute(query, parameters)
This will work, however, especially in Python, it is more common to use a package like SQLAlchemy to make queries more "flexible" (in other words, without manually constructing an actual string as a query string). You can do the same thing using SQLAlchemy core functionality
query = cln.select()
query = query.where(cln.customer_uid == num_cuid_number)
results = db.execute(query)
Note: I simplified "db" in both examples, you'd actually use a cursor, session, engine or similar to execute your queries, but that wasn't your question.

How to get Peewee ORM contains column working with join

I'm doing a join across two tables, pretty simple set up, but when I add a contains or startswith that references a column in the table being joined I can never get the results. No errors, but the count is always 0, despite me knowing that the records exist and being able to write the equivalent query in raw SQL and have it return all the results I expect.
Here's what it looks like, assume A and B are tables, they're related through a foreign key, and both the fields I'm using in the where clause are CharField.
This version does not work despite me expecting it to:
(A.select().join(B).where(
A.some_column.contains(B.other_column)
))
But this does work as expected:
(A.select().join(B).where(
SQL("t1.some_column ILIKE '%%' || t2.other_column || '%%'")
))
I would expect those two to be equivalent, but they're not. Looking at the output SQL from the first one it looks like this:
(SELECT "t1"."some_column" from "A" as "t1"
INNER JOIN "B" as "t2" ON ("t1"."b_id" = "t2"."id")
WHERE ("t1"."some_column" ILIKE %s)', ['%<CharField: B.other_column>%'])
The interesting thing to me about the SQL output is at the end where it's referencing B.other_column. I'm guessing that if it were t2.other_column instead then the query would work, but how do I make peewee do that? I've tried everything I can think of and I can't figure out a pure ORM way to get this working.

The contains method performs interpolation of the parameter.
To achieve what you're trying to do, you would stay away from the "contains" method and use the ILIKE operation.
A.select().join(B).where(
A.some_column % ('%' + B.other + '%'))
The first "%" is the operator overload for ILIKE. The '%' + B.other + '%' will concatenate the wildcards for substring search.
UPDATE: I felt like this was a legit issue, so I've made a small change to make the .contains(), .startswith() and .endswith() methods work properly when the right-hand-side value is, for example, a field. Going forward it should work more intuitively.
Commit here: https://github.com/coleifer/peewee/commit/0c98f3e1f556eba10cbbdf7c386c49c64f4da41c

KeyError when creating a view

I want to use SQLAlchemy to create a view in my PostgreSQL database. I'm using the CreateView compiler from sqlalchemy-views. I'm using the answer to this question as a reference:
How to create an SQL View with SQLAlchemy?
My code for creating the view looks like this:
def create_view(self, myparameter):
mytable = Table('mytable', metadata, autoload=True)
myview = Table('myview', metadata)
engine.execute(CreateView(myview, mytable.select().where(mytable.c.mycolumn==myparameter)))
However, when I attempt to run this query, the following exception is thrown:
KeyError: 'mycolumn_1'
Looking at the compiled query, it seems that a placeholder for my parameter value is not being replaced:
'\nCREATE VIEW myview AS SELECT mytable.mycolumn \nFROM mytable \nWHERE mytable.mycolumn = %(mycolumn_1)s\n\n'
Since the placeholder is not being replaced, the query obviously fails. However, I do not understand why the replacement does not happen, since my code does not differ much from the example.
My first suspicion was that maybe the type of the parameter and the column were incompatible. Currently, the parameter comes in as a unicode string, which should be mapped to a text column in my database. I have also tried mapping the parameter as a long to a bigint column with the same (failed) result.
Does anyone have another suggestion?

From the SQLAlchemy documentation, I can see that when one wants to pass the actual value that will be ultimately used at expression time, the bindparam() is used. A nice example is also provided:
from sqlalchemy import bindparam
stmt = select([users_table]).\
where(users_table.c.name == bindparam('username'))