We Keep Coding

How to resolve SQLAlchemy Union Throwing Error

I'm using SQL Alchemy(Python, SQLServer) Union on two queries. It throws me the below error. Please help me in resolving it.
query1 = db.query(Employee.LastName).filter(Employee.Age == 30).all()
query2 = db.query(Employee.LastName).filter(Employee.Salary > 25000).all()
query3 = union(query1, query2).all()
**"SELECT construct for inclusion in UNION or other set construct expected, got [('Joseph',),('Alan',),('Joseph',)]."**
Also tried the below query and it throws the below error
query1 = db.query(Employee.LastName).filter(Employee.Age == 30).all()
query2 = db.query(Employee.LastName).filter(Employee.Salary > 25000).all()
query3 = query1.union(query2).all()
**"'list' object has no attribute 'union'"**

Remove the .all() from the first two queries, it turns the queries into lists, but you want to pass Query instances to union.
query1 = db.query(Employee.LastName).filter(Employee.Age == 30) # <- Query
query2 = db.query(Employee.LastName).filter(Employee.Salary > 25000) # <- Query
result = query1.union(query2).all() # <- List

Python - SQL Connector: Update do not work

I want to update records in MySQL. However, I always get an error that the syntax does not work. I think it is a formatting error, however I can't manage to fix it.
Error message:
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''ovd' = 1 WHERE id = '16923'' at line 1
My code looks like:
func = ['OffizierIn vom Dienst Bezirk', 'EinsatzoffizierIn']
dbFields = ["ovd", "offizier"]
x = 0
for i in func:
el = chrome.find_element_by_id('RoleId')
for option in el.find_elements_by_tag_name('option'):
if option.text == i:
option.click()
chrome.find_element_by_id('SearchBtn').submit()
time.sleep(2)
tbody = chrome.find_element_by_id('SearchResults')
for row in tbody.find_elements_by_xpath('./tr'):
itemsEmployee = row.find_elements_by_xpath('./td')
cursor.execute('UPDATE employees SET %s = 1 WHERE id = %s;', (dbFields[x], itemsEmployee[1].text))
x = x + 1
In the first pass, the values are as in the error message: dbFields[x] = ovd itemsEmplyee[1] = 16923
The table was created as follows:
cursor.execute('CREATE TABLE IF NOT EXISTS employees (id INT NOT NULL UNIQUE, ovd BOOLEAN);')

You've encountered one of the annoyances in writing dynamic database queries: values must be quoted, if necessary, with quotation marks, as performed by the connector package, but table and column names, if quoted, are quoted with backticks. See the MySQL rules.
You need to add the column name using string formatting, then pass the value to a prepared statement:
stmt = f'UPDATE employees SET `{dbFields[x]}` = 1 WHERE id = %s;'
cursor.execute(stmt, (itemsEmployee[1].text,))

Executing a postgresql query with plpg-sql from sqlalchemy

I can't find examples using plpg-sql in raw SQL to be executed by sqlAlchemy these were the closest but no plpg-sql:
How to execute raw SQL in Flask-SQLAlchemy app
how to set autocommit = 1 in a sqlalchemy.engine.Connection
I've done research and I'm not sure if this is possible. I'm trying to either INSERT or UPDATE a record and there is no error. It must fail silently because there's no record created/updated in the database and I've explicitly set AutoCommit=True.
Python:
engine = db.create_engine(connstr, pool_size=20, max_overflow=0)
Session = scoped_session(sessionmaker(bind=engine, autocommit=True))
s = Session()
query = """DO $$
declare
ppllastActivity date;
percComplete numeric;
begin
select lastactivity into ppllastActivity FROM feeds WHERE email = :e and courseName=:c and provider = :prov;
IF COALESCE (ppllastActivity, '1900-01-01') = '1900-01-01' THEN
INSERT INTO feeds (email, courseName, completedratio, lastActivity, provider) VALUES (:e, :c, :p, :l, :prov);
ELSEIF ppllastActivity < :l THEN
UPDATE feeds set completedratio = :p,lastActivity = :l WHERE email = :e and courseName = :c and provider = :prov;
END if;
end; $$"""
params = {'e' : item.get('email').replace("'", "''").lower(), 'c' : item.get('courseName').replace("'", "''"), 'p' : item.get('progress'), 'l' : item.get('lastActivity'),'prov' : "ACG" }
result = s.execute(text(query),params)
I'm unable to troubleshoot since it doesn't give me any errors. Am I going down the wrong path? Should I just use psql.exe or can you do plpg-sql in raw SQL with sqlAlchemy?

While typing this question up I found a solution or a bug.
The automcommit=True doesn't work, you have to begin a transaction:
with s.begin():
result = s.execute(text(query),params)

SQLAlchemy order_by formula result

I am a novice in Python. Based on this SO post, I created a SQL query using PYODBC to search a MSSQL table of historic option prices and select the option symbol with a strike value closest to the desired value I specified. However, I am now trying to teach myself OOP by re-factoring this program, and to that end I am trying to implement the ORM in SQLAlchemy.
I cannot figure out how to implement a calculated Order_By statement. I don't think a calculated column would work because desired_strike is an argument that that is specified by the user(me) at each method call.
Here is the (simplified) original code:
import pyodbc
def get_option_symbol(stock, entry_date, exp_date, desired_strike):
entry_date = entry_date.strftime('%Y-%m-%d %H:%M:%S')
exp_date = exp_date.strftime('%Y-%m-%d %H:%M:%S')
cursor.execute("""select top(1) optionsymbol
from dbo.options_pricestore
where underlying=?
and quotedate=?
and expiration=?
and exchange='*'
and option_type=?
order by abs(strike - ?)""",
stock,
entry_date,
exp_date,
desired_strike,
)
row = cursor.fetchone()
return row
Maybe not the most Pythonic, but it worked. I am now encapsulating my formerly procedural code into classes, and to use SQLAlchemy's ORM, except that in this one case I cannot figure out how to represent abs(strike - desired_strike) in the Order_By clause. I have not used lambda functions much in the past, but here is what I came up with:
import sqlalchemy
class Option(Base):
__tablename__= 'options_pricestore'
<column definitions go here>
def get_option_symbol(stock, entry_date, exp_date, desired_strike):
entry_date = entry_date.strftime('%Y-%m-%d %H:%M:%S')
exp_date = exp_date.strftime('%Y-%m-%d %H:%M:%S')
qry = session.query(Option.optionsymbol).filter(and_
(Option.underlying == stock,
Option.quotedate == entry_date,
Option.expiration == exp_date,
Option.option_type== "put",
Option.exchange == "*")
).order_by(lambda:abs(Option.strike - desired_strike))
return qry
I get "ArgumentError: SQL expression object or string expected" - Any help would be greatly appreciated.

order_by wants a string - give it to it:
qry = session.query(Option.optionsymbol).filter(and_
(Option.underlying == stock,
Option.quotedate == entry_date,
Option.expiration == exp_date,
Option.option_type== "put",
Option.exchange == "*")
).order_by('abs(strike - %d)' % desired_strike)

DatabaseError: more than one row returned by a subquery used as an expression (Django)

Django through a DatabaseError when I try to use a merged query set. My code is
assetsNetwork = Asset.objects.filter(client=myClient, module__label__in=network_label_list)
vulnsNetworkRaw = Vuln.objects.none()
for asset in assetsNetwork:
vulnsNetworkRaw = vulnsNetworkRaw | asset.latest_vulns
logging.debug("+++%s+++"%vulnsNetworkRaw)
The error message is
DatabaseError: more than one row returned by a subquery used as an expression
The .latest_vulns method is
#property
def latest_scan(self):
from arachni.models import WebScan, Vulns as WebVuln
my_module = self.module
try:
return Scan.objects.filter(assets__id=self.id, status='Audit Complete').latest('completed_Date')
except:
return Scan.objects.none()
#property
def latest_vulns(self):
from arachni.models import WebScan, Vulns as WebVuln
latest_scan = self.latest_scan
return Vuln.objects.filter(scan=latest_scan, host=self.IP_Address)
Query:
2012-08-07 16:44:38 EDT STATEMENT: SELECT "pegasus_vuln"."id", "pegasus_vuln"."nvt_id", "pegasus_vuln"."scan_id", "pegasus_vuln"."host", "pegasus_vuln"."port", "pegasus_vuln"."risk_factor", "pegasus_vuln"."cvss_score", "pegasus_vuln"."status", "pegasus_vuln"."change", "pegasus_vuln"."comment", "pegasus_vuln"."description", "pegasus_vuln"."solution", "pegasus_vuln"."_order" FROM "pegasus_vuln" WHERE (("pegasus_vuln"."host" = '192.168.2.251' AND "pegasus_vuln"."scan_id" = 95 ) OR ("pegasus_vuln"."host" = '192.168.2.5' AND "pegasus_vuln"."scan_id" = (SELECT U0."id" FROM "pegasus_scan" U0)) OR ("pegasus_vuln"."host" = '10.1.10.244' AND "pegasus_vuln"."scan_id" = 109 ) OR ("pegasus_vuln"."host" = '192.168.2.5' AND "pegasus_vuln"."scan_id" = (SELECT U0."id" FROM "pegasus_scan" U0)) OR ("pegasus_vuln"."host" = '192.168.2.248' AND "pegasus_vuln"."scan_id" = (SELECT U0."id" FROM "pegasus_scan" U0))) ORDER BY "pegasus_vuln"."_order" ASC LIMIT 21
2012-08-07 16:44:38 EDT ERROR: more than one row returned by a subquery used as an expression
It successfully logs for several times, but gives an error also in the logging line. Could anybody help me? Thanks a lot.

Check code like this in your SQL. You need to use IN operator if more than one result can be fetched from the nested SQL.
"pegasus_vuln"."scan_id" = (SELECT U0."id" FROM "pegasus_scan" U0))

The problem have been solved weirdly. I added logging in latest_vulns to evaluate the query set, then everything works fine. It works even after I removed the logging.