We Keep Coding

Entering multiple lists at once in SQLi Database

I am using the following command to enter list values in a SQLi Database.
sql_comm = "INSERT INTO Revisions (Work_ID,Rev_No,Name,Date) VALUES (?,?,?,?);"
data = (wid,i,rev_name,rev_date)
crsr.execute(sql_comm,data)
sql_com2 = "INSERT INTO Revisions (Field,New_Value) VALUES (?,?)"
for elem in zip(changes,vals):
crsr.execute(sql_com2,elem)
connection.commit()
This is the resulting output.
Work_ID Rev_No Name Date Field New Value
172 1 Raul Gitta <raoul_g#ware.23biz> 2021-10-21T10:08:10.653Z null null
null null null null System.Rev 2
null null null null System.AuthorizedDate 2021-10-20T06:42:01.32Z
null null null null System.RevisedDate 2021-10-21T10:08:10.653Z
null null null null System.ChangedDate 2021-10-20T06:42:01.32Z
null null null null System.Watermark 198
Is there a way to avoid these null values?

All the null values stem from that fact that they are not inserted in the appropriate INSERT query. It might be easier to use lists for the parameters instead of tuples, since lists are mutable.
Something like:
data = [wid,i,rev_name,rev_date]
sql_com2 = "INSERT INTO Revisions (Work_ID,Rev_No,Name,Date,Field,New_Value)
VALUES (?,?,?,?,?,?)"
for elem in zip(changes,vals):
crsr.execute(sql_com2,data + list(elem))
connection.commit()

How to know which records cause an issue when I run a SQL MERGE statement in Python

I was running below codes in python. It's doing a merge from one table to another table. But sometimes it gave me errors due to duplicates. How do I know which records have been merge and which one has not so that I can trace the records and fix it. Or at least, how to make my code log hinted error message so that I can trace it?
# Exact match client on NAME/DOB (not yet using name_dob_v)
sql = """
merge into nf.es es using (
select id, name_last, name_first, dob
from fd.emp
where name_last is not null and name_first is not null and dob is not null
) es6
on (upper(es.patient_last_name) = upper(es6.name_last) and upper(es.patient_first_name) = upper(es6.name_first)
and es.patient_dob = ems6.dob)
when matched then update set
es.client_id = ems6.id
, es.client_id_comment = '2 exact name/exact dob match'
where
es.client_id is null -- exclude those already matched
and es.patient_last_name is not null and es.patient_first_name is not null and es.patient_dob is not null
and es.is_lock = 'Locked' and es.is_active = 'Yes' and es.patient_last_name NOT IN ('DOE','UNKNOWN','DELETE', 'CANCEL','CANCELLED','CXL','REFUSED')
"""
log.info(sql)
curs.execute(sql)
msg = "nf.es rows updated with es6 client_id due to exact name/dob match: %d" % curs.rowcount
log.info(msg)
emailer.append(msg)

You can't know, merge won't tell you. You have to actually find them and take appropriate action.
Maybe it'll help if you select distinct values:
merge into nf.es es using (
select DISTINCT --> this
id, name_last, name_first, dob
from fd.emp
...
If it still doesn't work, then join table to be merged with the one in using clause on all columns you're doing it already and see which rows are duplicate. Something like this:
SELECT *
FROM (SELECT d.id,
d.name_last,
d.name_first,
d.dob
FROM fd.emp d
JOIN nf.es e
ON UPPER (e.patient_last_name) = UPPER (d.name_last)
AND UPPER (e.patient_first_name) = UPPER (d.name_first)
WHERE d.name_last IS NOT NULL
AND d.name_first IS NOT NULL
AND d.dob IS NOT NULL)
GROUP BY id,
name_last,
name_first,
dob
HAVING COUNT (*) > 2;

Python Change part of a string based on a condition from a list

Hi im trying to make a python script which will update a line each time the script is run.
im having a bit of a headscratcher to how best to tackle the part where i have to update each line (a select statement) based on the value of a dataframe.
simplified i got a string 'select null r_cnt, null t_cnt, null r_dur, null t_dur from my_table'
and i got a list containing the fields for this line [t_cnt, r_dur]
I then want the new string output to be the first string where we have removed the null in front of the the values which was present in my list but kept null in front of those not in the list.
'select null r_cnt, t_cnt, r_dur, null t_dur from my_table'
my whole code looks something like this now where im stuck at the point i mentioned above
str_to_execute = f"select * from {db}.table_desc where grp_id in (400,500,300,1200) and id not in(127,140,125)"
cursor.execute(str_to_execute)
df = as_pandas(cursor)
for index, row in df.iterrows():
# print(row['name'])
str_to_execute = f"SHOW COLUMN STATS {db}.ctrl_{row['id']}"
cursor.execute(str_to_execute)
loop = as_pandas(cursor)
for index, row in loop.iterrows():
print(row['Column'])
str_to_execute = f"select concat(cast(ctrl_id as string),cast(ctrl_date as string)) primarykey, ctrl_id, ctrl_date,null r_cnt, null t_cnt, null r_dur, null t_dur,null r_amt, null t_amt,null p_cnt, null p_dur,null p_amt, null ro_vol, null t_vol, null r_vol, null p_vol, null ro_amt, null ro_cnt, from {db}.ctrl_{row['id']}"
if #This is where im stuck

Try:
s = 'select null r_cnt, null t_cnt, null r_dur, null t_dur from my_table'
lst = ['t_cnt', 'r_dur']
checklist = ['null r_cnt', 'null t_cnt', 'null r_dur']
checkliststr = ','.join(checklist)
for itm in lst:
if itm in checkliststr:
print('null ' + itm)
s=s.replace('null ' + itm, itm)
print(s)

You could just split the functionality of your string in this case Select and from my table from the input of your statement, which is more or less if I understand your correctly a list of strings.
One possible solution would be the following:
Define or get your lists of strings:
origin_list = ["null r_cnt", "null t_cnt", "null r_dur", "null t_dur"]
goal_list = ["t_cnt", "r_dur"]
For each element of your origin_list you want to edit the element according to its existence in the goal_list, I would do it with a map and lambda:
edited_list = list(map(lambda x: edit(x, goal_list), origin_list))
Now we have to define the function and logic from your post I derived the following logic or something similar:
Now you have the adjusted string and can merge it back together with your functionality.

SQL Query rounding issue

I am using pyodbc to fetch total quantity of a product based on a certain criteria called "Strength". The "Strength" column also has string value for some records, so it is a varchar column.
The user enters details like brand, product type, product line, date range, minimum quantity (in this case, 12), and strength range
This is my query:
SELECT SUM(CAST([Qty] AS decimal(10, 2))) AS Qty
FROM (
SELECT
[Brand],
[ProdType],
[Lot],
CAST([Strength] AS DECIMAL(10,4)) AS [Strength],
[ProductLine],
[Size],
[Stage],
[Customer],
[PackedOn],
[Qty],
[RefreshedBy],
[RefreshedOn]
FROM SalesData
WHERE
(isnumeric([Strength]) = 1)
AND [Stage]='WIP'
AND [PackedOn]>='2018-06-03'
AND [PackedOn]<='2020-06-03'
AND [Brand]='ABC'
AND [ProductLine]='DEF'
AND [Size]='15'
AND [Qty]>='12.0'
AND [Strength]>=0.2
AND [Strength]<=0.4
AND [ProdType] Is Null
) as outputdata
This is my table:
ID Brand ProdType Lot Strength ProductLine Size Stage Province PackedOn Qty
1 ABC NULL XXXXXXX 0.16 DEF 15 WIP NULL 2018-12-07 1200
This is the create statement
CREATE TABLE [dbo].[SalesData](
[ID] [int] NOT NULL,
[Brand] [varchar](max) NOT NULL,
[ProdType] [varchar](max) NULL,
[Lot] [varchar](max) NOT NULL,
[Strength] [varchar](max) NOT NULL,
[ProductLine] [varchar](max) NOT NULL,
[Size] [varchar](max) NOT NULL,
[Stage] [varchar](max) NOT NULL,
[Province] [varchar](max) NULL,
[PackedOn] [date] NOT NULL,
[Qty] [float] NOT NULL,
[RefreshedBy] [varchar](max) NULL,
[RefreshedOn] [varchar](max) NULL
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO
My problem is that this query results in a Quantity of 1200, even though it is outside the strength range. I am using SQL Server Management Studio V 18.4. How do I solve this?

In your WHERE clause you should use.
TRY_CAST([Strength] AS DECIMAL(10,4))>=0.2 AND TRY_CAST([Strength] AS DECIMAL(10,4))<=0.4
Because sql queries start working from where clauses( and joins) then executes other parts. SELECT is the least important part and if you only use CAST in your select it will be only useful for printing data as your preferred format.
SELECT SUM(CAST([Qty] AS decimal(10, 2))) AS Qty FROM
(SELECT [Brand], [ProdType], [Lot], CAST([Strength] AS DECIMAL(10,4)) AS [Strength], [ProductLine], [Size], [Stage], [Customer], [PackedOn], [Qty], [RefreshedBy], [RefreshedOn]
FROM SalesData
WHERE (isnumeric([Strength]) = 1) AND [Stage]='WIP' AND [PackedOn]>='2018-06-03'
AND [PackedOn]<='2020-06-03' AND [Brand]='ABC' AND [ProductLine]='DEF'
AND [Size]='15' AND [Qty]>='12.0' AND TRY_CAST([Strength] AS DECIMAL(10,4))>=0.2 AND TRY_CAST([Strength] AS DECIMAL(10,4))<=0.4 AND [ProdType] Is Null) as outputdata

You need to CAST() before doing numeric comparison, otherwise SQL Server compares strings, not numbers, which leads to unexpected results: as an example, string-wise, '2' is greater than '12' (since it starts with '2', which is greater than '1') That's true of all numeric comparisons involved in the query (Size is also concerned).
I would suggest TRY_CAST(), which avoids error and returns null if conversion fails (which will effectively fail the condition, and remove the corresponding row from the query).
Also, the subquery is unnecessary.
Consider:
SELECT SUM(Qty) Qty
FROM SalesData
WHERE
Stage = 'WIP'
AND PackedOn >= '2018-06-03'
AND PackedOn <= '2020-06-03'
AND Brand = 'ABC'
AND ProductLine = 'DEF'
AND ProdType Is Null
AND Qty >= 12
AND TRY_CAST(Strength AS DECIMAL(10, 4)) >= 0.2
AND TRY_CAST(Strength AS DECIMAL(10, 4)) <= 0.4
AND TRY_CAST(Size AS INT) = 15
If you want to cast your float output to decimal, it is more accurate to this after the sum(), so:
SELECT CAST(SUM(Qty) AS DECIMAL(10, 2)) Qty
FROM ...

SQLite: cast integer field value as float

I have an SQL database that I'm pulling values with a WHERE clause to filter. However, I want to see if an INTEGER field value is a substring of a TEXT field value. The caveat is that I need to use wildcards.
Some sample data:
id components
123 '234,345'
234 -
345 -
456 -
So I thought to use CAST(id AS TEXT) and concatenate it with '%' wildcard characters (using ||) in case the id is in the middle of the components string. But this doesn't work, i.e.
SELECT t1.*, t2.* FROM table t1 JOIN table t2 WHERE (t1.components IS NULL OR
(t1.components IS NOT NULL AND t1.components
NOT LIKE '%' || CAST(t2.id AS TEXT) || '%'))
OR (t2.components IS NULL OR (t2.components IS NOT NULL AND t2.components
NOT LIKE '%' ||
CAST(t1.id AS TEXT) || '%')))
Should NOT return record pairs [123,234] or [123,345]
In short, how do I get '%' || CAST(t2.id AS TEXT) || '%' read in as '%234%' in my query?
Thanks!
UPDATE 9-29-2014: Ok, I figured it out. It does work. I just had a problem with my parentheses and had an OR where I should have had an AND. The new working code (though probably not optimal #deets) is:
SELECT t1.*, t2.* FROM table t1 JOIN table t2 WHERE (t1.components IS NULL OR
(t1.components IS NOT NULL AND t1.components
NOT LIKE '%' || CAST(t2.id AS TEXT) || '%'))
AND (t2.components IS NULL OR (t2.components IS NOT NULL AND t2.components
NOT LIKE '%' || CAST(t1.id AS TEXT) || '%')))