Multiple rows from MySQL query saved in an array structure - python

I have this Python code:
self.lock_tables("read", ['nets_permissions as n', 'devices_permissions as d'])
usrs = self.db.get("SELECT n.user_id FROM nets_permissions as n \
left join devices_permissions as d \
on n.user_id = d.user_id \
where d.user_id is null \
and n.network_id=%s and n.perm<>3", netid)
self.unlock_tables()
for usr in usrs:
self.lock_tables("write", ['devices_permissions'])
self.db.execute("INSERT devices_permissions SET \
user_id=%s, network_id=%s, device_id=%s, perm=%s",\
usr, netid, sensid, perm)
self.unlock_tables();
I first do a query to retrieve some user_id from two tables. I want save this user_id in one variable and after do a for loop to insert this records in another table...
This code doesn't work. I obtain this error:
Exception: Multiple rows returned for Database.get() query
How can I retrieve this multiple rows and then process everyone of them at one time?
Thank you all.


The solution is to use self.db.query instead of self.db.get!
self.lock_tables("read", ['nets_permissions as n', 'devices_permissions as d'])
usrs = self.db.query("SELECT n.user_id FROM nets_permissions as n \
left join devices_permissions as d \
on n.user_id = d.user_id \
where d.user_id is null \
and n.network_id=%s and n.perm<>3", netid)
self.unlock_tables()
for usr in usrs:
self.lock_tables("write", ['devices_permissions'])
self.db.execute("INSERT devices_permissions SET \
user_id=%s, network_id=%s, device_id=%s, perm=%s",\
usr['user_id'], netid, sensid, perm)
self.unlock_tables();

Related

Python mysql code to insert dictionary containing an Array

I have a dictionary like this;
{'name': '0004', 'encodings': array([-2.05818519e-01, 1.50254071e-01, 6.18976653e-02, -4.57169749e-02,
-1.07391022e-01, 5.82340732e-02, 1.71395876e-02, -6.04623035e-02,
1.16265789e-01, -1.24150608e-02, 2.55038321e-01, 2.44104303e-03,
-2.83989906e-01, -7.16208220e-02, -1.18346401e-01, 6.68070763e-02,
-1.55324042e-01, -1.11675814e-01, -1.44206494e-01, -2.48661116e-02,
4.79197986e-02, -3.35404947e-02, -2.06724089e-02, 5.70063107e-02,
-1.29669383e-01, -2.63163120e-01, -2.25746073e-04, -1.47813573e-01,
6.61746860e-02, -2.05630586e-01, -2.89494134e-02, -8.06591734e-02,
-1.74903452e-01, -1.17690712e-01, -8.54253620e-02, 1.46108493e-03,
-7.83449411e-03, -7.44407028e-02, 2.03817844e-01, -4.55042198e-02,
-1.86186373e-01, -1.54956458e-02, 4.17447761e-02, 3.07781637e-01,
1.80454239e-01, 1.86630823e-02, 5.65212369e-02, -9.69169587e-02,
1.39696896e-01, -2.83250719e-01, -3.60675156e-04, 1.29852593e-01,
1.69919491e-01, 2.47877426e-02, 2.96924170e-02, -1.77335575e-01,
-2.26391852e-03, 1.38161883e-01, -1.87802404e-01, 1.11906916e-01,
4.17628363e-02, -6.03848845e-02, 4.18845750e-03, -5.18675111e-02,
2.16162637e-01, 4.84820902e-02, -1.24477677e-01, -8.92214701e-02,
1.42987236e-01, -1.07746974e-01, 1.67147964e-02, 1.29372582e-01,
-6.53869957e-02, -2.22480565e-01, -2.30741382e-01, 8.90350789e-02,
4.72032219e-01, 1.94205374e-01, -1.43704772e-01, 1.38391014e-02,
-2.22896904e-01, -4.31186557e-02, 2.22993959e-02, 5.01501486e-02,
-1.09650522e-01, 2.00281274e-02, -1.12852253e-01, 8.36469531e-02,
1.81203574e-01, -6.09542057e-03, 2.61690491e-03, 1.59612983e-01,
5.85054457e-02, -5.77166155e-02, 2.08678767e-02, 7.78703764e-02,
-1.74884677e-01, 4.89859655e-02, -4.20536213e-02, 2.84303911e-02,
5.88016734e-02, -9.87139642e-02, 1.04927823e-01, 4.22693267e-02,
-1.54544935e-01, 1.09288253e-01, -6.07409002e-03, -2.16740593e-02,
1.54772867e-03, -7.67392293e-02, -2.64447108e-02, 4.24488354e-03,
1.71442956e-01, -2.87759811e-01, 1.82956830e-01, 1.60583854e-01,
3.09638251e-02, 1.53580874e-01, 9.96040404e-02, 3.40097286e-02,
2.06465945e-02, 7.02249445e-03, -9.22998041e-02, -6.18107505e-02,
7.82211274e-02, -8.35414380e-02, 1.60512835e-01, -1.17839221e-02])}
And I used this SQL command to insert;
INSERT INTO image ( `name`, `encodings` ) VALUES ( '0004', '[-2.05818519e-01 1.50254071e-01 6.18976653e-02 -4.57169749e-02
-1.07391022e-01 5.82340732e-02 1.71395876e-02 -6.04623035e-02
1.16265789e-01 -1.24150608e-02 2.55038321e-01 2.44104303e-03
-2.83989906e-01 -7.16208220e-02 -1.18346401e-01 6.68070763e-02
-1.55324042e-01 -1.11675814e-01 -1.44206494e-01 -2.48661116e-02
4.79197986e-02 -3.35404947e-02 -2.06724089e-02 5.70063107e-02
-1.29669383e-01 -2.63163120e-01 -2.25746073e-04 -1.47813573e-01
6.61746860e-02 -2.05630586e-01 -2.89494134e-02 -8.06591734e-02
-1.74903452e-01 -1.17690712e-01 -8.54253620e-02 1.46108493e-03
-7.83449411e-03 -7.44407028e-02 2.03817844e-01 -4.55042198e-02
-1.86186373e-01 -1.54956458e-02 4.17447761e-02 3.07781637e-01
1.80454239e-01 1.86630823e-02 5.65212369e-02 -9.69169587e-02
1.39696896e-01 -2.83250719e-01 -3.60675156e-04 1.29852593e-01
1.69919491e-01 2.47877426e-02 2.96924170e-02 -1.77335575e-01
-2.26391852e-03 1.38161883e-01 -1.87802404e-01 1.11906916e-01
4.17628363e-02 -6.03848845e-02 4.18845750e-03 -5.18675111e-02
2.16162637e-01 4.84820902e-02 -1.24477677e-01 -8.92214701e-02
1.42987236e-01 -1.07746974e-01 1.67147964e-02 1.29372582e-01
-6.53869957e-02 -2.22480565e-01 -2.30741382e-01 8.90350789e-02
4.72032219e-01 1.94205374e-01 -1.43704772e-01 1.38391014e-02
-2.22896904e-01 -4.31186557e-02 2.22993959e-02 5.01501486e-02
-1.09650522e-01 2.00281274e-02 -1.12852253e-01 8.36469531e-02
1.81203574e-01 -6.09542057e-03 2.61690491e-03 1.59612983e-01
5.85054457e-02 -5.77166155e-02 2.08678767e-02 7.78703764e-02
-1.74884677e-01 4.89859655e-02 -4.20536213e-02 2.84303911e-02
5.88016734e-02 -9.87139642e-02 1.04927823e-01 4.22693267e-02
-1.54544935e-01 1.09288253e-01 -6.07409002e-03 -2.16740593e-02
1.54772867e-03 -7.67392293e-02 -2.64447108e-02 4.24488354e-03
1.71442956e-01 -2.87759811e-01 1.82956830e-01 1.60583854e-01
3.09638251e-02 1.53580874e-01 9.96040404e-02 3.40097286e-02
2.06465945e-02 7.02249445e-03 -9.22998041e-02 -6.18107505e-02
7.82211274e-02 -8.35414380e-02 1.60512835e-01 -1.17839221e-02]' );
But the encoding part is not an array anymore, but text. Therefore face recognition app is not working to check.
How can I insert and retrieve the images from MySQL database to control for face recognition?
Thanks in advance.
Edit:
I added the dictionary like this:
Dictionary;
{'name': '0001', 'encodings': array([-2.05818519e-01, 1.50254071e-01, 6.18976653e-02, -4.57169749e-02,
-1.07391022e-01, 5.82340732e-02, 1.71395876e-02, -6.04623035e-02,
1.16265789e-01, -1.24150608e-02, 2.55038321e-01, 2.44104303e-03,
-2.83989906e-01, -7.16208220e-02, -1.18346401e-01, 6.68070763e-02,
-1.55324042e-01, -1.11675814e-01, -1.44206494e-01, -2.48661116e-02,
4.79197986e-02, -3.35404947e-02, -2.06724089e-02, 5.70063107e-02,
-1.29669383e-01, -2.63163120e-01, -2.25746073e-04, -1.47813573e-01,
6.61746860e-02, -2.05630586e-01, -2.89494134e-02, -8.06591734e-02,
-1.74903452e-01, -1.17690712e-01, -8.54253620e-02, 1.46108493e-03,
-7.83449411e-03, -7.44407028e-02, 2.03817844e-01, -4.55042198e-02,
-1.86186373e-01, -1.54956458e-02, 4.17447761e-02, 3.07781637e-01,
1.80454239e-01, 1.86630823e-02, 5.65212369e-02, -9.69169587e-02,
1.39696896e-01, -2.83250719e-01, -3.60675156e-04, 1.29852593e-01,
1.69919491e-01, 2.47877426e-02, 2.96924170e-02, -1.77335575e-01,
-2.26391852e-03, 1.38161883e-01, -1.87802404e-01, 1.11906916e-01,
4.17628363e-02, -6.03848845e-02, 4.18845750e-03, -5.18675111e-02,
2.16162637e-01, 4.84820902e-02, -1.24477677e-01, -8.92214701e-02,
1.42987236e-01, -1.07746974e-01, 1.67147964e-02, 1.29372582e-01,
-6.53869957e-02, -2.22480565e-01, -2.30741382e-01, 8.90350789e-02,
4.72032219e-01, 1.94205374e-01, -1.43704772e-01, 1.38391014e-02,
-2.22896904e-01, -4.31186557e-02, 2.22993959e-02, 5.01501486e-02,
-1.09650522e-01, 2.00281274e-02, -1.12852253e-01, 8.36469531e-02,
1.81203574e-01, -6.09542057e-03, 2.61690491e-03, 1.59612983e-01,
5.85054457e-02, -5.77166155e-02, 2.08678767e-02, 7.78703764e-02,
-1.74884677e-01, 4.89859655e-02, -4.20536213e-02, 2.84303911e-02,
5.88016734e-02, -9.87139642e-02, 1.04927823e-01, 4.22693267e-02,
-1.54544935e-01, 1.09288253e-01, -6.07409002e-03, -2.16740593e-02,
1.54772867e-03, -7.67392293e-02, -2.64447108e-02, 4.24488354e-03,
1.71442956e-01, -2.87759811e-01, 1.82956830e-01, 1.60583854e-01,
3.09638251e-02, 1.53580874e-01, 9.96040404e-02, 3.40097286e-02,
2.06465945e-02, 7.02249445e-03, -9.22998041e-02, -6.18107505e-02,
7.82211274e-02, -8.35414380e-02, 1.60512835e-01, -1.17839221e-02])}
columns = ', '.join("`" + str(x).replace('/', '_') + "`" for x in data.keys())
values = ', '.join("'" + str(x).replace('/', '_') + "'" for x in data.values())
sql = "INSERT INTO %s ( %s ) VALUES ( %s );" % (self.table_name, columns, values)
with self.connection.cursor() as cursor:
cursor.execute(sql)
self.connection.commit()

You need to convert the string into array when retrieving the encode from database.
Here is a look at the sample python code.
encoding = "[-2.05818519e-01 1.50254071e-01 6.18976653e-02 -4.57169749e-02 -1.07391022e-01 5.82340732e-02 1.71395876e-02 -6.04623035e-02 1.16265789e-01 -1.24150608e-02 2.55038321e-01 2.44104303e-03 -2.83989906e-01]" // by select * from table
result = encoding[1:][:-1].split(" ")
print ("The converted array is", result)
// [-2.05818519e-01, 1.50254071e-01, 6.18976653e-02, -4.57169749e-02, -1.07391022e-01, 5.82340732e-02, 1.71395876e-02, -6.04623035e-02, 1.16265789e-01, -1.24150608e-02, 2.55038321e-01, 2.44104303e-03, -2.83989906e-01]
Maybe, inserting the dict into database would work well, if you change the type of dict into string before inserting it.

How to insert 1000 random int value rows into a column in Sqlite?

i'm a newbie in python3. My homework is create a Sqlite database include 10 tables, each table contains 50 columns, each columns contains 1000 rows, data is randomly generated using Python. I have almost done.
My code :
import sqlite3
conn = sqlite3.connect('testmydb.db')
cur = conn.cursor()
for table_number in range(1,11):
cur.execute('''CREATE TABLE table''' + str(table_number) + '''(id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT)''')
listOfColumns = ("column0",)
for column_number in range(1,49):
newColumn = ("column" + str(column_number),)
listOfColumns = listOfColumns + newColumn
for column_number in listOfColumns:
cur.execute('''ALTER TABLE table''' + str(table_number) + ''' ADD COLUMN %s TEXT''' % column_number)
conn.commit()
cur.close()
conn.close()
Now i want to insert 1000 row into 1 colums but i'm confusing when i wanted create a for loop more. Can anyone suggest me ?

The following should do what you want (see comments)
import sqlite3
import random
conn = sqlite3.connect('testmydb.db')
cur = conn.cursor()
#Only need to do this once
listOfColumns = ("column0",)
bindMarkers = ",?" #ADDED to allow values to be bound will be ?,?,?,?, ........ 49 ?
for column_number in range(1, 49):
newColumn = ("column" + str(column_number),)
listOfColumns = listOfColumns + newColumn
bindMarkers = bindMarkers + ",?"
for table_number in range(1,11):
cur.execute("DROP TABLE IF EXISTS table" + str(table_number)) #make it rerunnable
cur.execute('''CREATE TABLE IF NOT EXISTS table''' + str(table_number) + '''(id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT)''')
for column_number in listOfColumns:
cur.execute('''ALTER TABLE table''' + str(table_number) + ''' ADD COLUMN %s TEXT''' % column_number)
# The INSERT statement note null means that the id will be automatically generated
insertsql = "INSERT INTO table" + str(table_number) + " VALUES(null" + bindMarkers + ")"
#print the INSERT SQL (just the once)
if table_number == 1:
print(insertsql)
for row_number in range(1,1001):
# Generate a list of 49 random values
listOfRandomValues =[random.randint(1, 999999999999) for i in range(49)]
cur.execute(insertsql,listOfRandomValues) # insert the row
# extract the first 5 rows an print each row
cursor = cur.execute("SELECT * FROM table" + str(table_number) + " LIMIT 5")
result = "row in table table" + str(table_number) + " Data is "
for row in cursor:
print(row)
conn.commit()
cur.close()
conn.close()
This will produce output like (first line is the first INSERT statement/SQL) :-
INSERT INTO table1 VALUES(null,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
(1, '208968800970', '673486951978', '416011320117', '257739455602', '161014001387', '66915092142', '192394825558', '894946418178', '147479449787', '429768915009', '343072031065', '312483697033', '38240897968', '179592184222', '517690986147', '401721693004', '760956848808', '787028914225', '658523299261', '923606731801', '740090529164', '169600507787', '441903806645', '82302358448', '250921627878', '542116452618', '998918595471', '775548995005', '733089506549', '957054106540', '449321507524', '798501631292', '409382414444', '945602662286', '706232454927', '930118739979', '691405693853', '201175361297', '513975533346', '16690109599', '592944414377', '948709328664', '490207084748', '406188522423', '799744354342', '474761616653', '314527920015', '94102072722', '912028741567')
(2, '172875509043', '126844020427', '423436418690', '973535472434', '171412421537', '693479106176', '909004577995', '920911700813', '605955273811', '325652512054', '94057263900', '45907520985', '64928934172', '301130729226', '103229253943', '114469347031', '551553752113', '626314462779', '22617947251', '997836163264', '585793592332', '620096766798', '565760327235', '348031514661', '871589505728', '58320228377', '179652288652', '977988196994', '742110712624', '201181530463', '816248034687', '22951611374', '723154858722', '289036915539', '997272483698', '61348539011', '977373908399', '668284539899', '55348735729', '263726052214', '662603583920', '790720286573', '487793507420', '883073500835', '519633722649', '383008255347', '30563959610', '617324332661', '89956476106')
(3, '253567041183', '70027774987', '535230659770', '191267720449', '791090949115', '399626615217', '649492276413', '594283985270', '983353743022', '713002984294', '982490173135', '109850128623', '571489216078', '900560015434', '729185220526', '362712800267', '619582132251', '990925729743', '144006421433', '790742578660', '64886161120', '266462916556', '89211644675', '941650491818', '878437527129', '827767387129', '899754797443', '280555144440', '623469334050', '882001652568', '395198811620', '393149546360', '509545198950', '534252806675', '582802496697', '674715538387', '748829323303', '296068248515', '573789396002', '84015250035', '963083904856', '677426863455', '173505995385', '569976297792', '643158854425', '273191627696', '676364784545', '536715691007', '678846958313')
(4, '575055534615', '179094408882', '418242646417', '258767847915', '533305509121', '800410396430', '416643709991', '453093467839', '352906227023', '711478657972', '542560050616', '477511637703', '464619274323', '438591712313', '293891594997', '638717557413', '796607432824', '845617819673', '682479247215', '687662681530', '682910774205', '547150987433', '645097550529', '781225444825', '498491871793', '280308928866', '386747319120', '175187502068', '554032903538', '906897892968', '847200546291', '724824936579', '257524554306', '341479642174', '628478037881', '41911000836', '487139622046', '698641404274', '300203051807', '321147725978', '201308004931', '324554566932', '54668008952', '799888599714', '544776279131', '851164639529', '1118079080', '993554994315', '97774308420')
(5, '263377483252', '535276579958', '434436394255', '235123585872', '886866465625', '83437890933', '546739192349', '832929945092', '889303183895', '517501283515', '386452334064', '437005515113', '567305852696', '254940127493', '158473804439', '714105412308', '887616841407', '873758857265', '59024734698', '495085412255', '757296111012', '438130715784', '661863799528', '370244296694', '559859930401', '409259131854', '72716791778', '900054227569', '897455645761', '254989679831', '46456169823', '597888422562', '581408791663', '191438417130', '468539979785', '998729241595', '596707251066', '731997835957', '432001941801', '351970232680', '602771773558', '793033654396', '205236245465', '547142878108', '973842386021', '742055066627', '455501634405', '130419180039', '870186517783')
(1, '472841964440', '177094420514', '859773622393', '943573354468', '909606787130', '278659426379', '129796913302', '67857238168', '104155180296', '581639712382', '451184580063', '917433785632', '226959780068', '190462507493', '256274613979', '919674630928', '976702823134', '121337013780', '254022515917', '293782992065', '903483153770', '147697931939', '279062893088', '553519369139', '962433270653', '640822114280', '816716757345', '999707836592', '697963179054', '104305203866', '735705858863', '617083342099', '262076004375', '797912340506', '205887749382', '576489282235', '705096989440', '670969562520', '649164826831', '311493582872', '760367591190', '749686855909', '819181100789', '466265188300', '304292298579', '420782152623', '854335337149', '916391611738', '964274785687')
(2, '621325506597', '776006955683', '137683264810', '351906945610', '682429690372', '965366508605', '666337420753', '453325880143', '70778770818', '103682937480', '868216544504', '229703959756', '41004116292', '507097353534', '871910281669', '251530835311', '836500603189', '601460038094', '897559700303', '681312522817', '161143454247', '553960203443', '777460295192', '458302954528', '977754347041', '892360041754', '681995024692', '248485864749', '348381577064', '450879805019', '650777503736', '353872867221', '97506344721', '747237255889', '455629065944', '861413783175', '214743871915', '77511793017', '621196858622', '825422146350', '489409477723', '908004452720', '238639741015', '426722798842', '980323652543', '561628376666', '838205614824', '784039262073', '949055065484')
(3, '736008123891', '923934389646', '546159245294', '429258073881', '583372466354', '50804206500', '273716995212', '733988654121', '788160350686', '749598895287', '551751993459', '916986772574', '622366294456', '687624270621', '185660393899', '329963428664', '928661078668', '875765821125', '754653923243', '151547845857', '248763933358', '636547599095', '87140063802', '267688269107', '224477253917', '641792646340', '59046381016', '103443043545', '485267444040', '387215340714', '268223896307', '480068950182', '225811319773', '492031230630', '502916805016', '514567127425', '178032451267', '750288734257', '825600642728', '641081438590', '207022050440', '902457228778', '115373751089', '348372424350', '768147081429', '715162751738', '210598155420', '196905259558', '873091126544')
(4, '560125266801', '378302831641', '471084702841', '679900688640', '201624340251', '909766550240', '687623074376', '116508086811', '217573740193', '378086229046', '466649195230', '932285473013', '648745964471', '968517127245', '748917121449', '224930472692', '698734544540', '793428186573', '153336974374', '24843476682', '42926459163', '503345524005', '116363947828', '524399560588', '238188045685', '3353134402', '97245283198', '780904780984', '768226492682', '337351478339', '761762114083', '4108216481', '715457129140', '718946387960', '808632491477', '283509135313', '750631442686', '302040053814', '354520401885', '30869550070', '831081853310', '317334330124', '175699898404', '316762996417', '144843539429', '647890863625', '500905345131', '686585819856', '439083530058')
(5, '786320993918', '418227705376', '222672045565', '50994821164', '445050766070', '655740733971', '144925180595', '178456995314', '968483620704', '217344736719', '659133382247', '699130444999', '645737723689', '211418136852', '977174813693', '404005933734', '416012774264', '498694089898', '286235598876', '105048705716', '745323502156', '22320974963', '287621972357', '484051431377', '677832782489', '175141638805', '652237666867', '633826915005', '826792363302', '181964153730', '549735148579', '820006084751', '622355043852', '615716362152', '337022948655', '280970738440', '264064973515', '550249406679', '912858473551', '542805313957', '43397863679', '257720759974', '189160263335', '265086252271', '692156831796', '860245023055', '769544988002', '856033591981', '865669688852')
(1, '29773154022', '105812125224', '923886735040', '494040618517', '406872772654', '964605045362', '483548207268', '222657267987', '728533595865', '427758006630', '250839721516', '246117222632', '625392752778', '372756660516', '276521371279', '677307428516', '434498176501', '757867858941', '568841625163', '315224423736', '939706907834', '567757610656', '977473375050', '476473505693', '921117900131', '344700573908', '350627473109', '569315794206', '780528101292', '957322180230', '952406583209', '435610932961', '463449885730', '174468401098', '916963726643', '193968348451', '297427605119', '481930164885', '685603984144', '543719297225', '612929787721', '475021539217', '176642603133', '74400339089', '95276914071', '808000358479', '79312180687', '502877681225', '659274942719')
..........
i still don't get what is variable bindMarkers and if table_number == 1: print(insertsql)
First the 2nd the line if table_number == 1: print(insertsql)
Is just printing out the INSERT statement to show what it looks like. It was just included for that and is not necessary. BUT, it's useful to know what it looks like to explain the ? placeholder and binding values.
So the INSERT statement is along the lines of
INSERT INTO tablex VALUES(null,?,? ....... (49 ?'s)
tablex where x represents 1-10
First null as per the comment allows SQLite to generate a unique value for the id column.
Each ? is a placeholder and will be replaced by a bound value. This technique prevents SQL injection.
bindMarkers is just a string that is generated with 1 ? per column so it's a string of 49 ?'s (easier than typing VALUES(null,?,?,?,?,? .....) and also more flexible/adaptable if the column number were to change).
You see that the line listOfRandomValues =[random.randint(1, 999999999999) for i in range(49)] creates a List of 49 random values each will be used to replace a single ? (the first value replaces the first ?, the second value replaces the second ? and so on).
This is considered better practice than building a statement along the lines of
INSERT INTO tablex VALUES(null,'208968800970', '673486951978', '416011320117', '257739455602', '161014001387', '66915092142', '192394825558', '894946418178', '147479449787', '429768915009', '343072031065', '312483697033', '38240897968', '179592184222', '517690986147', '401721693004', '760956848808', '787028914225', '658523299261', '923606731801', '740090529164', '169600507787', '441903806645', '82302358448', '250921627878', '542116452618', '998918595471', '775548995005', '733089506549', '957054106540', '449321507524', '798501631292', '409382414444', '945602662286', '706232454927', '930118739979', '691405693853', '201175361297', '513975533346', '16690109599', '592944414377', '948709328664', '490207084748', '406188522423', '799744354342', '474761616653', '314527920015', '94102072722', '912028741567')
The statement itself is shorter (i.e 1 ? instead of 12 digits) and therefore less likely to cause issues with limits.

How use sqlalchemy join and union together?

query1 = self.session.query(ID_Streets, ID_Nps, ID_Streets_history, Modify_reason)\
.join(ID_Nps, ID_Nps.id_np == ID_Streets.id_np)\
.outerjoin(ID_Streets_history, text('(ID_Streets.id_np=ID_Streets_history.id_np '\
'AND ID_Streets.id_street=ID_Streets_history.id_street)'))\
.outerjoin(Modify_reason, text('(ID_Streets_history.code_reason=Modify_reason.code_reason '\
'AND ID_Streets_history.code_detail=Modify_reason.code_detail)'))\
.group_by(ID_Streets.id_np, ID_Streets.id_street, ID_Nps.id_region,\
ID_Nps.id_atu, ID_Nps.id_selsov, ID_Nps.id_np, ID_Streets_history.id_np,\
ID_Streets_history.id_street, ID_Streets_history.id_row,\
Modify_reason.code_reason, Modify_reason.code_detail)
query2 = self.session.query(\
sql.null().label('id_streets_id_np'),\
sql.null().label('id_streets_id_street'),\
sql.null().label('id_streets_name_street'),\
sql.null().label('id_streets_type_street'),\
ID_Streets_history, ID_Nps, Modify_reason)\
.outerjoin(ID_Nps, ID_Nps.id_np == ID_Streets_history.id_np)\
.outerjoin(Modify_reason, text('(ID_Streets_history.code_reason=Modify_reason.code_reason '\
'AND ID_Streets_history.code_detail=Modify_reason.code_detail)'))\
.group_by(ID_Nps.id_region,\
ID_Nps.id_atu, ID_Nps.id_selsov, ID_Nps.id_np, ID_Streets_history.id_np,\
ID_Streets_history.id_street, ID_Streets_history.id_row,\
Modify_reason.code_reason, Modify_reason.code_detail)
query = query1.union(query2)
Error:
Don't know how to join from NULL; please use select_from() to establish the left entity/selectable of this join
If fix query2:
self.session.query(\
ID_Streets_history, ID_Nps, Modify_reason\
sql.null().label('id_streets_id_np'),\
sql.null().label('id_streets_id_street'),\
sql.null().label('id_streets_name_street'),\
sql.null().label('id_streets_type_street'))
Error:
(Psycopg2.ProgrammingError) ERROR: in the UNION construct, you can not generalize the types of character varying and integer
LINE 4: ... istory.id_street AS id_streets_history_id_street, id_streets ..
Generated SQL:
SELECT anon_1.id_streets_id_np AS anon_1_id_streets_id_np, anon_1.id_streets_id_street AS anon_1_id_streets_id_street, anon_1.id_streets_name_street AS anon_1_id_streets_name_street, anon_1.id_streets_type_street AS anon_1_id_streets_type_street, anon_1.id_nps_id_region AS anon_1_id_nps_id_region, anon_1.id_nps_id_atu AS anon_1_id_nps_id_atu, anon_1.id_nps_id_selsov AS anon_1_id_nps_id_selsov, anon_1.id_nps_id_np AS anon_1_id_nps_id_np, anon_1.id_nps_name_np AS anon_1_id_nps_name_np, anon_1.id_nps_type_np AS anon_1_id_nps_type_np, anon_1.id_nps_okato AS anon_1_id_nps_okato, anon_1.id_nps_oktmo AS anon_1_id_nps_oktmo, anon_1.id_streets_history_id_np AS anon_1_id_streets_history_id_np, anon_1.id_streets_history_id_street AS anon_1_id_streets_history_id_street, anon_1.id_streets_history_id_row AS anon_1_id_streets_history_id_row, anon_1.id_streets_history_name_street AS anon_1_id_streets_history_name_street, anon_1.id_streets_history_type_street AS anon_1_id_streets_history_type_street, anon_1.id_streets_history_code_reason AS anon_1_id_streets_history_code_reason, anon_1.id_streets_history_code_detail AS anon_1_id_streets_history_code_detail, anon_1.id_streets_history_creater AS anon_1_id_streets_history_creater, anon_1.id_streets_history_createddate AS anon_1_id_streets_history_createddate, anon_1.id_streets_history_updater AS anon_1_id_streets_history_updater, anon_1.id_streets_history_updateddate AS anon_1_id_streets_history_updateddate, anon_1.id_streets_history_recordcomment AS anon_1_id_streets_history_recordcomment, anon_1.id_streets_history_sourcecomment AS anon_1_id_streets_history_sourcecomment, anon_1.modify_reason_code_reason AS anon_1_modify_reason_code_reason, anon_1.modify_reason_code_detail AS anon_1_modify_reason_code_detail, anon_1.modify_reason_comment_reason AS anon_1_modify_reason_comment_reason
FROM (SELECT id_streets.id_np AS id_streets_id_np, id_streets.id_street AS id_streets_id_street, id_streets.name_street AS id_streets_name_street, id_streets.type_street AS id_streets_type_street, id_nps.id_region AS id_nps_id_region, id_nps.id_atu AS id_nps_id_atu, id_nps.id_selsov AS id_nps_id_selsov, id_nps.id_np AS id_nps_id_np, id_nps.name_np AS id_nps_name_np, id_nps.type_np AS id_nps_type_np, id_nps.okato AS id_nps_okato, id_nps.oktmo AS id_nps_oktmo, id_streets_history.id_np AS id_streets_history_id_np, id_streets_history.id_street AS id_streets_history_id_street, id_streets_history.id_row AS id_streets_history_id_row, id_streets_history.name_street AS id_streets_history_name_street, id_streets_history.type_street AS id_streets_history_type_street, id_streets_history.code_reason AS id_streets_history_code_reason, id_streets_history.code_detail AS id_streets_history_code_detail, id_streets_history.creater AS id_streets_history_creater, id_streets_history.createddate AS id_streets_history_createddate, id_streets_history.updater AS id_streets_history_updater, id_streets_history.updateddate AS id_streets_history_updateddate, id_streets_history.recordcomment AS id_streets_history_recordcomment, id_streets_history.sourcecomment AS id_streets_history_sourcecomment, modify_reason.code_reason AS modify_reason_code_reason, modify_reason.code_detail AS modify_reason_code_detail, modify_reason.comment_reason AS modify_reason_comment_reason
FROM id_streets JOIN id_nps ON id_nps.id_np = id_streets.id_np LEFT OUTER JOIN id_streets_history ON (ID_Streets.id_np=ID_Streets_history.id_np AND ID_Streets.id_street=ID_Streets_history.id_street) LEFT OUTER JOIN modify_reason ON (ID_Streets_history.code_reason=Modify_reason.code_reason AND ID_Streets_history.code_detail=Modify_reason.code_detail) GROUP BY id_streets.id_np, id_streets.id_street, id_nps.id_region, id_nps.id_atu, id_nps.id_selsov, id_nps.id_np, id_streets_history.id_np, id_streets_history.id_street, id_streets_history.id_row, modify_reason.code_reason, modify_reason.code_detail UNION SELECT id_streets_history.id_np AS id_streets_history_id_np, id_streets_history.id_street AS id_streets_history_id_street, id_streets_history.id_row AS id_streets_history_id_row, id_streets_history.name_street AS id_streets_history_name_street, id_streets_history.type_street AS id_streets_history_type_street, id_streets_history.code_reason AS id_streets_history_code_reason, id_streets_history.code_detail AS id_streets_history_code_detail, id_streets_history.creater AS id_streets_history_creater, id_streets_history.createddate AS id_streets_history_createddate, id_streets_history.updater AS id_streets_history_updater, id_streets_history.updateddate AS id_streets_history_updateddate, id_streets_history.recordcomment AS id_streets_history_recordcomment, id_streets_history.sourcecomment AS id_streets_history_sourcecomment, id_nps.id_region AS id_nps_id_region, id_nps.id_atu AS id_nps_id_atu, id_nps.id_selsov AS id_nps_id_selsov, id_nps.id_np AS id_nps_id_np, id_nps.name_np AS id_nps_name_np, id_nps.type_np AS id_nps_type_np, id_nps.okato AS id_nps_okato, id_nps.oktmo AS id_nps_oktmo, modify_reason.code_reason AS modify_reason_code_reason, modify_reason.code_detail AS modify_reason_code_detail, modify_reason.comment_reason AS modify_reason_comment_reason, NULL AS id_streets_id_np, NULL AS id_streets_id_street, NULL AS id_streets_name_street, NULL AS id_streets_type_street
FROM id_streets_history LEFT OUTER JOIN id_nps ON id_nps.id_np = id_streets_history.id_np LEFT OUTER JOIN modify_reason ON (ID_Streets_history.code_reason=Modify_reason.code_reason AND ID_Streets_history.code_detail=Modify_reason.code_detail) GROUP BY id_nps.id_region, id_nps.id_atu, id_nps.id_selsov, id_nps.id_np, id_streets_history.id_np, id_streets_history.id_street, id_streets_history.id_row, modify_reason.code_reason, modify_reason.code_detail) AS anon_1

query2 = self.session.query(\
sql.null().label('id_streets_id_np'),\
sql.null().label('id_streets_id_street'),\
sql.null().label('id_streets_name_street'),\
sql.null().label('id_streets_type_street'),
ID_Streets_history, ID_Nps, Modify_reason)\
.select_from(ID_Streets_history, ID_Nps, Modify_reason)\
.outerjoin(ID_Nps, ID_Nps.id_np == ID_Streets_history.id_np)\
.outerjoin(Modify_reason, text(\
'(ID_Streets_history.code_reason=Modify_reason.code_reason '\
'AND ID_Streets_history.code_detail=Modify_reason.code_detail)'))

What does this Python/SQL code do?

So if anyone is knowledgeable on Animal Shelter Manager, I'm looking for some help. I'm trying to figure out what each line of code means.. the first is
from animalcontrol.py
def get_animalcontrol_query(dbo):
return "SELECT ac.*, ac.ID AS ACID, s.SpeciesName, x.Sex AS SexName, " \
"co.OwnerName AS CallerName, co.HomeTelephone, co.WorkTelephone, co.MobileTelephone, " \
"o1.OwnerName AS OwnerName, o1.OwnerName AS OwnerName1, o2.OwnerName AS OwnerName2, o3.OwnerName AS OwnerName3, " \
"o1.OwnerName AS SuspectName, o1.OwnerAddress AS SuspectAddress, o1.OwnerTown AS SuspectTown, o1.OwnerCounty AS SuspectCounty, o1.OwnerPostcode AS SuspectPostcode, " \
"o1.HomeTelephone AS SuspectHomeTelephone, o1.WorkTelephone AS SuspectWorkTelephone, o1.MobileTelephone AS SuspectMobileTelephone, " \
"vo.OwnerName AS VictimName, vo.OwnerAddress AS VictimAddress, vo.OwnerTown AS VictimTown, vo.OwnerCounty AS VictimCounty, vo.OwnerPostcode AS VictimPostcode," \
"vo.HomeTelephone AS VictimHomeTelephone, vo.WorkTelephone AS VictimWorkTelephone, vo.MobileTelephone AS VictimMobileTelephone, " \
"ti.IncidentName, ci.CompletedName, pl.LocationName " \
"FROM animalcontrol ac " \
"LEFT OUTER JOIN species s ON s.ID = ac.SpeciesID " \
"LEFT OUTER JOIN lksex x ON x.ID = ac.Sex " \
"LEFT OUTER JOIN owner co ON co.ID = ac.CallerID " \
"LEFT OUTER JOIN owner o1 ON o1.ID = ac.OwnerID " \
"LEFT OUTER JOIN owner o2 ON o2.ID = ac.Owner2ID " \
"LEFT OUTER JOIN owner o3 ON o3.ID = ac.Owner3ID " \
"LEFT OUTER JOIN owner vo ON vo.ID = ac.VictimID " \
"LEFT OUTER JOIN pickuplocation pl ON pl.ID = ac.PickupLocationID " \
"LEFT OUTER JOIN incidenttype ti ON ti.ID = ac.IncidentTypeID " \
"LEFT OUTER JOIN incidentcompleted ci ON ci.ID = ac.IncidentCompletedID"
What does return "SELECT ac.*, ac.ID AS ACID, mean.
and if I wanted to differ this code from what it is currently what would I have to change. ei "ac." or "ACID"
I know I will have to change def get_animalcontrol_query(dbo):

The code you've referenced is selecting all the rows from the animalcontrol table and JOINING all the data that is also contained in the species, lksex, owner, pickuplocation, incidenttype, and incidentcompleted tables. Basically it is taking all the information from the animalcontrol table and looking for related information in each of the other tables, then returning all those results to the user. For more on 'LEFT OUTER JOIN' see: http://www.1keydata.com/sql/left-outer-join.html The query is also "aliasing" the column names http://www.w3schools.com/sql/sql_alias.asp so that the results make sense to the user.
If you want to change this code, you should learn some basic SQL so you understand what you are changing.

custom sorting for find command output

I'm trying to get sorted directory/file list with unix "find" command.
# find . -type f
.
./bin
./data
./data/disks
./inc
./inc/calls
./inc/calls/show
./inc/calls/show/system
./inc/calls/show/cli
./inc/calls/show/network
./inc/calls/show/stats
./inc/calls/services
./inc/calls/services/ntp
./inc/calls/services/tsa
./inc/calls/services/webgui
./inc/calls/services/engine
./inc/calls/system
./inc/calls/change
./inc/calls/change/password
./inc/calls/change/network
./inc/calls/disk
./inc/calls/disk/encr
./inc/etc
I want to sort it like:
./inc/calls/show/system \
./inc/calls/show/cli \
./inc/calls/show/network \
./inc/calls/show/stats \
./inc/calls/services/ntp \
./inc/calls/services/tsa \
./inc/calls/services/webgui \
./inc/calls/services/engine \
./inc/calls/change/password \
./inc/calls/change/network \
./inc/calls/disk/encr \
./inc/calls/system \
./inc/calls/change \
./inc/calls/services \
./inc/calls/disk \
./inc/calls/show \
./inc/calls \
./data/disks \
./inc/etc \
./bin \
./data \
./inc
Which node (directory/file) has more child (directory/files) should be first... i want to do it with bash or python... What is the best way to do that?

Match lines containing / and prepend the number of fields to the line using / as the separator, sort on the numbers of fields and remove the count.
$ awk -F/ '/\//{print NF,$0}' file | sort -nrk1 | cut -d' ' -f2-
./inc/calls/show/system
./inc/calls/show/stats
./inc/calls/show/network
./inc/calls/show/cli
./inc/calls/services/webgui
./inc/calls/services/tsa
./inc/calls/services/ntp
./inc/calls/services/engine
./inc/calls/disk/encr
./inc/calls/change/password
./inc/calls/change/network
./inc/calls/system
./inc/calls/show
./inc/calls/services
./inc/calls/disk
./inc/calls/change
./inc/etc
./inc/calls
./data/disks
./inc
./data
./bin

I would use python and try to convert:
a/b
a/c
b/e/f
b/e/g
in something like:
{'a': {'b': {}, 'c': {}},
'b': {'e': {'f': {}, 'g': {}}},
}
To achieve this:
def add_list_to_dict(lst,d):
key, lst = lst[0], lst[1:]
if not key in d:
d[key] = {}
if lst:
add_list_to_dict(lst,d[key])
d = {}
for path in paths:
add_list_to_dict(path.split('/'),d)

Categories