Reducing execution time Python - python

given that I have a list like this (river discharge tree):
String type (11 elements).
From;To
1;2
2;3
5;4
3;4
4;-999
9;6
6;5
10;7
8;7
7;5
If you imagine it as a tree, it should be like (direction from top to bottom):
1 9 8 10
| | \/
2 6 7
| \ /
3 5
| /
4
|
I just want to expand the list so I would have all the combinations like
From;To
1;2
2;3
5;4
3;4
4;-999
9;6
6;5
10;7
8;7
7;5
1;3
1;4
6;4
9;4
9;5
7;4
8:4
8:5
10:5
10:4
There must be connection in the tree and the order must be from top to bottom.
What is the best way to do this?
I wrote a code for this but this would take me about 6 days of executing for 6000 rows.
should_restart = False
for b in range(1, lengthchange):
row1 = str(tree[b])
position2 = row1.find(delimeter)
position3 = row1.find(end)
from1 = (row1[0:position2])
to1 = row1[position2+1:position3]
for c in range(1, lengthchange):
row2 = str(tree[c])
position4 = row2.find(delimeter)
position5 = row2.find(end)
from2 = (row2[0:position4])
to2 = row2[position4+1:position5]
if to1 == from2 and not to2 == "-999":
append1 = str(from1)+";"+str(to2)+"\n"
seen = set(tree)
if append1 not in seen:
seen.add(append1)
tree.append(append1)
should_restart = True
count_test = count_test+1
print(count_test)
lengthchange = len(tree)
Could you check my code and give me some advices?
Thank you very much!

So the key to doing this efficiently is ensuring we don't have to revisit nodes over and over again. We can do this by starting with the output and working our way back:
crivers = rivers[:] # copy the rivers list, as this process is destructive
ckeys = set(river.split(";")[0] for river in crivers) # make a set for O(1) lookup
result = {}
while crivers:
for river in crivers[:]:
key, value = river.split(";")
if value in ckeys:
continue # skip rivers that are flowing into unprocessed rivers
result[int(key)] = [int(value)] + result.get(int(value), [])
ckeys.remove(key)
crivers.remove(river)
If the rivers list is sorted properly, this is O(n), if it's not sorted (or, in the worst case, reverse sorted), it's O(n**2). "Sorted properly", in this case, means they are sorted from root to leaf in our upside down tree... as our processing order is: 4, 5, 3, 6, 7, 2, 9, 10, 8, 1
The final result is:
{1: [2, 3, 4, -999],
2: [3, 4, -999],
3: [4, -999],
4: [-999],
5: [4, -999],
6: [5, 4, -999],
7: [5, 4, -999],
8: [7, 5, 4, -999],
9: [6, 5, 4, -999],
10: [7, 5, 4, -999]}
Which can be converted to your final format via:
fmt_lst = []
for key in result:
for val in result[key]:
fmt_lst.append("%s;%s" % (key, val))
['1;2', '1;3', '1;4', '1;-999',
'2;3', '2;4', '2;-999',
'3;4', '3;-999',
'4;-999',
'5;4', '5;-999',
'6;5', '6;4', '6;-999',
'7;5', '7;4', '7;-999',
'8;7', '8;5', '8;4', '8;-999',
'9;6', '9;5', '9;4', '9;-999',
'10;7', '10;5', '10;4', '10;-999']

Related

Grouping numbers and sums python

I was given some numbers lets say
1 2 3 4 5 6 7
Would there be some way to find a way to group them so that the sum of the multiple groups is odd then even? I was given this problem for fun.
So far, I have tried something like this,
numbers="7 1 3 5 7 9 11 13"
a_list = numbers.split()
map_object = map(int, a_list)
list_of_integers = list(map_object)
print(list_of_integers)
N=list_of_integers[0]
def subset_sumed(numbers, target, partial=[]):
s = sum(partial)
if s == target:
print("sum(%s)=%s" % (partial, target))
print(partial)
return(s)
if s >= target:
return
for i in range(len(numbers)):
n = numbers[i]
remaining = numbers[i + 1:]
subset_sumed(remaining, target, partial + [n])
del list_of_integers[0]
list_of_integers.sort()
sumed=list_of_integers[-1]+list_of_integers[-2]
sumss=3
possible_combinations=[]
while sumss<sumed:
a=subset_sumed(list_of_integers,int(sumss))
sumss = int(sumss)+1
possible_combinations.append(a)
i=0
nextint1=2
nextint2=1
groupnumber=0
integersused=[]
print(possible_combinations)
This will get even and then odd groups (groups of alternating parity).
The approach uses recursion and the fact that adding an even number to a number doesn't alter the parity.
Explanation is in the comments below (it is mostly comments).
def get_groups(remaining, curr_sum, group, target, result):
# case 1. End condition / base case
if not remaining:
# If the current group's sum matches target parity, simply append it as its own group
# case 1.a
if curr_sum % 2 == target:
return result + [group]
if result:
# case 1.b
# combine this group with the previous one
result[-1] = result[-1] + group
else:
# case 1.d
# There's no previous group to combine with
print('impossible!')
return None
# case 1.c
# If this group's sum is odd, but the target parity is even (which is true at this point)
# then the previous group's sum was odd and adding the group to previous group makes even
# and an even group can be added to its previous group without changing that group's parity
if curr_sum % 2 == 1:
last = result.pop()
result[-1] = result[-1] + last
return result
# case 2. reached target parity with current group's sum
if curr_sum > 0 and curr_sum % 2 == target:
return get_groups(remaining, 0, [], 1 - target, result + [group])
# case 3. keep moving rightwards, including the leftmost element into current group
head = remaining[0]
return get_groups(remaining[1:], curr_sum + head, group + [head], target, result)
def get_even_odd_groups(a):
return get_groups(a, 0, [], 0, [])
A = [7, 1, 3, 5, 7, 9, 11, 13]
B = [7, 1, 3, 5, 7, 9, 11, 13, 2, 4, 6]
C = [7, 1, 3, 5, 7, 9, 11]
D = [9, 2, 2]
print(get_even_odd_groups(A)) # case 1.a
print(get_even_odd_groups(B)) # case 1.b (but not 1.c)
print(get_even_odd_groups(C)) # case 1.c (and therefore also 1.b)
print(get_even_odd_groups(D)) # case 1.c (impossible!)
Output:
[[7, 1], [3], [5, 7], [9], [11, 13]]
[[7, 1], [3], [5, 7], [9], [2, 11, 13, 2, 4, 6]]
[[7, 1], [3], [5, 7, 9, 11]]
impossible!
None

Watching a counter, tally total and counting missed counts

I am attempting to create a piece of code that will watch a counter with an output something like:
a1 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30]
I want the code to be able to tally the total and tell me how many counts are missed for example if this happened:
a1 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24, 25, 26, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,1,2]
I would get a total of 92 still, but get feedback that 8 are missing.
I have gotten very close with the following code:
Blk_Tot = 0
CBN = 0
LBN = 0
x = 0
y = 0
z = 0
MissedBlocks = 0
for i in range(len(a1)):
CBN = a1[i]
if CBN - LBN <= 0:
if LBN == 30:
y = 30 - abs(CBN - LBN)
elif LBN < 30:
z = 30 - LBN
y = 30 - abs(CBN - LBN) + z
print(z)
Blk_Tot = Blk_Tot + y
else:
x = CBN - LBN
Blk_Tot = Blk_Tot + x
if x > 1:
MissedBlocks = MissedBlocks - 1 + x
LBN = CBN
print(Blk_Tot)
print(MissedBlocks)
If I delete anywhere between 1 and 30 it works perfectly, however if I delete across 30, say 29,30,1,2 it breaks.I don't expect it to be able to miss 30 in a row and still be able to come up with a proper total however.
Anyone have any ideas on how this might be achieved? I feel like I am missing an obvious answer :D
Sorry I think I was unclear, a1 is a counter coming from an external device that counts from 1 to 30 and then wraps around to 1 again. Each count is actually part of a message to show that the message was received; so say 1 2 4, I know that the 3rd message is missing. What I am trying to do is found out the total that should have been recieved and how many are missing from the count.
Update after an idea from the posts below, another method of doing this maybe:
Input:
123456
List[1,2,3,4,5,6]
1.Check first input to see which part of the list it is in and start from there (in case we don't start from zero)
2.every time an input is received check if that matches the next value in the array
3.if not then how many steps does it take to find that value
You don't need to keep track if you past the 30 line.
Just compare with the ideal sequence and count the missing numbers.
No knowledge if parts missing at the end.
No knowledge if more than 30 parts are missing in a block.
from itertools import cycle
def idealSeqGen():
for i in cycle(range(1,31)):
yield(i)
def receivedSeqGen():
a1 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,
5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,
1,2]
for i in a1:
yield(i)
receivedSeq = receivedSeqGen()
idealSeq = idealSeqGen()
missing = 0
ideal = next(idealSeq)
try:
while True:
received = next(receivedSeq)
while received != ideal:
missing += 1
ideal = next(idealSeq)
ideal = next(idealSeq)
except StopIteration:
pass
print (f'There are {missing} items missing')
Edit
The loop part can be a little bit simpler
missing = 0
try:
while True:
ideal = next(idealSeq)
received = next(receivedSeq)
while received != ideal:
missing += 1
ideal = next(idealSeq)
except StopIteration:
pass
print (f'There are {missing} items missing')
In general, if you want to count the number of differences between two lists, you can easily use a dictionary. The other answer would also work, but it is highly inefficient for even slightly larger lists.
def counter(lst):
# create a dictionary with count of each element
d = {}
for i in lst:
if d.get(i, None):
d[i] += 1
else:
d[i] = 1
return d
def compare(d1, d2):
# d1 and d2 are dictionaries
ans = 0
for i in d1.values():
if d2.get(i, None):
# comapares the common values in both lists
ans += abs(d1[i]-d2[i])
d2[i] = 0
else:
#for elements only in the first list
ans += d1[i]
for i in d2.values():
# for elements only in the second list
if d2[i]>0:
ans += d2[i]
return ans
l1 = [...]
l2 = [...]
print(compare(counter(l1), counter(l2)))
New code to check for missing elements from a repeating sequence pattern
Now that I have understood your question more clearly, here's the code. The assumption in this code is the list will always be in ascending order from 1 thru 30 and then repeats again from 1. There can be missing elements between 1 and 30 but the order will always be in ascending order between 1 and 30.
If the source data is as shown in list a1, then the code will result in 8 missing elements.
a1 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,
5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,
1,2]
a2 = a1.copy()
c = 1
missing = 0
while a2:
if a2[0] == c:
c+=1
a2.pop(0)
elif a2[0] > c:
missing +=1
c+=1
elif a2[0] < c:
missing += 31-c
c = 1
if c == 31: c=1
print (f'There are {missing} items missing in the list')
The output of this will be:
There are 8 items missing in the list
Let me know if this addresses your question
earlier code to compare two lists
You cannot use set as the items are repeated. So you need to sort them and find out how many times each element is in both lists. The difference will give you the missing counts. You may have an element in a1 but not in a2 or vice versa. So finding out the absolute count of missing items will give you the results.
I will update the response with better variables in my next update.
Here's how I did it:
code with comments:
a1 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30]
a2 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,
5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,1,2]
#step 1: Find out which list is longer. We will use that as the master list
if len(a1) > len(a2):
master_list = a1.copy()
second_list = a2.copy()
else:
master_list = a2.copy()
second_list = a1.copy()
#step 2: We must sort both master and second list
# so we can compare against each other
master_list.sort()
second_list.sort()
#set the counter to zero
missing = 0
#iterate through the master list and find all values in master against second list
#for each iteration, remove the value in master[0] from both master and second list
#when you have iterated through the full list, you will get an empty master_list
#this will help you to use while statement to iterate until master_list is empty
while master_list:
#pick the first element of master list to search for
x = master_list[0]
#count the number of times master_list[0] is found in both master and second list
a_count = master_list.count(x)
b_count = second_list.count(x)
#absolute difference of both gives you how many are missing from each other
#master may have 4 occurrences and second may have 2 occurrences. abs diff is 2
#master may have 2 occurrences and second may have 5 occurrences. abs diff is 3
missing += abs(a_count - b_count)
#now remove all occurrences of master_list[0] from both master and second list
master_list = [i for i in master_list if i != x]
second_list = [i for i in second_list if i != x]
#iterate until master_list is empty
#you may end up with a few items in second_list that are not found in master list
#add them to the missing items list
#thats your absolute total of all missing items between lists a1 and a2
#if you want to know the difference between the bigger list and shorter one,
#then don't add the missing items from second list
missing += len(second_list)
#now print the count of missig elements between the two lists
print ('Total number of missing elements are:', missing)
The output from this is:
Total number of missing elements are: 7
If you want to find out which elements are missing, then you need to add a few more lines of code.
In the above example, elements 27,28,29,30, 4, 5 are missing from a2 and 31 from a1. So total number of missing elements is 7.
code without comments:
a1 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30]
a2 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,
5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,1,2]
if len(a1) > len(a2):
master_list = a1.copy()
second_list = a2.copy()
else:
master_list = a2.copy()
second_list = a1.copy()
master_list.sort()
second_list.sort()
missing = 0
while master_list:
x = master_list[0]
a_count = master_list.count(x)
b_count = second_list.count(x)
missing += abs(a_count - b_count)
master_list = [i for i in master_list if i != x]
second_list = [i for i in second_list if i != x]
missing += len(second_list)
print ('Total number of missing elements are:', missing)

solving 8 puzzle problem with BFS DFS (Using Python. Needs some suggestion)

My final state is
0 1 2 3 4 5 6 7 8
my graph would look like this
graph = {0 :[1, 3],
1 :[0, 4, 2],
2 :[1, 5],
3 :[0, 4, 6],
4 :[1, 3, 5, 7],
5 :[2, 4, 8],
6 :[3, 7],
7 :[4, 6, 8],
8 :[5 ,7]
}
1 - I was wondering if I should try some other methods such as list, if else statement than graph(above).
2 - Is anything wrong with the graph?
The problem given -
Example [1,5,3,2,0,4,7,8,6] <- more like this 1 5 3 2 0 4
7 8 6
I am supposed to find final state with given state
Thank You
So, there are 4 corner cases:
Top row
Bottom row
Most left column
Most right column
(And combinations)
We can handle them easy like this:
data = [1, 5, 3,
2, 0, 4,
7, 8, 6]
width = 3
height = 3
graph = {number: list() for number in data}
for idx, number in enumerate(data):
current_height = int(idx / width)
current_width = idx % width
if current_width != width - 1: # if next element in same row
graph[number].append(data[idx + 1])
if current_width != 0: # if prev element in same row
graph[number].append(data[idx - 1])
if current_height != 0: # if there is top element
graph[number].append(data[idx - 3])
if current_height != height - 1: # if there is bottom element
graph[number].append(data[idx + 3])
import pprint
pprint.pprint(graph)
This code will construct graph, but is this all for that puzzle?.

To obtain a range of particular value form a list

Objective:
To obtain sum of the integers from a list, except ignore sections of numbers starting with a 6 and extending to the next 9 (every 6 will be followed by at least one 9). Return 0 for no numbers.
Input Master List
a=[4, 5, 6, 7, 9,2, 6,8,9]
My code so far
z6_ind=[]
z9_ind=[]
z69_ind=[]
x=None
for i,v in enumerate(a):
if v==6:
z6_ind.append(i)
if v==9:
z9_ind.append(i)
print(z6_ind. z9_ind)
My idea is to obtain the indexes of the master list in terms of separate lists (eg. z6_ind, z9_ind and finally the problematic z69_ind which should contain the range in terms of [[2,4],[6,8]] that should be excluded from the master list while sum calculation.
From above script, it gives z9_ind equals to [4, 8] whereas z6_ind equals to [2, 6].
Thanks !
I am not sure if I caught it correctly, but do you want this code?
a = [4, 5, 6, 7, 9, 2, 6, 8, 9]
sigma = 0
exc_ind = []
flag = False
for ind, item in enumerate(a):
if item == 6 and not flag:
flag = True
exc_ind.append([])
exc_ind[-1].append(ind)
elif item == 9 and flag:
exc_ind[-1].append(ind)
flag = False
elif not flag:
sigma += item
print(sigma)
print(exc_ind)
The result:
11
[[2, 4], [6, 8]]
A verbose version with generator:
a=[4, 5, 6, 7, 9,2, 6,8,9]
def iterate(iterable):
stop = False
for i in iterable:
if i == 6:
if stop == False:
stop = True
continue
elif i == 9:
if stop == True:
stop = False
continue
if stop == False:
yield i
print(sum(iterate(a))) # will sum 4, 5 and 2
Prints:
11
If you want to use the part of your code:
a_sum = sum(a)
to_reduce = 0
for (bot,top) in zip(z6_ind, z9_ind):
to_reduce += sum(a[bot:top+1])
Basically zip "pairs" both indexes and get sums between them - numbers you want to reduce a_sum by:
result = a_sum - to_reduce
You can do the following:
import numpy as np
a=np.array([4, 5, 6, 7, 9,2, 6,8,9])
ind1 = np.where(a == 6)[0]
ind2 = np.where(a == 9)[0]
indices = [item for i in zip(ind1, ind2) for item in np.arange(i[0], i[1]+1)]
result = sum(a) - sum(a[indices])
print (result)
Output
11

Summing numbers in a list except those between 6 and 7

I'm attempting to write a function which takes a list and sums all the numbers in the list, except it ignores sections of the list starting with a list and extending to a 7, but continues to sum after the 7. Here is my code:
def sum67(nums):
i = 0
sum = 0
while i < len(nums):
k = 0
if nums[i] != 0:
sum += nums[i]
i += 1
if nums[i] == 6:
for j in range(i + 1, len(nums)):
if nums[j] != 7:
k += 1
if nums[j] == 7:
k += 2
i += k
Test cases show that 6 and proceeding numbers up until and including 7 are ignored while other numbers are added to the sum, and numbers after the 7 are also added to the sum (as was intended), but for some reason any 7 after the first 7 after a 6 is not summed - this is not what I want and I'm not sure why it's happening. Any suggestions?
Test case results:
[1, 2, 2 Expected: 5. My result: 5 (OK)
[1, 2, 2, 6, 99, 99, 7] Expected: 5. My result: 5 (OK)
[1, 1, 6, 7, 2] Expected: 4 My result: 4 (Chill)
[1, 6, 2, 2, 7, 1, 6, 99, 99, 7] Expected: 2 My result: 1 (Not chill)
[1, 6, 2, 6, 2, 7, 1, 6, 99, 99, 7] Expected: 2 My result: 1 (Not chill)
[2, 7, 6, 2, 6, 7, 2, 7] Expected: 18 My result: 9 (Not chill)
`
def sum67(nums):
# flag to know if we are summing
active = True
tot = 0
for n in nums:
# if we hit a 6 -> deactivate summing
if n == 6:
active = False
if active:
tot += n
# if we hit a seven -> reactivate summing
if n == 7 and not active:
active = True
return tot
The posted code is completely broken.
For example for a list without any 6,
i will be out of bounds of the list when reaching the nums[i] == 6 condition on the last element.
You need to completely rethink the conditions inside the loop.
Here's one approach that will work.
If the current number is 6,
then skip until you see a 7, without adding to the sum.
Otherwise add to the sum.
After performing either of these two actions (skipping numbers or adding to the sum),
increment i.
def sum67(nums):
i = 0
total = 0
while i < len(nums):
if nums[i] == 6:
for i in range(i + 1, len(nums)):
if nums[i] == 7:
break
else:
total += nums[i]
i += 1
return total
Here is an intermediate alternative for learning new Python techniques:
import itertools as it
def span_sum(iterable, start=6, end=7):
"""Return the sum of values found between start and end integers."""
iterable = iter(iterable)
flag = [True]
result = []
while flag:
result.extend(list(it.takewhile(lambda x: x!=start, iterable)))
flag = list(it.dropwhile(lambda x: x!=end, iterable))
iterable = iter(flag)
next(iterable, [])
return sum(result)
# Tests
f = span_sum
assert f([1, 2, 2]) == 5
assert f([1, 2, 2, 6, 99, 99, 7] ) == 5
assert f([1, 6, 2, 2, 7, 1, 6, 99, 99, 7]) == 2
assert f([1, 6, 2, 6, 2, 7, 1, 6, 99, 99, 7]) == 2
assert f([2, 7, 6, 2, 6, 7, 2, 7]) == 18
In principle, this function filters the input, collecting values into a result that meet your condition and drops the rest, then the sum is returned. In particular you can observe the following techniques:
extending a list
itertools, e.g. itertools.takewhile, itertools.dropwhile
iterators and generators
next() function and default values
sum() function
assertion testing

Categories