We Keep Coding

python infinite loop and numpy delete do not work properly

I wrote a function and it does not end. Logically len(array) should be decreasing but it stuck in 227. I think numpy delete does not work properly or I made mistake somewhere??
def segmenting (file, threshold):
segments = []
check = True
count = 0
while check == True:
if len(file) <= 2:
check = False
sequence = []
ids = []
for i in range(1, len(file)):
vector = [file[i,1] - file[0,1] , file[i,2]- file[0,2] ]
magnitude = math.sqrt(vector[0]**2 + vector[1]**2)
print(i)
if magnitude <= threshold:
sequence.append(file[i])
ids.append(i)
if i == len(file) and len(sequence) == 0:
file = np.delete(file, 0 , axis = 0)
break
if len(ids) >0 and len(sequence)>0 :
segments.append(sequence)
file = np.delete(file, ids , axis = 0)
print('sequence after :',sequence)
sequence = []
ids = []
print(len(file))
return segments

The following (simplified) logic will never be executed
for i in range(1, len(file)):
if i == len(file):
file = np.delete(file, 0)
Without having a way to remove the first line of the file, you have no way to exhaust your array. This check is superfluous anyway since after each iteration you won't need the first line anymore.
As a first fix you can put the check outside the loop and only check whether you've found any matches
for i in range(1, len(file)):
...
if len(sequence) == 0:
file = np.delete(file, 0)
But that way you would have one iteration where you find (and remove) matches and then one more with no more matches where you then remove it. Therefore, as said above, you should always remove the first line after each iteration.
With more simplifications, your code can be reduced down to:
def segmenting(file, threshold):
segments = []
while len(file) > 2:
idx = np.sqrt(np.sum((file[1:,1:3] - file[0,1:3])**2, axis=1)) <= threshold
file = file[1:]
segments.append(list(file[idx]))
file = file[np.logical_not(idx)]
return segments

It's likely due to the fact you are removing element from file array within a for loop, and also trying to iterate over for loop using file array. Try iterate over a clean version of file array(no modification on it), and do the deletion on a copy of file array
For example, one possible solution is to fix this line
for i in range(1, len(file)):
Fix like below
N=len(file)
for i in range(1, N):
Also you could remove flag variable 'check' and replace with break statement

Automatically disallowing large indentations

In our code base, I often find functions formatted in the following way:
some_function_with_a_very_long_name(parameter_a,
parameter_b,
parameter_c)
This moves a lot of information to the right side of the screen and less readable than the cleaner alternative:
some_function_with_a_very_long_name(
parameter_a,
parameter_b,
parameter_c,
)
This could be detected by checking that the number of indentations in any given line is at most one indentation level greater than the line before.
Is there any linting rule (in Flake8, SonarQube or similar) that I can use to automatically check that this is done properly in our CI/CD pipeline?

Here is a function I just wrote, it returns a list of all (should be all) blocks where there is overindentation like here (also the line of code (starting at line 1)):
some_function_with_a_very_long_name(parameter_a,
parameter_b,
parameter_c)
Then you can iterate over that list to print out all those cases
Code:
def find_over_indentations(file_name, level_depth, search_possible=True):
def count_spaces_before(lst):
counter = 0
for item in lst:
if item == '':
counter += 1
counter = counter
else:
break
return counter
faulty_blocks = list()
faulty_block = ''
previous_line = ''
caution = False
previous_indentation_level = 0
with open(file_name) as file:
for index, line in enumerate(file):
split_by_space = line.split(' ')
spaces_before = count_spaces_before(split_by_space)
current_indentation_level = spaces_before // level_depth
possible_fault = spaces_before % level_depth if search_possible else 0
if ((current_indentation_level - previous_indentation_level > 1 or possible_fault != 0)
and faulty_block == ''):
caution = True
faulty_block += str(index) + ' ' + previous_line
faulty_block += str(index + 1) + ' ' + line
elif caution and current_indentation_level == previous_indentation_level:
faulty_block += str(index + 1) + ' ' + line
else:
if faulty_block != '':
faulty_blocks.append(faulty_block)
faulty_block = ''
caution = False
previous_indentation_level = current_indentation_level
previous_line = line
if faulty_block != '':
faulty_blocks.append(faulty_block)
return faulty_blocks
if __name__ == '__main__':
over_indents = find_over_indentations('test.py', 4)
for over_indented in over_indents:
print(over_indented)
Basically you can just use this code (just replace the file name with whatever file you need) and it should print out all those lines where is such issues (it will also show the "starting" line for example in the above case this will also get printed: some_function_with_a_very_long_name(parameter_a,), also possible to toggle possible faults (default is true) and it basically checks if there are indents outside of the level depth for example an extra space or two, level depth argument is how deep is level for example usually it is 4 spaces (tab)
EDIT1: there was a slight issue with the first if statement in a case when there was a possible fault (out of indention depth) where it run that if statement twice so I added a condition so that that if statement runs only if there is nothing appended to the faulty_block. (also this introduces an interesting situation in the current where it is formatted correctly as per indentation level but it would be appended to the list if this function run on this file, that is why it shows line number and previous line so that human can go and check them manually but they don't have to look all over the file)

Why is looping through a file line by line printing too many results?

I'm looping through lines in a file to create a dict with the start/stop positions, however, am getting way too many results and I'm unsure why. It looks like every addition of the variable ref_start and ref_end is being added multiple times in the dictionary.
def main():
#initialize variables for counts
gb_count = 0
glimmer_count = 0
exact_count = 0
five_prime_count = 0
three_prime_count = 0
no_matches_count = 0
#protein_id list
protein_id = []
#initialize lists for start/stop coordinates
reference = []
prediction = []
#read in GeneBank file
for line in open('file'):
line = line.rstrip()
if "protein_id=" in line:
pro_id = line.split("=")
pro_id = pro_id[1].replace('"','')
protein_id.append(pro_id)
elif "CDS" in line:
if "join" in line:
continue
elif "/translation" in line:
continue
elif "P" in line:
continue
elif "complement" in line:
value = " ".join(line.split()).replace('CDS','').replace("(",'').replace(")",'').split("complement")
newValue = value[1].split("..")
ref_start = newValue[1]
ref_end = newValue[0]
gb_count += 1
else:
test = " ".join(line.split()).replace('CDS','').split("..")
ref_start = test[0]
ref_end = test[1]
gb_count += 1
reference.append({'refstart': ref_start, 'refend': ref_end})
print(reference)

I initially posted something else that was wrong, but I copied over the code and ran a dummy file and I think I figured it out. Your problem is: for line in open('file').
What it is doing (what it did for me) is loading the file up by character. Instead of 'line' = "protein_id=", you're getting 'line' = "p" then 'line' = "r", etc.
The fix is too simple. This is what I did:
file = open('file')
for line in file:
I'm not 100% on this explanation, but I think it has to do with the way python is loading the file. Since it hasn't been established as one long string, it's loading up each individual element. Once it has been made a string, it can break it down by line. Hope this helped.

Getting a binary search to work in Python

I am trying to get a binary search to work in Python. I have a massive, sorted list of passwords. The plan is to get a password input from the user and see if it is in the list. I've decided to implement a binary search because of the size of the list.
Here's my code:
Found = False
Password = user_input("Enter a password: ")
with io.open('final.txt', encoding='latin-1') as myfile:
data = myfile.readlines()
low = 0
high = (int(len(data))+1)
while (low < high) and not Found:
mid = int((low+high)/2)
if data[mid] == Password:
Found = True
break
elif Password < str(data[mid]):
high = mid - 1
elif Password > str(data[mid]):
low = mid + 1
I am guessing it is because of the string comparison? Any ideas? The binary search never returns true, even if I explicitly search something that I know is in the list.
I used this code to sort the password list.
import io
with io.open('result.txt', encoding='latin-1') as myfile:
data = myfile.readlines()
def partition(data, start, end):
pivot = data[end] # Partition around the last value
bottom = start-1 # Start outside the area to be partitioned
top = end # Ditto
done = 0
while not done: # Until all elements are partitioned...
while not done: # Until we find an out of place element...
bottom = bottom+1 # ... move the bottom up.
if bottom == top: # If we hit the top...
done = 1 # ... we are done.
break
if data[bottom] > pivot: # Is the bottom out of place?
data[top] = data[bottom] # Then put it at the top...
break # ... and start searching from the top.
while not done: # Until we find an out of place element...
top = top-1 # ... move the top down.
if top == bottom: # If we hit the bottom...
done = 1 # ... we are done.
break
if data[top] < pivot: # Is the top out of place?
data[bottom] = data[top] # Then put it at the bottom...
break # ...and start searching from the bottom.
data[top] = pivot # Put the pivot in its place.
return top # Return the split point
def quicksort(data, start, end):
if start < end: # If there are two or more elements...
split = partition(data, start, end) # ... partition the sublist...
quicksort(data, start, split-1)
quicksort(data, split+1, end)
quicksort(data, 0, (int(len(data))-1))
with io.open('final.txt', 'w', encoding='latin-1') as f:
for s in data:
f.write(s)
The sorted list looks something like this: whitespace, then symbols, then numbers, then capital letters (alphabetically sorted), then common letters (alphabetically sorted).

Do not write your own binary search, it's a bit tricky to get them right. Use bisect module instead.
from bisect import bisect_left
def binary_search(lst, el):
# returns lower bound of key `el` in list `lst`
index = bisect_left(lst, el)
# check that: (1) the lower bound is not at the end of the list and
# (2) the element at the index matches `el`
return index < len(lst) and lst[index] == el
Usage:
test = ["abc", "def", "ghi"]
print(binary_search(test, "def")) # True
print(binary_search(test, "xyz")) # False

You probably have a new line character at the end of each password after calling readlines, use rstrip() to remove it
Found = False
Password = user_input("Enter a password: ")
with io.open('final.txt', encoding='latin-1') as myfile:
data = myfile.readlines()
low = 0
high = len(data)-1 #no need to cast to int, should be len()-1
while (low <= high) and not Found: #less than or equal to
mid = int((low+high)/2)
if data[mid].rstrip() == Password: #Remove newline character before compare
Found = True
break
elif Password < str(data[mid]):
high = mid - 1
elif Password > str(data[mid]):
low = mid + 1

If you only want to search the password in your list then In your code
data = myfile.readlines()
you have already taken all the passwords into the memory.
so if you just want to check if a given password is present in your list or not, you can directly check by using
if Password in data:
print "yes it is present in the list"
else:
print "Not present in the list"
hope it may help.

This is example of binary search
def binarySearch(alist, item):
first = 0
last = len(alist)-1
found = False
while first<=last and not found:
midpoint = (first + last)//2
if alist[midpoint] == item:
found = True
else:
if item < alist[midpoint]:
last = midpoint-1
else:
first = midpoint+1
return found
mylist1 = [0, 1, 2, 8, 9, 17, 19, 32, 42,]
print(binarySearch(mylist1, 3))
print(binarySearch(mylist1, 13))
mylist2 = [0, 1, 2, 8, 9, 17, 19, 32, 42, 99]
print(binarySearch(mylist2, 2))
print(binarySearch(mylist2, 42))
I got then
False
False
True
True
Yes and I am sure that you need new line character at the end of each password after calling readlines,as Eamon pointed out.

There are two problems .
Your binary search algorithm is wrong .
The repeat condition should be
while (low <= high)
or your can't find the first and the last element .
readlines() will read \n but user_input() does not .
Which causes `Password` == `Password\n' be false forever.

You're skipping parts of your list because of the way you're setting low and high. Because of this, low == high occurs after updating and before checking, causing you to jump out of the loop prematurely.
There are two easy solutions:
Either..
set high = mid or low = mid instead of mid -/+ 1, triggering an extra iteration,
or..
Check if high == low and data[low] == Password after the loop
terminates, as you might still find Password there.

Unbroken chain? Python iteration not being processed

So I've written a bit of code to stack integers in a list from the zeroth position. For some reason I cannot decipher, the while loop below is not being processed. I have followed all good style and syntax requirements that I know, and the while loop works when run by itself.
def row(line):
"""
Function that merges a single row or column.
"""
result_length = len(line)
print result_length
# Create a list of zeros the same length as the 'line' argument
pts_alloc = 0
dummy = 0
result = line
result[0:] = [pts_alloc for dummy in range(len(result))]
print result
#Iterate over the 'line' list looking for non-zero entries and
#stack them from 'result[0]'
line_count = 0
result_place = 0
while (line_count <= (len(line)-1)):
if (line[line_count] > 0):
result[result_place] = line[line_count]
print result
result_place += 1
line_count += 1
return result
print row([4, 0, 0, 5])
Is there a major error in this code that I've missed? Is there some syntax requirement that I am unaware of?

The problems seems to be this part:
result = line
result[0:] = [pts_alloc for dummy in range(len(result))]
By replacing a slice of result, with result = line, you are replacing that same slice in line, too, as result is just another reference to the same list, not a copy.
Since the slice is the entire list, anyway, just do:
result = [pts_alloc for dummy in range(len(result))]
Also, you are declaring a lot of unnecessary variables. You could shorten your code to this:
def row(line):
result = [0] * len(line)
result_place = 0
for x in line:
if x > 0:
result[result_place] = x
result_place += 1
return result
Or even this:
def row(line):
non_zero = [x for x in line if x > 0] # take non-zero values
return non_zero + [0] * (len(line) - len(non_zero)) # pad with zeros