I have a nested loop where I perform a calculation over every grid-point in a 3-dimensional array. Some of these grid-points trigger an error. I want to calculate every possible grid-point and skip those that trigger the error.
I've tried:
with sharedmem.MapReduce(np=45) as pool:
def calc_func(abc):
for k in range(241):
try:
for j in range(int(datetime_range)):
for l in range((abc), abc+1):
value = calc(pr[j,k,l], te[j,k,l], de[j,k,l])
array[j,k,l] = value
except (IndexError, ValueError):
continue
pool.map(cape_func, range(0, 479))
^ Above, some grid-points are calculated, but I think when k values are caught in the exception, the code doesn't calculate the rest of the grid-points for that k.
Below, I tried adding another try-except block to iterate over all of the grid-points. I don't think this worked - it seems to be looping infinitely.
with sharedmem.MapReduce(np=45) as pool:
def calc_func(abc):
for k in range(241):
try:
for j in range(int(datetime_range)):
try:
for l in range((abc), abc+1):
value = calc(pr[j,k,l], te[j,k,l], de[j,k,l])
array[j,k,l] = value
except (IndexError, ValueError):
continue
except (IndexError, ValueError):
continue
pool.map(cape_func, range(0, 479))
I think you might be looking for a pass statement instead of a continue. But I might not have understood your questions well. On the side note, can you not vectorise it?
It also looks like the only thing that could throw Index or Value errors is the calc function, since all your loop are over a ranges. Why don't you just wrap the 2 code lines with the function and array assignment?
Related
I have a function that its called recursively. When I run it I get the error "maximum recursion depth exceeded while calling a Python object"
How can increase the limit on mac? If I use the following, I get the error "cannot increase the limit on mac"
resource.setrlimit(resource.RLIMIT_STACK, (2**24,-1))
sys.setrecursionlimit(10**6)
I had a problem where I had the possibility of recurring several billions of times, and the way I did it was by flattening the recursion. I don't know if this method has been documented before, because I came up with it on my own instead of finding it. All you really have to do is put the local namespace of each function in a list. This will require a change in your actual code, if there is no workaround. Here's how it works:
Say I have this function:
def flatten_a_list(obj):#[[6,5],7,[3,[9,0]]] -> [6,5,7,3,9,0]
flattened = []
for item in obj:
if type(item) == list:
flattened.append(flatten_a_list(item))
else:
flattened.append(item)
return flattened
Now, this is traditionally recursive. To make it so that it will work for however many nestings there are with no limit, I would do this:
from copy import deepcopy
def improved(obj):#[[6,5],7,[3,[9,0]]] -> [6,5,7,3,9,0]
flattened = []
position = [0]
while True:
print('position: {}'.format(str(position)))
x = deepcopy(obj)
try:
for index in position:
x = x[index]
except (IndexError, TypeError):
break
if type(x) == list:
position.append(0)
print('continuing')
continue
else:
flattened.append(x)
#Test the next position
test = deepcopy(position)
test[-1] += 1
x = deepcopy(test)
print('x: {}'.format(x))
try:
y = deepcopy(obj)
for index in x:
y = y[index]
position = deepcopy(test)
except (IndexError, TypeError):
position = position[:-1]
try:
position[-1] += 1
except IndexError:
break
return flattened
Two words: Mind Bending
The function I wrote works fine, but it is unoptimized. If you want speed, first make sure you understand the function, then combine the checkings of index overflow by taking the 'x' and 'y' blocks of code are polymorphesizing them.
You will have to adapt this to your code, but as long as you understand it, it shouldn't be much or a problem. Plus, the answer is cross platform and unlimited.
I have code similar to this structure:
def my_gen(some_str):
if some_str == "":
raise StopIteration("Input was empty")
else:
parsed_list = parse_my_string(some_str)
for p in parsed_list:
x, y = p.split()
yield x, y
for x, y in my_gen()
# do stuff
# I want to capture the error message from StopIteration if it was raised manually
Is it possible to do this by using a for loop? I couldn't find a case similar to this elsewhere.
If using a for loop isn't possible, what are some other alternatives?
Thanks
You cannot do this in a for loop - because a for loop will implicitly catch the StopIteration exception.
One possible way to do this is with an infinite while:
while True:
try:
obj = next(my_gen)
except StopIteration:
break
print('Done')
Or you can use any number of consumers from the itertools library - have a look at the recipe section at the bottom for examples.
Having a bit of trouble with while loops. I understand this basic for loop runs through whatever is passed into the function but how would I change the for loop to a while loop? Thought it would be as easy as changing the for to while but apparently not so.
def print_data(items):
for item in items:
print(item)
You can do something like this to have the same printing functionality with a while loop:
def print_data(items):
i = 0
while i < len(items):
print items[i]
i += 1
Here is a while loop version that works by constructing an iterator and manually walking it. It works regardless of whether the input is a generator or a list.
def print_data(items):
it = iter(items)
while True:
try:
print next(it)
except StopIteration:
break
print_data([1,2,3])
One option, that works on both lists and generators, is to construct an iterator and then use Python's built-in next function. When the iterator reaches the end, the next function will raise a StopIteration exception, which you can use to break the loop:
def print_data(items):
it = iter(items)
while True:
try:
print next(it)
except StopIteration:
break
print_data(['a', 'b', 'c'])
print_data(('a', 'b', 'c'))
There's more about the next built-in function and iterators in the docs.
If you are learning Python:
If you need to iterate over an iterable (a list, a generator, a string etc.. in short and not precise words something that contains things and can "give" those things one by one..) you better use for.
In Python for was made for iterables, so you don't need a while.
If you are learning programming in general:
while needs a condition to be satisfied to keep looping, you create your own condition making a counter that will increment every loop, and making the while loop go while this counter is less or equal to the lenght of your items, as showed in Mathias711's answer.
The for-loop you are using is iterating through a so called iterator.
This means to walk through iterable objects (lists, generators, dicts,...) and return the next item from the iterator which is returned by the built-in function [next()][2]. If there is no item left, calling this function will raise an so called StopIteration error which causes to stop iteration.
So the pythonic way to iterate througth iteratable objects is in fact using the for-loop you provided in your question. However, if you really want to use a while loop (which at least in general is not recommended at all) you have to iterate using a try-except-block and handle the StopIteration error raised if no item is left.
def iterate_manually(items):
# convert items list into iterator
iterator = iter(items)
while True:
try:
print(next(iterator))
# handle StopIteration error and exit while-loop
except StopIteration:
break
iterate_manually(['foo', 'bar', 'baz'])
You can try this
def print_data(items):
i =0
while items:
if i < len(items):
print items[i]
i = i+1
else:
break
The question is use iterator to find the num from list who is greater than the nums in left and right
For example, select([0,1,0,2,0,3,0,4,0]) return [1,2,3,4]
My first try is
def select(iterable):
answer = []
it = iter(iterable)
try:
v2 = next(it)
while True:
v1,v2= v2,next(it)
if v2>v1 and v2>next(it):
answer.append(v2)
except StopIteration:
pass
return answer
This code fails.
I think the next(it) in the while loop would be the same iterator,but the next() still iter next one in the code.
then I change the code to below one, it works.
try:
v1,v2,v3 = next(it),next(it),next(it)
while True:
if v2>v1 and v2>v3:
answer.append(v2)
v1,v2,v3 = v2,v3,next(it)
except StopIteration:
pass
Can someone explain what is difference happen here?
There are two issues with the first snippet:
Every time you call next(it), it advances the iterator. You have to store the returned value if you want to access it more than once. Calling next(it) again won't do that; it will advance the iterator yet again.
Even if the first point weren't an issue, the following would still be problematic:
if v2>v1 and v2>next(it):
The issue here is short-circuit evaluation. Depending on whether v2>v1, this may or may not advance the iterator.
The error lies in this line:
if v2>v1 and v2>next(it):
Your calling next(it), but you don't store the return value. You just compare it to the v2. So the value gets skipped.
edit:
Btw, if you compare multiple values, the following comparison is much more cleaner:
if v1 < v2 > v3:
In checking for amicable numbers I've made the following if-statement (dict is dictionary):
if n == dict[lib[n]]:
amic[n] = dict[n]
But if the n value is not in the dictionary it returns an error; as it should. But I'd like it to continue because an error means "it's not equal" and it should continue to the next n.
Is this possible?
You probably want dict.get() which returns None (or another default of your choosing). For example n == lib.get(lib.get(n))
You could wrap in a try block, and ignore the exception, but given a bit more context, there's probably a better way... (lib[lib[n]] just looks odd...)
This will catch an IndexError for both lib and amic though...
try:
if n == lib[lib[n]]:
amic[n] = lib[n]
except IndexError as e: # or KeyError if a dict
pass