Suppose i want to track the progress of a loop using the progress bar printer ProgressMeter (as described in this recipe).
def bigIteration(collection):
for element in collection:
doWork(element)
I would like to be able to switch the progress bar on and off. I also want to update it only every x steps for performance reasons. My naive way to do this is
def bigIteration(collection, progressbar=True):
if progressBar:
pm = progress.ProgressMeter(total=len(collection))
pc = 0
for element in collection:
if progressBar:
pc += 1
if pc % 100 = 0:
pm.update(pc)
doWork(element)
However, I am not satisfied. From an "aesthetic" point of view, the functional code of the loop is now "contaminated" with generic progress-tracking code.
Can you think of a way to cleanly separate progress-tracking code and functional code? (Can there be a progress-tracking decorator or something?)
It seems like this code would benefit from the null object pattern.
# a progress bar that uses ProgressMeter
class RealProgressBar:
pm = Nothing
def setMaximum(self, max):
pm = progress.ProgressMeter(total=max)
pc = 0
def progress(self):
pc += 1
if pc % 100 = 0:
pm.update(pc)
# a fake progress bar that does nothing
class NoProgressBar:
def setMaximum(self, max):
pass
def progress(self):
pass
# Iterate with a given progress bar
def bigIteration(collection, progressBar=NoProgressBar()):
progressBar.setMaximum(len(collection))
for element in collection:
progressBar.progress()
doWork(element)
bigIteration(collection, RealProgressBar())
(Pardon my French, er, Python, it's not my native language ;) Hope you get the idea, though.)
This lets you move the progress update logic from the loop, but you still have some progress related calls in there.
You can remove this part if you create a generator from the collection that automatically tracks progress as you iterate it.
# turn a collection into one that shows progress when iterated
def withProgress(collection, progressBar=NoProgressBar()):
progressBar.setMaximum(len(collection))
for element in collection:
progressBar.progress();
yield element
# simple iteration function
def bigIteration(collection):
for element in collection:
doWork(element)
# let's iterate with progress reports
bigIteration(withProgress(collection, RealProgressBar()))
This approach leaves your bigIteration function as is and is highly composable. For example, let's say you also want to add cancellation this big iteration of yours. Just create another generator that happens to be cancellable.
# highly simplified cancellation token
# probably needs synchronization
class CancellationToken:
cancelled = False
def isCancelled(self):
return cancelled
def cancel(self):
cancelled = True
# iterates a collection with cancellation support
def withCancellation(collection, cancelToken):
for element in collection:
if cancelToken.isCancelled():
break
yield element
progressCollection = withProgress(collection, RealProgressBar())
cancellableCollection = withCancellation(progressCollection, cancelToken)
bigIteration(cancellableCollection)
# meanwhile, on another thread...
cancelToken.cancel()
You could rewrite bigIteration as a generator function as follows:
def bigIteration(collection):
for element in collection:
doWork(element)
yield element
Then, you could do a great deal outside of this:
def mycollection = [1,2,3]
if progressBar:
pm = progress.ProgressMeter(total=len(collection))
pc = 0
for item in bigIteration(mycollection):
pc += 1
if pc % 100 = 0:
pm.update(pc)
else:
for item in bigIteration(mycollection):
pass
My approach would be like that:
The looping code yields the progress percentage whenever it changes (or whenever it wants to report it). The progress-tracking code then reads from the generator until it's empty; updating the progress bar after every read.
However, this also has some disadvantages:
You need a function to call it without a progress bar as you still need to read from the generator until it's empty.
You cannot easily return a value at the end. A solution would be wrapping the return value though so the progress method can determine if the function yielded a progress update or a return value. Actually, it might be nicer to wrap the progress update so the regular return value can be yielded unwrapped - but that'd require much more wrapping since it would need to be done for every progress update instead just once.
Related
I might be asking a simple question. I have a python program that runs every minute. But I would like a block of code to only run once the condition changes? My code looks like this:
# def shortIndicator():
a = int(indicate_5min.value5)
b = int(indicate_10min.value10)
c = int(indicate_15min.value15)
if a + b + c == 3:
print("Trade posible!")
else:
print("Trade NOT posible!")
# This lets the processor work more than it should.
"""run_once = 0 # This lets the processor work more than it should.
while 1:
if run_once == 0:
shortIndicator()
run_once = 1"""
I've run it without using a function. But then I get an output every minute. I've tried to run it as a function, when I enable the commented code it sort of runs, but also the processing usage is more. If there perhaps a smarter way of doing this?
It's really not clear what you mean, but if you only want to print a notification when the result changes, add another variable to rembember the previous result.
def shortIndicator():
return indicate_5min.value5 and indicate_10min.value10 and indicate_15min.value15
previous = None
while True:
indicator = shortIndicator()
if previous is None or indicator != previous:
if indicator:
print("Trade possible!")
else:
print("Trade NOT possible!")
previous = indicator
# take a break so as not to query too often
time.sleep(60)
Initializing provious to None creates a third state which is only true the first time the while loop executes; by definition, the result cannot be identical to the previous result because there isn't really a previous result the first time.
Perhaps also notice the boolean shorthand inside the function, which is simpler and more idiomatic than converting each value to an int and checking their sum.
I'm guessing the time.sleep is what you were looking for to reduce the load of running this code repeatedly, though that part of the question remains really unclear.
Finally, check the spelling of possible.
If I understand it correctly, you can save previous output to a file, then read it at the beginning of program and print output only if previous output was different.
This question already has answers here:
Static variable in Python?
(6 answers)
Closed 1 year ago.
I'm trying to write a function that updates its local variable each time it is run but it is not working for some reason.
def max_equity(max_equity=0):
if current_equity() > max_equity:
max_equity = current_equity()
print(max_equity)
return max_equity
else:
print(max_equity)
return max_equity
and the function which it is calling
def current_equity():
for n in range(len(trade_ID_tracker)-1):
equity_container = 0
if (trade_ID_tracker[n,2]) == 0:
break
else:
if (trade_ID_tracker[n, 1].astype(int) == long):
equity_container += (df.loc[tbar_count,'Ask_Price'] - trade_ID_tracker[n, 2]) * trade_lots * pip_value * 1000
elif (trade_ID_tracker[n, 1].astype(int) == short):
equity_container += 0 - (df.loc[tbar_count,'Ask_Price'] - trade_ID_tracker[n, 2]) * trade_lots * pip_value * 10000
return (current_balance + equity_container)
but for some reason the max_equity() function prints current_equity() which I can only imagine means that either:
if current_equity() > max_equity:
is not doing it's job and is triggering falsely
or
max_equity = current_equity()
is not doing its job and max_equity starts at zero every time it is run.
In other words if I put max_equity() in a loop where current_equity() is
[1,2,3,4,5,4,3,2,1]
then max_equity() should return
[1,2,3,4,5,5,5,5,5]
But instead it returns
[1,2,3,4,5,4,3,2,1]
Here's a quick example test
ar = [1,2,3,4,5,4,3,2,1]
def stuff(max_equity=0):
if ar[n] > max_equity:
max_equity = ar[n]
print(max_equity)
else:
print(max_equity)
for n in range(len(ar)):
stuff()
Either way I'm kind of stumped.
Any advice?
local function variables are reset at each function call. This is essential for the behavior of functions as idempotent, and is a major factor in the success of the procedural programming approach: a function can be called form multiple contexts, and even in parallel, in concurrent threads, and it will yield the same result.
A big exception, and most often regarded as one of the bigger beginner traps of Python is that, as parameters are reset to the default values specified in the function definition for each call, if these values are mutable objects, each new call will see the same object, as it has been modified by previous calls.
This means it could be done on purpose by, instead of setting your default value as 0 you would set it as a list which first element was a 0. At each run, you could update that value, and this change would be visible in subsequent calls.
This approach would work, but it is not "nice" to depend on a side-effect of the language in this way. The official (and nice) way to keep state across multiple calls in Python is to use objects rather than functions.
Objects can have attributes tied to them, which are both visible and writable by its methods - which otherwise have their own local variables which are re-started at each call:
class MaxEquity:
def __init__(self):
self.value = 0
def update(max_equity=0):
current = current_equity()
if current > self.value:
self.value = current
return self.value
# the remainder of the code should simply create a single instance
# of that like ]
max_equity = MaxEquity()
# and eeach time yoiu want the max value, you should call its "update"
# method
I am trying to use Shady to present a sequence of image frames. I'm controlling the flow from another machine, so that I first instruct the machine running Shady to present the first frame, and later on to run the rest of the frames.
I create a World instance, and attach to it an animation callback function. Within this callback I listen for communications from the other machine (using UDP).
First I receive a command to load a given sequence (stored as a numpy array), and I do
def loadSequence(self, fname):
yy = np.load(fname)
pages = []
sz = yy.shape[0]
for j in range(yy.shape[1]/yy.shape[0]):
pages.append(yy[:, j*sz:(j+1)*sz])
deltax, deltay = (self.screen_px[0] - sz) / 2, (self.screen_px[1] - sz) / 2
if (self.sequence is None):
self.sequence = self.wind.Stimulus(pages, 'sequence', multipage=True, anchor=Shady.LOCATION.UPPER_LEFT, position=[deltax, deltay], visible=False)
else:
self.sequence.LoadPages(pages, visible=False)
When I receive the command to show the first frame, I then do:
def showFirstFrame(self, pars):
self.sequence.page = 0 if (pars[0] == 0) else (len(self.sequence.pages) - 1)
self.sequence.visible = True
But what do I do now to get the other frames to be be displayed? In the examples I see, s.page is set as a function of time, but I need to show all frames, regardless of time. So I was thinking of doing something along these lines:
def showOtherFrames(self, pars, ackClient):
direction, ack = pars[0], pars[2]
self.sequence.page = range(1, len(self.sequence.pages)) if (direction == 0) else range(len(self.sequence.pages)-2, -1, -1)
But this won't work. Alternatively I thought of defining a function that takes t as argument, but ignores it and uses instead a counter kept in a global variable, but I'd like to understand what is the proper way of doing this.
When you make s.page a dynamic property, the function assigned to it must take one argument (t), but you can still just use any variables in the space when defining that function, and not even use the time argument at all.
So, for example, you could do something as simple as:
w = Shady.World(...)
s = w.Stimulus(...)
s.page = lambda t: w.framesCompleted
which will set the page property to the current frame count. That sounds like it could be useful for your problem.
Your global-variable idea is one perfectly valid way to do this. Or, since it looks like you're defining things as methods of an instance of your own custom class, you could use instance methods as your animation callbacks and/or dynamic property values—then, instead of truly global variables, it makes sense to use attributes of self:
import Shady
class Foo(object):
def __init__(self, stimSources):
self.wind = Shady.World()
self.stim = self.wind.Stimulus(stimSources, multipage=True)
self.stim.page = self.determinePage # dynamic property assignment
def determinePage(self, t):
# Your logic here.
# Ignore `t` if you think that's appropriate.
# Use `self.wind.framesCompleted` if it's helpful.
# And/or use custom attributes of `self` if that's
# helpful (or, similarly, global variables if you must).
# But since this is called once per frame (whenever the
# frame happens to be) it could be as simple as:
return self.stim.page + 1
# ...which is indefinitely sustainable since page lookup
# will wrap around to the number of available pages.
# Let's demo this idea:
foo = Foo(Shady.PackagePath('examples/media/alien1/*.png'))
Shady.AutoFinish(foo.wind)
Equivalent to that simple example, you could have the statement self.stim.page += 1 (and whatever other logic) inside a more-general animation callback.
Another useful tool for frame-by-frame animation is support for python's generator functions, i.e. functions that include a yield statement. Worked examples are included in python -m Shady demo precision and python -m Shady demo dithering.
It can also be done in a StateMachine which is always my preferred answer to such things:
import Shady
class Foo(object):
def __init__(self, stimSources):
self.wind = Shady.World()
self.stim = self.wind.Stimulus(stimSources, multipage=True)
foo = Foo(Shady.PackagePath('examples/media/alien1/*.png'))
sm = Shady.StateMachine()
#sm.AddState
class PresentTenFrames(sm.State):
def ongoing(self): # called on every frame while the state is active
foo.stim.page += 1
if foo.stim.page > 9:
self.ChangeState()
#sm.AddState
class SelfDestruct(sm.State):
onset = foo.wind.Close
foo.wind.SetAnimationCallback(sm)
Shady.AutoFinish(foo.wind)
I am designing a Tkinter interface to be compatible with my datalogging script. Both are still WIP, but close to getting merged. This script is used for a Raspberry Pi to periodically make measurements using different sensors. In this case, of plants.
I have various Entry widgets to access my variables (input channels, measure interval and measurements per average value ), so i can change them without reprogramming the script itself.
3 entry widgets are associated with the textvariables:
gpio_light1_entry, interval_light1_entry, amount_light1_entry
The actual variables used by the datalogging script are:
gpio_light1, interval_light1, amount_light1
I want to define a function (That i will bind to a button.). That retrieves the entry window value (gpio_light1_entry.get()) and that updates the script variable.
Now I could just use:
gpio_light1 = gpio_light1_entry.get()
However I have at least 12 variables per plant. So coding it like this for some 12 times seems very inefficient to me.
I was thinking of using a for loop and lists.
settings_gpio1 = [gpio_light1, gpio_temp1, etc]
settings_gpio1_entry = [gpio_light1_entry, gpio_temp1_entry, etc]
But this had some problems:
1- It seems that changing a value in a list, does not change the variable used to construct the list.
2- I do not know how to make a "double" for loop to use both the _entry and non entry lists.
3- The _entry list needs a .get() function to retrieve the values, this function does not work on lists directly, but can be solved with a for loop.
Does anyone know a more efficient or easier way to reach my goal?
I advise you to create a class for each Gpio access:
The class it self should hold the values for accessing the hardware and displaying it.
If, for some reason, you already have the list of available gpios and entries, you can do some thing like this:
class Gpio:
def __init__(self,setting, entry):
self.setting = setting
self.entry = entry
def Get(self):
#Do whatever you need with self.entry
return 0
def __str__(self):
return self.setting
#staticmethod
def FromArray(names,settings, entries):
assert(len(settings) == len(entries) == len(names))
ret = {}
for i in range(len(settings)):
ret[names[i]] = Gpio(settings[i], entries[i])
return ret
Here I put a random example, I don't know what values you are going to use:
gpio_light1 = 33
gpio_light_entry = "hardware_address"
gpio_temp_entry = "0x80001234"
names = ["gpio_light1", "gpio_temp1"]
settings_gpio = [gpio_light1, "gpio_temp1"]
settings_gpio_entry = [gpio_light_entry, gpio_temp_entry]
access = Gpio.FromArray(names, settings_gpio, settings_gpio_entry)
print access["gpio_light1"].Get()
just registered so I could ask this question.
Right now I have this code that prevents a class from updating more than once every five minutes:
now = datetime.now()
delta = now - myClass.last_updated_date
seconds = delta.seconds
if seconds > 300
update(myClass)
else
retrieveFromCache(myClass)
I'd like to modify it by allowing myClass to update twice per 5 minutes, instead of just once.
I was thinking of creating a list to store the last two times myClass was updated, and comparing against those in the if statement, but I fear my code will get convoluted and harder to read if I go that route.
Is there a simpler way to do this?
You could do it with a simple counter. Concept is get_update_count tracks how often the class is updated.
if seconds > 300 or get_update_count(myClass) < 2:
#and update updatecount
update(myClass)
else:
#reset update count
retrieveFromCache(myClass)
Im not sure how you uniquely identify myClass.
update_map = {}
def update(instance):
#do the update
update_map[instance] = update_map.get(instance,0)+1
def get_update_count(instance):
return update_map[instance] or 0