I have a function that reads in two large files and merges them, and I would like to add a progress bar using TK to show the user that the function is still running.
def makeDF():
# DF SETUP---
print("Starting dataframe creation")
qData = pd.read_excel("fileOne.xlsx", sheet_name='Sheet1')
yData= pd.read_excel(fileTwo.xlsx", sheet_name='Sheet1')
allData = pd.merge(qData, yData, on=['Var1','Var2','Var3'], how="outer")
I have read through various threads and not found how to link a progress bar to a function that isn't looping or incrementing anything. I just want to start the progress bar, execute the function, have the progress bar update/show activity while the function runs, then end the progress bar when the function is complete. So far I have seen articles on threading, but they always involve some list of items that is being processed or worked through in a queue, not just one lengthy operation.
Thank you!
Related
I have a class that, calls a method from other and execute it in parallel, the function that is executed has a for loop that iterates over a dataframe. How can I display a progress bar for each thread?
class App:
otherClass = OtherClass()
n_cpus = os.cpu_count()
def call_multiprocessing(self):
with mp.Pool(processes=self.n_cpus) as executor:
for result in executor.map(otherClass.some_method, self.dataframes):
pass
class OtherClass()
def some_method(self, df):
for index, row in tqdm(dataframe.iterrows(), desc="Progress: ", total= len(df)):
#For each thread that this function is executing
#I want to display a progress bar and keep her in the screen
The way that I'm doing each time a progress bar from a thread is updated it replaces the one that is in console. I want to show then simultaneously, is it possible? Example: If the PC has 4 CPUs, 4 progress bars.
Progress bars should only run in the main thread. Use a queue or similar to pass the status to the main thread and display from there.
Here is an example using Enlighten and multiprocessing, but you should be able to achieve something similar using the threading and queue modules.
I am developing an app in Kivy and have one function that seems to take a long time to finish. Therefore, when the button that calls this function is pressed, i first open a modalview with a progress bar. I now want to update the value in the progressbar every 500ms while the main function is executing
My first attempt was to use Clock.schedule_interval(). Pseudo code looks something like this:
Class MainClass(Screen):
def button_callback_function(self):
#open modalview with progress bar
view = ModalView(autodismiss=True)
view.open()
Clock.schedule_interval(self.update_progress_bar_in_modalview,0.5)
self.function_that_takes_too_long()
def function_that_takes_too_long(self):
/* code here*/
def update_progress_bar_in_modalview(self,dt):
/*updates value of progress bar*/
With this code, the progress bar does indeed get updated but only after function_that_takes_too_long() finishes, not parallel to it.
My second attempt was to use python threads:
Class MainClass(Screen):
def button_callback_function(self):
#open modalview with progress bar
view = ModalView(autodismiss=True)
view.open()
x=threading.Thread(target=self.update_progress_bar_in_modalview)
x.start()
self.function_that_takes_too_long()
def function_that_takes_too_long(self):
/* code here*/
def update_progress_bar_in_modalview(self):
/*updates value of progress bar*/
timer.sleep(0.5)
Here the second thread to update the progress bar is never even started. It seems the main thread has to pause to give the second thread a chance to start.
So is there any way to call update_progress_bar_in_modalview() while function_that_takes_too_long() is still executing? Something like periodic interrupts in micro controllers. Or maybe start the thread that updates the progress bar on a separate core?
I appreciate any hints.
Alex
I've designed a GUI and after a button click, I am implementing a function named librosa.effects.hpss(), which is taking a lot of time for execution. As it is taking a lot of time for execution, I want to see the execution(with start and done indication also) of that function using a progress bar onto the GUI itself. How do I do it?
I've tried these implementations https://www.themarketingtechnologist.co/progress-timer-in-python/, but they are all for the for() loop implementations, which I don't have any.
This is my code for the function call.
def hpssop(self):
self.file = "file.wav"
self.y, self.sr = librosa.load(self.file)
self.margin_harms = self.margin_har.value()
self.margin_pers = self.margin_per.value()
self.harmonic, self.percussive = librosa.effects.hpss(self.y,
margin=(self.margin_harms, self.margin_pers))
librosa.output.write_wav("harmonic.wav", self.harmonic, self.sr)
librosa.output.write_wav("percussive.wav", self.percussive, self.sr)
def onseparateclick(self):
self.hpss.clicked.connect(self.hpssop)
This is the GUI but instead of separating status I want progress bar
I have a piece of code that I have used elsewhere (where it does exactly what I want), but in this case it does not. Basically my function body calculates a bunch of things, lumps the results into a pandas DataFrame and plots this to a matplotlib canvas.
I put the code below at the top of my function body, hoping for it to display a dialog box for the duration of the function call. However, all that happens is the dialog box appears, with the box contents reflecting what is immediately behind it on screen. This remains on the screen for the duration of the function call and then correctly closes when it has finished running.
Any ideas what is going wrong?
In case it's useful, the parent (self), is a QWidget.
Oh and one more thing, if I remove the progress.cancel() line, once the main part of the function has finished executing, the QProgressDialog actually paints itself with the progress bar and label text ('Updating...').
Thanks very much for your help!
progress = QProgressDialog( parent = self )
progress.setCancelButton( None )
progress.setLabelText( 'Updating...' )
progress.setMinimum( 0 )
progress.setMaximum( 0 )
progress.forceShow()
#calculate and plot DataFrame
progress.cancel()
I found myself having the same problem at some point and came up with this decorator that I apply to functions that I want to run blocking, while permitting GUI thread to process events.
def nongui(fun):
"""Decorator running the function in non-gui thread while
processing the gui events."""
from multiprocessing.pool import ThreadPool
from PyQt4.QtGui import QApplication
def wrap(*args, **kwargs):
pool = ThreadPool(processes=1)
async = pool.apply_async(fun, args, kwargs)
while not async.ready():
async.wait(0.01)
QApplication.processEvents()
return async.get()
return wrap
Then, it's easy to write your calculating function normally with the decorator:
#nongui
def work(input):
# Here you calculate the output and set the
# progress dialog value
return out
and then run it as usual:
out = work(input)
progress = QProgressDialog( parent = self )
progress.setCancelButton( None )
progress.setLabelText( 'Updating...' )
progress.setMinimum( 0 )
progress.setMaximum( 10 )
progress.forceShow()
progress.setValue(0)
#calculate and plot DataFrame
domystuff(progress) # pass progress to your function so you can continue to increment progress
qApp.processEvents() # qApp is a reference to QApplication instance, available within QtGui module. Can use this whereever you are in your code, including lines in your domystuff() function.
progress.setValue(10) # progress bar moves to 100% (based on our maximum of 10 above)
progress.done() # I don't think you want to cancel. This is inherited from QDialog
I would like to update a progress bar that I have on a main window with the progress of a task that I am doing on another sub-routine, would it be possible??
To be as clear as possible, I would have 2 files:
In my Mainwindow.py I would have something like:
import Calculations
#some code
self.ui.progressBar
Calculations.longIteration("parameters")
Then I would have a separate file for the calculations: Calculations.py
def longIteration("parameters")
#some code for the loop
"here I would have a loop running"
"And I would like to update the progressBar in Mainwindow"
Is that possible?
Or it should be done on a different way?
Thanks.
The simplest of methods would be to simply pass the GUI object:
self.ui.progressBar
Calculations.longIteration("parameters", self.ui.progressBar)
and update progressBar on Calculations. This has two problems, though:
You're mixing GUI code with Calculations, who probably shouldn't know anything about it
if longIteration is a long running function, as its name implies, you're blocking your GUI main thread, which will make many GUI frameworks unhappy (and your application unresponsive).
Another solution is running longIteration in a thread, and pass a callback function that you use to update your progress bar:
import threading
def progress_callback():
#update progress bar here
threading.Thread(target=Calculations.longIteration, args=["parameters", progress_callback]).run()
then, inside longIteration, do:
def longIteration( parameters, progress_callback ):
#do your calculations
progress_callback() #call the callback to notify of progress
You can modify the progress_callback to take arguments if you need them, obviously