We Keep Coding

Perform modifications in the scene and update custom window from a QThread in Maya

Context
I'm creating a PySide2 tool running in Maya. The tool is executing a lot of long tasks, some modifying the scene (cleaning tasks), some creating files (exporting tasks).
Because this is a long task, I'd like to display feedback (progress bar) while it's running.
Problems
Unfortunately, so far, the whole UI does not seem to be updated during the executing.
Also, because I had odd behaviors (Maya freezing forever) in the real code, I'm guessing this is not a safe use of threads.
Example code
Here is a simplified bit of code showing where I am so far. Is this the right way to use QThread? I'm from a CG Artist background, not a professional programmer, so I'm probably misusing or misunderstanding the concepts I'm trying to use (threads, PySide...)
import time
from PySide2.QtGui import *
from PySide2.QtCore import *
from PySide2.QtWidgets import *
import maya.cmds as cmds
class Application(object):
def __init__(self):
self.view = View(self)
def do_something(self, callback):
start = int(cmds.playbackOptions(q=True, min=True))
end = int(cmds.playbackOptions(q=True, max=True))
# First operation
for frame in xrange(start, end + 1):
cmds.currentTime(frame, edit=True)
# Export ...
callback(33)
time.sleep(1)
# Second operation
for frame in xrange(start, end + 1):
cmds.currentTime(frame, edit=True)
# Export ...
callback(66)
time.sleep(1)
# Third operation
for frame in xrange(start, end + 1):
cmds.currentTime(frame, edit=True)
# Export ...
callback(100)
time.sleep(1)
class View(QWidget):
def __init__(self, controller):
super(View, self).__init__()
self.controller = controller
self.thread = None
self.setLayout(QVBoxLayout())
self.progress = QLabel()
self.layout().addWidget(self.progress)
self.button = QPushButton('Do something')
self.layout().addWidget(self.button)
self.button.clicked.connect(self.do_something)
self.show()
def do_something(self):
self.thread = DoSomethingThread(self.controller)
self.thread.updated.connect(lambda progress: self.progress.setText(str(progress) + '%'))
self.thread.run()
class DoSomethingThread(QThread):
completed = Signal()
updated = Signal(int)
def __init__(self, controller, parent=None):
super(DoSomethingThread, self).__init__(parent)
self.controller = controller
def run(self):
self.controller.do_something(self.update_progress)
self.completed.emit()
def update_progress(self, progress):
self.updated.emit(int(progress))
app = Application()

Threads are difficult to use correctly in Maya Python (you can see this from the number of questions listed here)
Generally there are two hard rules to observe:
all work that touches the Maya scene (say selecting or moving an object) has to happen in the main thread
all work that touches Maya GUI also has to happen in the main thread.
"main thread" here is the thread you get when you run a script from the listener, not on you're creating for yourself
This obviously makes a lot of things hard to do. Generally a solution will involve the a controlling operation running on the main thread while other work that does not touch Maya GUI or scene objects is happening elsewhere. A thread-safe container (like a python Queue can be used to move completed work out of a worker thread into a place where the main thread can get to it safely, or you can use QT signals to safely trigger work in the main thread.... all of which is a bit tricky if you're not far along in your programming career.
The good news is -- if all the work you want to do in Maya is in the scene you aren't losing much by not having threads. Unless the work is basically non-Maya work -- like grabbing data of the web using an HTTP request, or writing a non-Maya file to disk, or something else that does not deal with Maya-specific data -- adding threads won't get you any additional performance. It looks like your example is advancing the time line, doing work, and then trying to update a PySide GUI. For that you don't really need threads at all (you also don't need a separate QApplication -- Maya is already a QApplication)
Here's a really dumb example.
from PySide2.QtCore import *
from PySide2.QtGui import *
from PySide2.QtWidgets import *
import maya.cmds as cmds
class DumbWindow(QWidget):
def __init__(self):
super(DumbWindow, self).__init__()
#get the maya app
maya_app = QCoreApplication.instance()
# find the main window for a parent
for widget in maya_app.topLevelWidgets():
if 'TmainWindow' in widget.metaObject().className():
self.setParent(widget)
break
self.setWindowTitle("Hello World")
self.setWindowFlags(Qt.Window)
self.layout = QVBoxLayout()
self.setLayout(self.layout)
start_button = QPushButton('Start', self)
stop_button = QPushButton('Stop', self)
self.layout.addWidget(start_button)
self.layout.addWidget(stop_button)
self.should_cancel = False
self.operation = None
self.job = None
# hook up the buttons
start_button.clicked.connect(self.start)
stop_button.clicked.connect(self.stop)
def start(self):
'''kicks off the work in 'this_is_the_work'''
self.operation = self.this_is_the_work()
self.should_cancel = False
self.job = cmds.scriptJob(ie=self.this_makes_it_tick)
def stop(self):
''' cancel before the next step'''
self.should_cancel = True
def this_is_the_work(self):
print "--- started ---"
for frame in range(100):
cmds.currentTime(frame, edit=True)
yield "advanced", frame
print "--- DONE ----"
def bail(self):
self.operation = None
def kill_my_job():
cmds.scriptJob(k=self.job)
print "job killed"
cmds.scriptJob(ie = kill_my_job, runOnce=True)
def this_makes_it_tick(self):
'''
this is called whenever Maya is idle and thie
'''
# not started yet
if not self.operation:
return
# user asked to cancel
if self.should_cancel:
print "cancelling"
self.bail()
return
try:
# do one step. Here's where you can update the
# gui if you need to
result = next(self.operation)
print result
# example GUI update
self.setWindowTitle("frame %i" % result[-1])
except StopIteration:
# no more stpes, we're done
print "completed"
self.bail()
except Exception as e:
print "oops", e
self.bail()
test = DumbWindow()
test.show()
Hitting start creates a maya scriptJob that will try to run whatever operation is in the function called this_is_the_work(). It will run to the next yield statement and then check to make sure the user hasn't asked to cancel the job. Between yields Maya will be busy (just as it would if you entered some lines in the listener) but if you're interacting with Maya when a yield comes up, the script will wait for you instead. This allows for safe user interaction without a separate thread, though of course it's not as smooth as a completely separate thread either.
You'll notice that this kicks off a second scriptJob in the bail() method -- that's because a scriptJob can't kill itself, so we create another one which will run during the next idle event and kill the one we don't want.
This trick is basically how most of the Maya's MEL-based UI works under the hood -- if you run cmds.scriptJob(lj=True) in the listener you'll usually see a lot of scriptJobs that represent UI elements keeping track of things.

How to correctly lock Qthreads in pyqt5 using Python3

I am relatively new to python, but was able to get a reasonably useful program to run to crunch a lot of data. I am able to run it over multiple sets of data sequentially using another python script to call the program serially, but I wanted to create a GUI and use multithreading to allow others to use it without knowing all the ins and outs of programming. I created the GUI successfully, and can feed data bidirectionally using signals and slots. What I am having trouble with is creating multiple threads with the same function.
I have done some research and it appears that the function needs to be threadsafe, and unfortunately mine is not because I am using curve_fit() from scipy, which is not threadsafe. So, based on what I have read in this forum and others, I should be using mutex.lock(), but I get the "SystemError: null argument to internal routine" when calling curve_fit()
Here is some sample code to demonstrate what I have done:
import sip
sip.setapi('QString', 2)
import sys, time
from PyQt5 import QtCore, QtGui, uic, QtWidgets
from ZthCalculation import ZthObject
qtCreatorFile = "PyQtZthUI_01.ui" # Enter file here.
Ui_MainWindow, QtBaseClass = uic.loadUiType(qtCreatorFile)
#class MyApp(QtGui.QMainWindow, Ui_MainWindow):
class MyApp(QtWidgets.QMainWindow, Ui_MainWindow):
def __init__(self):
super(self.__class__, self).__init__()
QtWidgets.QMainWindow.__init__(self)
Ui_MainWindow.__init__(self)
self.setupUi(self)
self.RunButton.clicked.connect(self.RunZthTest)
.
.
.
def RunZthTest(self):
#create as processes instead of threads???
# self.Process1 = QtCore.QProcess()
self.Thread1 = QtCore.QThread()
self.obj1 = ZthObject(self.InputWet1.text(), self.InputDry1.text(), self.Output1.text(), self.side1)
self.obj1.moveToThread(self.Thread1)
self.Thread1.started.connect(self.obj1.ZthCalculation)
self.obj1.textBox.connect(self.updateTextBox1)
self.signal1 = self.obj1.finished.connect(self.Thread1.quit)
self.Thread1.setObjectName("Thread1")
self.Thread1.start()
time.sleep(.1)
self.Thread2 = QtCore.QThread()
self.obj2 = ZthObject(self.InputWet2.text(), self.InputDry2.text(), self.Output2.text(), self.side2)
self.obj2.moveToThread(self.Thread2)
self.Thread2.started.connect(self.obj2.ZthCalculation)
self.obj2.textBox.connect(self.updateTextBox2)
self.signal2 = self.obj2.finished.connect(self.Thread2.quit)
self.Thread2.setObjectName("Thread2")
self.Thread2.start()
time.sleep(.1)
self.Thread3 = QtCore.QThread()
self.obj3 = ZthObject(self.InputWet3.text(), self.InputDry3.text(), self.Output3.text(), self.side3)
self.obj3.moveToThread(self.Thread3)
self.Thread3.started.connect(self.obj3.ZthCalculation)
self.obj3.textBox.connect(self.updateTextBox3)
self.signal3 = self.obj3.finished.connect(self.Thread3.quit)
self.Thread3.setObjectName("Thread3")
self.Thread3.start()
.
.
.
if __name__ == "__main__":
app = QtWidgets.QApplication(sys.argv)
window = MyApp()
window.show()
# sys.exit(app.exec_())
app.exec_()
In another file, I have the main function that I am calling as a thread:
class ZthObject(QtCore.QObject):
killthread = False
finished = QtCore.pyqtSignal()
textBox = QtCore.pyqtSignal(str)
def __init__(self, wetFilePath, dryFilePath, outFilePath, side, parent=None):
super(self.__class__, self).__init__()
self.wetFilePath = wetFilePath
self.dryFilePath = dryFilePath
self.outFilePath = outFilePath
self.side = side
self.mutex = QtCore.QMutex()
def cleanup(self):
ZthObject.killthread = True
# def ZthCalculation(self, wetFilePath, dryFilePath, outFilePath, side):
def ZthCalculation(self):
#calculations here
.
.
.
print("waypoint2")
self.mutex.lock()
popt, pcov = curve_fit(Foster6, timeShort, ZthjcShort, p0 = [Rs, taus])
self.mutex.unlock()
.
.
.
self.finished.emit()
I can successfully run the code only calling one thread, but if I call multiple threads, then the output window prints out 'waypoint2' for each thread called, then crashes with the system error I mentioned above.
What am I doing wrong? Do I need to use separate processes instead of Qthreads? Am I misunderstanding how threads work? I want them to operate in independent variable spaces.

Using a mutex really only makes something thread safe if all of the other things sharing the internals of the function also respects the mutex. In this case, it won't because, while using a mutex prevents simultaneous calls to curve_fit, you don't know what it is that is thread unsafe about the function, and so you can't be sure that something else won't also use the thread unsafe bit of code at the same time in another thread (e.g. the main thread).
Coupled with the fact that the Python GIL prevents true threading (threading only provides a speed boost in Python if your task is IO bound rather than CPU bound), I would suggest moving to a multiprocess model.

Why does pyInstaller packed program creates guardian when creating a new process on Windows?

I am trying to write an alarm clock program in python using multiprogramming module on Windows 7.
It all runs good in the interpreter. But when packed in one file by pyinstaller, every time the code create a process, there turn out to be 2 processes, one is a parent and the other is its child. When the code kills the parent process, the child become an orphan process.
The code:
from multiprocessing import Process,freeze_support
import time
import winsound
def startout(seconds,name):
freeze_support()
print name+':pid '+str(os.getpid())+' is created'
startTime=time.time()
while (time.time()-startTime)<seconds:
time.sleep(1)
winsound.PlaySound('SystemQuestion', winsound.SND_ALIAS)
print name+' end'
class alarmCenter:
def __init__(self):
self.alarmList={'alarm1':None,'alarm2':None,'alarm3':None}
def newAlarm(self,seconds,name):
if self.alarmList[name]!=None:
if self.alarmList[name].is_alive():
return False
ala=Process(target=startout, args=(seconds,name))
ala.deamon=True
ala.start()
self.alarmList[name]=ala
return True
def stopAlarm(self,name):
try:
self.alarmList[name].terminate()
self.alarmList[name].join()
self.alarmList[name]=None
except Exception:
pass
def terminateAll(self):
for each in self.alarmList.keys():
if self.alarmList[each]!=None:
self.alarmList[each].terminate()
if __name__=='__main__':
freeze_support()
#....
Note that multiprocessing.freeze_support() is already there.
Could anyone please show me how to kill the child process or fix this bug?

Is using python threading with gtk, and cmd ok?

I have a script I wrote out of modifications of helloworld for gkt, and cmd.
#!/usr/bin/python
import cmd
from gi.repository import Gtk
import threading
class GtkInterface(object):
def __init__(self):
win = Gtk.Window()
win.connect("delete-event", Gtk.main_quit)
win.show_all()
self.window = win;
def create_button(self):
self.button = Gtk.Button(label="Click Here")
self.button.connect("clicked", self.on_button_clicked)
self.window.add(self.button)
self.window.show_all()
def on_button_clicked(self, widget):
print 'something happened'
return
class HelloWorld(cmd.Cmd):
#Simple command processor example.
prompt='>'
def __init__(self, gtk_object):
cmd.Cmd.__init__(self)
# or cmd.Cmd.__init__(self)
self.gtk_object = gtk_object
def do_greet(self, line):
print "hello"
def do_setbutton(self, line):
self.gtk_object.create_button()
def do_exit(self, line):
return True
gtk_o = GtkInterface()
hello = HelloWorld(gtk_o)
def worker(num):
"""thread worker function"""
#print 'Worker: %s' % num
hello.cmdloop()
return
def worker2(num):
Gtk.main()
threads = []
t = threading.Thread(target=worker, args=(1,))
threads.append(t)
t2 = threading.Thread(target=worker2, args=(2,))
threads.append(t2)
if __name__ == '__main__':
#HelloWorld().cmdloop()
#Gtk.main()
t.start()
t2.start()
This works. What I'd like to know is this ok? Are there issues to look out for? This is my first time trying this so there are a lot of unknowns for me. I understand that both cmd, and gtk are blocking. The Gtk.main, and cmd loops work flawlessly so far. I'm just being cautious.
My first time using threading too. When cmd gets the command to setbutton the button is set. When the button is clicked 'something happened' prints. The command line continues as if nothing out of the ordinary happened. I was really surprised at how seamless it all works. Yet I am still a little worried.

GTK has its' own threading library, and you need to be careful I think with complex applications : http://faq.pygtk.org/index.py?req=show&file=faq20.006.htp - for instance when you have threads which update your GUI indepedently of the main thread.
In your example you do have a threaded application, although in fact your entire GTK application is running in a single thread - so you are ok.

GTK+ is not thread safe and there are a few problems with the example which may cause instability. The example loads GTK+ in the main thread, runs the GTK+ main loop in a different thread and creates GTK+ widgets in yet another thread. All the GTK+ API calls should occur in the main thread with other threads communicating back to the main GUI thread by adding idle or timer callbacks. Have a read through the PyGObject threading wiki [1].
https://wiki.gnome.org/Projects/PyGObject/Threading

GUI freezes after os.system call

My problem is with os.system. Until it finishes formatting, GUI freezes, and I can't fix it.
class ImageDialog(QtGui.QMainWindow):
def __init__(self):
QtGui.QDialog.__init__(self)
self.ui = uic.loadUi("Recursos/main.ui",self)
self.connect(self.ui.proteger_Button, QtCore.SIGNAL("clicked()"),self,
QtCore.SLOT("protec()"))
#QtCore.pyqtSlot()
def protec(self):
self.USB = "G:"
comando = "format %s /fs:ntfs /q /v:test /y" %(self.USB)
os.system(comando)`
if __name__ == '__main__':
app = QtGui.QApplication(sys.argv)
window = ImageDialog()
window.show()
sys.exit(app.exec_())

The simplest fix in your case is to add:
from threading import Thread
to your imports list, and then:
#QtCore.pyqtSlot()
def protec(self):
self.USB = "G:"
comando = "format %s /fs:ntfs /q /v:test /y" %(self.USB)
t = Thread(target = lambda: os.system(comando))
t.start()
This way the GUI thread will not get stuck waiting for the external process to finish.
You could probably remember the t's value and manage it in some way, so the number of threads running won't increase without limit, in case the external commands will hang, or run for a long time.
Note that it is not a good way to handle external processes. subprocess module is recommended for this.