We Keep Coding

kochat in use RuntimeError: main thread is not in main loop

kochat is a Korean chatbot, and I ran into a problem while practicing it.
github imformation : https://github.com/hyunwoongko/kochat
It is environment setting
python3.8
pip install kochat
JPype is reinstalled ues JPype1-1.2.0-cp38-cp38-win_amd64.whl <-download
pytorch is same ver with cuda 11.1
start code
### from kochat.proc import DistanceClassifier ###
from kochat.data import Dataset
from kochat.proc import GensimEmbedder, DistanceClassifier
from kochat.model import intent, embed
from kochat.loss import CenterLoss
dataset = Dataset(ood=True)
emb = GensimEmbedder(model=embed.Word2Vec())
# 프로세서 생성
clf = DistanceClassifier(
model=intent.CNN(dataset.intent_dict),
loss=CenterLoss(dataset.intent_dict)
)
# 되도록이면 DistanceClassifier는 Margin 기반의 Loss 함수를 이용해주세요
# 현재는 CenterLoss, COCOLoss, Cosface, GausianMixture 등의
# 거리기반 Metric Learning 전용 Loss함수를 지원합니다.
# 모델 학습
clf.fit(dataset.load_intent(emb))
# 모델 추론 (인텐트 분류)
clf.predict(dataset.load_predict("오늘 서울 날씨 어떨까", emb))
error message
C:\projectkyc\kochat3.8\Scripts\python.exe "C:/projectkyc/kochat3.8/test3.8/!from kochat.proc import DistanceClassifier.py"`enter code here`
Exception ignored in: <function Image.__del__ at 0x000001E7E1566D30>
Traceback (most recent call last):
File "c:\users\user\appdata\local\programs\python\python38\lib\tkinter\__init__.py", line 4014, in __del__
Exception ignored in: <function Variable.__del__ at 0x000001E7E154B430>
Traceback (most recent call last):
File "c:\users\user\appdata\local\programs\python\python38\lib\tkinter\__init__.py", line 351, in __del__
if self._tk.getboolean(self._tk.call("info", "exists", self._name)):
RuntimeError: main thread is not in main loop
Exception ignored in: <function Image.__del__ at 0x000001E7E1566D30>
Traceback (most recent call last):
File "c:\users\user\appdata\local\programs\python\python38\lib\tkinter\__init__.py", line 4014, in __del__
self.tk.call('image', 'delete', self.name)
RuntimeError: main thread is not in main loop
#
# A fatal error has been detected by the Java Runtime Environment:
#
# Internal Error (os_windows_x86.cpp:144), pid=11208, tid=0x00000000000037f4
# guarantee(result == EXCEPTION_CONTINUE_EXECUTION) failed: Unexpected result from topLevelExceptionFilter
#
# JRE version: Java(TM) SE Runtime Environment (8.0_281-b09) (build 1.8.0_281-b09)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.281-b09 mixed mode windows-amd64 compressed oops)
# Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# C:\projectkyc\kochat3.8\test3.8\hs_err_pid11208.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
Exception ignored in: <function Image.__del__ at 0x000001E7E1566D30>
Traceback (most recent call last):
File "c:\users\user\appdata\local\programs\python\python38\lib\tkinter\__init__.py", line 4014, in __del__
self.tk.call('image', 'delete', self.name)
RuntimeError: main thread is not in main loop
Tcl_AsyncDelete: async handler deleted by the wrong thread
Process finished with exit code 1
If I haven't posted it or if I have the information I need, I will add it immediately upon confirmation if requested.

This is such an annoying issue and to me setting matplotlib.use('Agg') does not cut it. I dont want to comprise on this.
The only thing that seems to do it somewhat consistently is having to close the figure before creating a new plot via plt.close(). This is annoying and if i forget it does this.

this error
Matplotlib - Tcl_AsyncDelete: async handler deleted by the wrong thread?
same Question But did not know where to apply
The places to apply are kochat\utils\visualizer.py and kochat\proc\utils\visualizer.py
to be. In the
from matplotlib import pyplot as plt
part of these two places,
import matplotlib
matplotlib.use('Agg')
from matplotlib import pyplot as plt
The problem is solved.

Traceback not showing in multithread context

At the moment, I heavily rely on traceback to check python errors (on my console). However, the traceback is not output on the console when the error happens in a thread different from the main one:
def test(n):
time.sleep(10-n)
print(n)
assert False
#assert False #if uncommented the console correctly show the traceback
with futures.ThreadPoolExecutor(4) as executor:
res=executor.map(test,range(10))
print("DONE finally")
If I uncomment the "assert false" the traceback returned is the following one:
Traceback (most recent call last):
File "C:/Users/massimo.bono/Documents/PythonWorkspace/AntaresPythonUtils/AntaresUtils/thread_try.py", line 14, in
assert False
AssertionError
If leave the comment, the output of the console is the following:
3
2
1
0
4
6
5
7
9
8
DONE finally
Process finished with exit code 0
Other infos useful to help me:
I'm using python 3.4.3;
my python console is the one embedded in PyCharm Community Edition 4.5.2
I'm on windows 8.1 64bit;
I tried to run the same python file from the windows powershell by entering in the python console, but the traceback was not displayed.

For what I understood it seems that the traceback of the threads still existed in the python runtime: they weren't simply printed on the stderr.
You can solve the issue by catching any exception in the method called by the executor, like this:
#this will be executed in a different thread. If any exception is caught
#it will be printed on the console
def thread_safe(n):
try:
test(n)
except Exception as e:
traceback.print_exc()
with futures.ThreadPoolExecutor(4) as executor:
res=executor.map(thread_safe,range(10))
print("DONE finally")

Getting "ImportError: No module named" with parallel python and methods in a package

I'm trying to use parallel python in order to do some distributed benchmarking (essentially, coordinate and run some code on a set of machines from a central server). The code I had was working perfectly fine until I moved the functionality to a separate package. From then on, I keep getting ImportError: No module named some.module.pp_test.
My question is actually two-fold: has anyone ever came across this problem with pp, and if yes, how to solve it? I tried using dill (import dill), but didn't help. Also, is there a good replacement for parallelpython, that doesn't require any additional infrastructure?
The exact error I get is:
RUNNING TEST
Waiting for hosts to finish booting....A fatal error has occured during the function execution
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/ppworker.py", line 86, in run
__args = pickle.loads(__sargs)
ImportError: No module named some.module.pp_test
Caught exception in the run phase 'NoneType' object is not iterable
Traceback (most recent call last):
File "test.py", line 5, in <module>
p.ping_pong()
File "/home/ubuntu/workspace/pp-test/some/module/pp_test.py", line 5, in ping_pong
a_test.run()
File "/home/ubuntu/workspace/pp-test/some/module/pp_test.py", line 27, in run
pong, hostname = ping()
TypeError: 'NoneType' object is not iterable
The code is structured this way:
pp-test/
test.py
some/
__init__.py
module/
__init__.py
pp_test.py
The test.py is implemented as:
from some.module.pp_test import MWE
p = MWE()
p.ping_pong()
While pp_test.py is:
class MWE():
def ping_pong(self):
print "RUNNING TEST "
a_test = PPTester()
a_test.run()
import pp
import time
from sys import stdout, exit
class PPTester(object):
def run(self):
try:
ppservers = ('10.10.10.10', )
time.sleep(5)
job_server = pp.Server(0, ppservers=ppservers)
stdout.write("Waiting for hosts to finish booting...")
while len(job_server.get_active_nodes()) - 1 < len(ppservers):
stdout.write(".")
stdout.flush()
time.sleep(1)
ppmodules = ()
pings = [(server, job_server.submit(self.run_pong, modules=ppmodules)) for server in ppservers]
for server, ping in pings:
pong, hostname = ping()
print "Host ", hostname, " is alive!"
print "All servers booted up, starting benchmarks..."
job_server.print_stats()
except Exception as e:
print "Caught exception in the run phase", e
raise
pass
def run_pong(self):
import subprocess
p = subprocess.Popen("hostname", stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
(output, err) = p.communicate()
p_status = p.wait()
return "pong ", output

dill won't work with pp out of the box, because pp doesn't serialize the python objects -- pp extracts the object's source code (like the inspect module in the standard python library).
To enable pp to use dill (actually dill.source, which is inspect augmented by dill), you have to use a fork of pp called ppft. ppft installs as pp (i.e. imports with import pp), but it has much stronger source inspection, so you can automatically "serialize" most python objects and have ppft track down their dependencies automatically.
Get ppft here: https://github.com/uqfoundation
ppft is also pip installable and python 3.x compatible.

Why doesn't sys.excepthook work?

Why isn't the sys.excepthook function called if I try to execute this code?
import sys;
def MyExcepthook(ex_cls, ex, tb):
print("Oops! There's an Error.\n");
a=open("./ERR.txt","w"); #Fixed as suggested by unutbu BUT the problem is the same!
a.write("Oops! There's an Error.\n");
a.close();
sys.excepthook = MyExcepthook;
def main():
print(1/0);
if (__name__=="__main__"):
main();
Output:
Traceback (most recent call last):
File "C:\Users\Path\to\my\python\file.py", line 13, in <module>
main();
File "C:\Users\Path\to\my\python\file.py", line 10, in main
print(1/0);
ZeroDivisionError: division by zero
Expected Output (by print):
Oops! There's an Error.
and a new file (Err.txt) should be created (by open)
The print function doesn't show the text and the file is not created because the sys.excepthook function is not called - why?
-->EDIT
My Problem is caused by a bug in idle-python 3.4 because now i tried to run the code by interpreter python (command line) and it works! this makes my Question useless if not to warn about this bug in idle-python 3.4 i'm sorry and thanks for your help!
[SOLUTION] if someone has my same problem => Try to Run your code by command line! and not from IDE.

Your custom excepthook must not itself raise an exception:
a=open("./ERR.txt") # opens the file in read mode
should be
a=open("./ERR.txt", 'w') # open the file in write mode.
When the custom excepthook raises an exception, you should see
something like
Oops! There's an Error.
Error in sys.excepthook:
...
IOError: [Errno 2] No such file or directory: './ERR.txt'
Original exception was:
...
ZeroDivisionError: integer division or modulo by zero
PS. Don't forget to delete all those unnecessary semicolons!
import sys
def my_excepthook(ex_cls, ex, tb):
msg = "Oops! There's an Error.\n"
print(msg)
with open("./ERR.txt", 'w') as a:
a.write(msg)
sys.excepthook = my_excepthook
def main():
print(1/0)
if __name__=="__main__":
main()

IronPython TypeError exception has no traceback

In my code I am having a TypeError exception that is crashing my code. In this one particular program (I created a test file to reproduce the error and couldn't) the traceback only says TypeError: 'int' object is unsubscriptable with no information about where it was occuring. I tried creating my own exception before the function call that this must be happening in by 1[0] and got the same problem. When I try causing a different kind of exception by foo_that_doesn't_exist() I get a proper traceback with where the error occurred. I'm running the IronPython interpreter and because of some dependencies on C# code I can't test with CPython.
EDIT: I tracked down the problem in my own code, and then found a way to reproduce it. The problem can be recreated with:
import traceback
import sys
try:
try:
1[0]
except:
raise
except:
traceback.print_tb(sys.exc_info()[2])
Which returns nothing while:
import traceback
import sys
try:
1[0]
except:
traceback.print_tb(sys.exc_info()[2])
returns
File "a.py", line 6, in <module>
1[0]