I am writing a Python script that uses 3rd party modules from GDAL. The GDAL functions do not raise exceptions when an error occurs, but do send messages to stdout. Generally, the errors that occur with GDAL functions do not warrant stopping the process and I don't need to know about the error having occurred.
Is there a way I can intercept the messages that are being sent to stdout before they are printed in the console? The GDAL messages interfere with the messages that I have provided in my own code.
As described in "Python Gotchas", you can turn on Exceptions using gdal.UseExceptions(), e.g.:
from osgeo import gdal
dsrc = gdal.Open('nonexist')
# ... silence
gdal.UseExceptions()
dsrc = gdal.Open('nonexist')
# Traceback (most recent call last):
# File "<interactive input>", line 1, in <module>
# RuntimeError: `nonexist' does not exist in the file system,
# and is not recognised as a supported dataset name.
You could always then use a try except block get the actual error message string:
try:
dsrc = gdal.Open('nonexist')
except RuntimeError as e:
print(str(e))
which will print the error message:
`nonexist' does not exist in the file system,
and is not recognised as a supported dataset name.
Related
kochat is a Korean chatbot, and I ran into a problem while practicing it.
github imformation : https://github.com/hyunwoongko/kochat
It is environment setting
python3.8
pip install kochat
JPype is reinstalled ues JPype1-1.2.0-cp38-cp38-win_amd64.whl <-download
pytorch is same ver with cuda 11.1
start code
### from kochat.proc import DistanceClassifier ###
from kochat.data import Dataset
from kochat.proc import GensimEmbedder, DistanceClassifier
from kochat.model import intent, embed
from kochat.loss import CenterLoss
dataset = Dataset(ood=True)
emb = GensimEmbedder(model=embed.Word2Vec())
# 프로세서 생성
clf = DistanceClassifier(
model=intent.CNN(dataset.intent_dict),
loss=CenterLoss(dataset.intent_dict)
)
# 되도록이면 DistanceClassifier는 Margin 기반의 Loss 함수를 이용해주세요
# 현재는 CenterLoss, COCOLoss, Cosface, GausianMixture 등의
# 거리기반 Metric Learning 전용 Loss함수를 지원합니다.
# 모델 학습
clf.fit(dataset.load_intent(emb))
# 모델 추론 (인텐트 분류)
clf.predict(dataset.load_predict("오늘 서울 날씨 어떨까", emb))
error message
C:\projectkyc\kochat3.8\Scripts\python.exe "C:/projectkyc/kochat3.8/test3.8/!from kochat.proc import DistanceClassifier.py"`enter code here`
Exception ignored in: <function Image.__del__ at 0x000001E7E1566D30>
Traceback (most recent call last):
File "c:\users\user\appdata\local\programs\python\python38\lib\tkinter\__init__.py", line 4014, in __del__
Exception ignored in: <function Variable.__del__ at 0x000001E7E154B430>
Traceback (most recent call last):
File "c:\users\user\appdata\local\programs\python\python38\lib\tkinter\__init__.py", line 351, in __del__
if self._tk.getboolean(self._tk.call("info", "exists", self._name)):
RuntimeError: main thread is not in main loop
Exception ignored in: <function Image.__del__ at 0x000001E7E1566D30>
Traceback (most recent call last):
File "c:\users\user\appdata\local\programs\python\python38\lib\tkinter\__init__.py", line 4014, in __del__
self.tk.call('image', 'delete', self.name)
RuntimeError: main thread is not in main loop
#
# A fatal error has been detected by the Java Runtime Environment:
#
# Internal Error (os_windows_x86.cpp:144), pid=11208, tid=0x00000000000037f4
# guarantee(result == EXCEPTION_CONTINUE_EXECUTION) failed: Unexpected result from topLevelExceptionFilter
#
# JRE version: Java(TM) SE Runtime Environment (8.0_281-b09) (build 1.8.0_281-b09)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.281-b09 mixed mode windows-amd64 compressed oops)
# Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# C:\projectkyc\kochat3.8\test3.8\hs_err_pid11208.log
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
Exception ignored in: <function Image.__del__ at 0x000001E7E1566D30>
Traceback (most recent call last):
File "c:\users\user\appdata\local\programs\python\python38\lib\tkinter\__init__.py", line 4014, in __del__
self.tk.call('image', 'delete', self.name)
RuntimeError: main thread is not in main loop
Tcl_AsyncDelete: async handler deleted by the wrong thread
Process finished with exit code 1
If I haven't posted it or if I have the information I need, I will add it immediately upon confirmation if requested.
This is such an annoying issue and to me setting matplotlib.use('Agg') does not cut it. I dont want to comprise on this.
The only thing that seems to do it somewhat consistently is having to close the figure before creating a new plot via plt.close(). This is annoying and if i forget it does this.
this error
Matplotlib - Tcl_AsyncDelete: async handler deleted by the wrong thread?
same Question But did not know where to apply
The places to apply are kochat\utils\visualizer.py and kochat\proc\utils\visualizer.py
to be. In the
from matplotlib import pyplot as plt
part of these two places,
import matplotlib
matplotlib.use('Agg')
from matplotlib import pyplot as plt
The problem is solved.
I'm trying to use parallel python in order to do some distributed benchmarking (essentially, coordinate and run some code on a set of machines from a central server). The code I had was working perfectly fine until I moved the functionality to a separate package. From then on, I keep getting ImportError: No module named some.module.pp_test.
My question is actually two-fold: has anyone ever came across this problem with pp, and if yes, how to solve it? I tried using dill (import dill), but didn't help. Also, is there a good replacement for parallelpython, that doesn't require any additional infrastructure?
The exact error I get is:
RUNNING TEST
Waiting for hosts to finish booting....A fatal error has occured during the function execution
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/ppworker.py", line 86, in run
__args = pickle.loads(__sargs)
ImportError: No module named some.module.pp_test
Caught exception in the run phase 'NoneType' object is not iterable
Traceback (most recent call last):
File "test.py", line 5, in <module>
p.ping_pong()
File "/home/ubuntu/workspace/pp-test/some/module/pp_test.py", line 5, in ping_pong
a_test.run()
File "/home/ubuntu/workspace/pp-test/some/module/pp_test.py", line 27, in run
pong, hostname = ping()
TypeError: 'NoneType' object is not iterable
The code is structured this way:
pp-test/
test.py
some/
__init__.py
module/
__init__.py
pp_test.py
The test.py is implemented as:
from some.module.pp_test import MWE
p = MWE()
p.ping_pong()
While pp_test.py is:
class MWE():
def ping_pong(self):
print "RUNNING TEST "
a_test = PPTester()
a_test.run()
import pp
import time
from sys import stdout, exit
class PPTester(object):
def run(self):
try:
ppservers = ('10.10.10.10', )
time.sleep(5)
job_server = pp.Server(0, ppservers=ppservers)
stdout.write("Waiting for hosts to finish booting...")
while len(job_server.get_active_nodes()) - 1 < len(ppservers):
stdout.write(".")
stdout.flush()
time.sleep(1)
ppmodules = ()
pings = [(server, job_server.submit(self.run_pong, modules=ppmodules)) for server in ppservers]
for server, ping in pings:
pong, hostname = ping()
print "Host ", hostname, " is alive!"
print "All servers booted up, starting benchmarks..."
job_server.print_stats()
except Exception as e:
print "Caught exception in the run phase", e
raise
pass
def run_pong(self):
import subprocess
p = subprocess.Popen("hostname", stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
(output, err) = p.communicate()
p_status = p.wait()
return "pong ", output
dill won't work with pp out of the box, because pp doesn't serialize the python objects -- pp extracts the object's source code (like the inspect module in the standard python library).
To enable pp to use dill (actually dill.source, which is inspect augmented by dill), you have to use a fork of pp called ppft. ppft installs as pp (i.e. imports with import pp), but it has much stronger source inspection, so you can automatically "serialize" most python objects and have ppft track down their dependencies automatically.
Get ppft here: https://github.com/uqfoundation
ppft is also pip installable and python 3.x compatible.
In my code I am having a TypeError exception that is crashing my code. In this one particular program (I created a test file to reproduce the error and couldn't) the traceback only says TypeError: 'int' object is unsubscriptable with no information about where it was occuring. I tried creating my own exception before the function call that this must be happening in by 1[0] and got the same problem. When I try causing a different kind of exception by foo_that_doesn't_exist() I get a proper traceback with where the error occurred. I'm running the IronPython interpreter and because of some dependencies on C# code I can't test with CPython.
EDIT: I tracked down the problem in my own code, and then found a way to reproduce it. The problem can be recreated with:
import traceback
import sys
try:
try:
1[0]
except:
raise
except:
traceback.print_tb(sys.exc_info()[2])
Which returns nothing while:
import traceback
import sys
try:
1[0]
except:
traceback.print_tb(sys.exc_info()[2])
returns
File "a.py", line 6, in <module>
1[0]
I am building or trying to build a python script which check's a list of ip addresses (ips.txt) for a specific program using the wmi python module. However, no matter how I handle the exceptions on assets with no RPC service running the script stops running on an error. I am using python 2.7.5
Can I catch and pass the error's to proceed?
Can I catch the error and print or return a note that the ip was not alive or rpc was not running?
Thank you in advance
Here is my code:
import wmi
list = open("ips.txt")
for line in list.readlines():
asset = line.strip('\n')
c = wmi.WMI(asset)
try:
for process in c.Win32_Process (name="SbClientManager.exe"):
print asset, process.ProcessId, process.Name
except Exception:
pass
I have tried handling the exceptions in multiple way's to continue parsing my list, but the script continues to error out with the following:
Traceback (most recent call last):
File ".\check_service.py", line 12, in <module>
c = wmi.WMI(asset)
File "C:\Python27\lib\site-packages\wmi.py", line 1290, in connect
handle_com_error ()
File "C:\Python27\lib\site-packages\wmi.py", line 241, in handle_com_error
raise klass (com_error=err)
wmi.x_wmi: <x_wmi: Unexpected COM Error (-2147023174, 'The RPC server is unavailable.', None, None)>
Ultimately, I am just trying to continue the script and catch the error. Maybe a note stating that IP was not responsive would be helpful. Here are the exceptions samples that I have tried:
except Exception:
sys.exc_clear()
except:
pass
except wmi.x_wmi, x:
pass
The traceback you pasted says that the error is in the c = wmi.WMI(asset) line. You need to put that line inside the try block.
Like so:
import wmi
list = open("ips.txt")
bad_assets = []
for line in list.readlines():
asset = line.strip('\n')
try:
c = wmi.WMI(asset)
for process in c.Win32_Process (name="SbClientManager.exe"):
print asset, process.ProcessId, process.Name
except Exception:
bad_assets.append(asset)
Also, trying to catch the right exception is recommended.
I must be doing something wrong that I can't figure out from other resources on teh web.
At one point in my script I try to run:
username= win32security.LookupAccountSid(None, ace[2])[0]
From which I get and error
Traceback (most recent call last):
File "<pyshell#10>", line 1, in <module>
win32security.LookupAccountSid(None, ace[2])[0]
error: (1332, 'LookupAccountSid', 'No mapping between account names and security IDs was done.')
I figured out this happens when you have a security registry for which there no longer is a user so it shows up on the permissions as S-1-xxx...
In order to get the script to move on I wrote:
try:
username= win32security.LookupAccountSid(None, ace[2])[0]
except:
nouser +=1
This code will still give the error and stop my script. How can I get my script to ignore when this error happens?