I have a module I'm writing that uses matplotlib. However, I need it to work with a display(e.g. command-line execution), or when theres no display ( SGE/qsub cluster job, or a cron job).
I found this answer ( Automatic detection of display availability with matplotlib ), which works for the command-line execution and within a cluster job. But, in a cron job, it fails to properly import matplotlib, and seems to throw an exception before I import it.
In the cronjob, I try to import my module with the following script:
import os
os.environ['PYTHONPATH'] = 'path/to/my/module/dir:%s' % os.environ['PYTHONPATH']
print 'now loading mymodule...'
import mymodule
and mymodule.py just has the code from the earlier linked answer:
#!/usr/bin/python
import os
print 'testing import method...'
import matplotlib
r = 0
try :
r = os.system('python -c "import matplotlib.pyplot as plt;plt.figure()"')
except RuntimeError :
pass
print 'r=%d' % r
if r != 0:
print 'matplotlib: running in cluster, using Agg!'
matplotlib.use('Agg')
import matplotlib.pyplot as plt
I get this error message from the cron service:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/afs/ifh.de/user/u/user/.local/lib/python2.6/site-packages/matplotlib/pyplot.py", line 109, in <module>
_backend_mod, new_figure_manager, draw_if_interactive, _show = pylab_setup()
File "/afs/ifh.de/user/u/user/.local/lib/python2.6/site-packages/matplotlib/backends/__init__.py", line 32, in pylab_setup
globals(),locals(),[backend_name],0)
File "/afs/ifh.de/user/u/user/.local/lib/python2.6/site-packages/matplotlib/backends/backend_gtkagg.py", line 14, in <module>
from matplotlib.backends.backend_gtk import gtk, FigureManagerGTK, FigureCanvasGTK,\
File "/afs/ifh.de/user/u/user/.local/lib/python2.6/site-packages/matplotlib/backends/backend_gtk.py", line 16, in <module>
import gtk; gdk = gtk.gdk
File "/usr/lib64/python2.6/site-packages/gtk-2.0/gtk/__init__.py", line 64, in <module>
_init()
File "/usr/lib64/python2.6/site-packages/gtk-2.0/gtk/__init__.py", line 52, in _init
_gtk.init_check()
RuntimeError: could not open display
now loading mymodule...
testing import method...
r=256
matplotlib: running in cluster, using Agg!
I think its trying and failing to import matplotlib, which is throwing a RuntimeError, before getting to my code in mymodule.py . I can see that it gets to my code and runs it, but the Traceback message makes me nervous.
Why is it loading matplotlib(and crashing) before I tell it to(with Agg)? How can I fix/hide that traceback message?
As a dirty trick to hide the traceback message, just pipe all errors in the cronjob to /dev/null:
script.py 2> /dev/null
Related
i tried to run my Python Program on my Linux maschine but when i try to run it it is having a hard time finding all the nessasary modules needed for it to work.
i installed CanOpen to my project
import threading
import sys
sys.path.append('venv/Lib/site-packages')
sys.path.append('venv/Lib/site-packages/can/interfaces/kvaser')
sys.path.append('/test/Python39/Lib')
import canopen
from canopen.network import MessageListener, NodeScanner
from canopen.nmt import NmtMaster
from canopen.sync import SyncProducer
from canopen.timestamp import TimeProducer
if __name__ == '__main__':
network = canopen.Network()
network.connect(bustype='kvaser', channel=0, bitrate=250000)
it is showing me following error in the Terminal:
Kvaser canlib is unavailable.
Traceback (most recent call last):
File "/test/Can_T1/main.py", line 60, in <module>
network.connect(bustype='kvaser', channel=0, bitrate=250000)
File "/test/Can_T1/venv/Lib/site-packages/canopen/network.py", line 112, in connect
self.bus = can.interface.Bus(*args, **kwargs)
File "/test/Can_T1/venv/Lib/site-packages/can/interface.py", line 120, in __new__
bus = cls(channel, *args, **kwargs)
File "/test/Can_T1/venv/Lib/site-packages/can/interfaces/kvaser/canlib.py", line 437, in __init__
canGetNumberOfChannels(ctypes.byref(num_channels))
NameError: name 'canGetNumberOfChannels' is not defined
Is there any way to define 'canGetNumberOfChannels'?
You have to install the canlib on your machine as well. Right now, you just have installed the SDK. If you are running on linux you have to install libcanlib.so on your machine. Take a look at the part that is related to your operating system on this link: https://pypi.org/project/canlib/
I'm trying to import and use matplotlib.pyplot in a script and I'm getting the following error:
Traceback (most recent call last):
File "<pyshell#18>", line 1, in <module>
from matplotlib import pyplot as plt
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/matplotlib/pyplot.py", line 36, in <module>
from matplotlib.figure import Figure, figaspect
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/matplotlib/figure.py", line 40, in <module>
from matplotlib.axes import Axes, SubplotBase, subplot_class_factory
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/matplotlib/axes/__init__.py", line 4, in <module>
from ._subplots import *
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/matplotlib/axes/_subplots.py", line 10, in <module>
from matplotlib.axes._axes import Axes
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/matplotlib/axes/_axes.py", line 22, in <module>
import matplotlib.dates as _ # <-registers a date unit converter
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/matplotlib/dates.py", line 126, in <module>
from dateutil.rrule import (rrule, MO, TU, WE, TH, FR, SA, SU, YEARLY,
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/dateutil/rrule.py", line 13, in <module>
from fractions import gcd
ImportError: cannot import name 'gcd'
The weird part is that I can import it just fine if I restart the Python shell and import it directly, but as soon as I try to run my script, even if I run the script and then import it after running the script to generate data, I get the error. The import line, if it matters, is always the same:
import matplotlib
from matplotlib import pyplot
My script is being run from a folder on my desktop, and I installed matplotlib in terminal with pip3.
Your file called fractions.py is shadowing the builtin module of the same name, causing problems when other libraries try to use that library. Name your file something else.
I have a script that starts like this with many imports:
from reportlab.graphics import shapes
from reportlab.lib.utils import ImageReader
from reportlab.graphics import barcode
from reportlab.lib.units import mm
from reportlab.pdfbase.pdfmetrics import stringWidth
import reportlab.rl_settings
import PIL
from cStringIO import StringIO
import labels
import pyodbc
import pandas
from os.path import expanduser
from time import sleep
import sys
I struggled massively with py2exe to even get an executable file. I finally managed it with the following setup script (most of which is just copy and pasted from similar issues and suggests on stackoverflow).
from distutils.core import setup
import distutils
import py2exe
import sys
import zmq
import os
sys.setrecursionlimit(5000)
distutils.core.setup(
options = {
"py2exe": {
"dll_excludes": ["MSVCP90.dll"]
}
},
)
sys.path.append('C:\\WINDOWS\\WinSxS\\x86_microsoft.vc90.crt_1fc8b3b9a1e18e3b_9.0.30729.4148_none_5090ab56bcba71c2')
packages=[
'reportlab',
'reportlab.graphics'
'reportlab.lib.utils'
'reportlab.rl_settings'
'reportlab.lib.units'
'reportlabl.pdfbase.pdfmetrics',
],
os.environ["PATH"] = \
os.environ["PATH"] + \
os.path.pathsep + os.path.split(zmq.__file__)[0]
setup( console=[{"script": "working.py"}],
options={
"py2exe": {
"includes":
["zmq.utils", "zmq.utils.jsonapi",
"zmq.utils.strtypes"] } } )
I am sure my script is inelegant. It has three connected define functions and a final output.
try:
makeyourlabels()
except:
Print "Sorry, something went wrong."
I get an error when I run the file:
Traceback (most recent call last):
File "working.py", line 3, in <module>
File "reportlab\graphics\barcode\__init__.pyc", line 72, in <module>
File "reportlab\graphics\barcode\__init__.pyc", line 42, in _reset
File "reportlab\graphics\barcode\widgets.pyc", line 162, in <module>
File "reportlab\graphics\barcode\widgets.pyc", line 95, in _BCW
File "reportlab\lib\utils.pyc", line 243, in rl_exec
File "<string>", line 1, in <module>
File "<string>", line 1, in <module>
ImportError: No module named common
If anyone can make any sense of all this and get my .exe running, I would be ever grateful!
As there is no testable code in the question I can not guarantee this will fix your issue but I can explain why you get this error. It is because py2exe didn't bundle reportlab.graphics.barcode.common into your .exe.
How do I know this, I followed the traceback to see what Reportlab was doing in the rl_exec call, it turns out it is make this call using exec:
from reportlab.graphics.barcode.common import I2of5
But because the import is only done dynamically py2exe doesn't know about the need for this package.
So how do you fix it? Simply add 'reportlab.graphics.barcode.common' to your package list, that should help py2exe locate the module it is looking for.
I have a python module and a unit test for it. When i run the unit test i get this:
flumotion
test
test_common_gstreamer ... [ERROR]
===============================================================================
[ERROR]
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/twisted/trial/runner.py", line 660, in loadByNames
things.append(self.findByName(name))
File "/usr/lib/python2.7/dist-packages/twisted/trial/runner.py", line 470, in findByName
return reflect.namedAny(name)
File "/usr/lib/python2.7/dist-packages/twisted/python/reflect.py", line 464, in namedAny
topLevelPackage = _importAndCheckStack(trialname)
File "/flumotion/flumotion/test/test_common_gstreamer.py", line 20, in <module>
from gi.repository import Gst
File "/usr/lib/python2.7/dist-packages/gi/__init__.py", line 23, in <module>
from ._gi import _API, Repository
exceptions.ImportError: could not import gobject (error was: ImportError('When using gi.repository you must not import static modules like "gobject". Please change all occurrences of "import gobject" to "from gi.repository import GObject".',))
flumotion.test.test_common_gstreamer
I know the error is self explanatory, but i can't find where the "import gobject" happens, so i've been advised to make an import hook that will print any time something is doing an import. I don't know where and how to implement the hook, i've only read something about import hooks here http://xion.org.pl/2012/05/06/hacking-python-imports/ .
Any help is appreciated
everyone,I am new to PP but got stuck in a problem when importing NumPy with PP.
Basically what I tried to do was submitting a function to the ppserver which depends on NumPy. I have imported it at the very beginning of the code using (import NumPy as nu) but when I ran the code, it gave me the error that cannot find the shared object multiarray.so.
The situation is exactly the same here: parallel python forum
the code is attached as below: (I am running on python 2.7.2 + pp 1.6.0 + numpy 1.5.1)
import numpy as nu
import pylab as pl
import pp
job_server = pp.Server(secret="123456")
print "Starting pp with", job_server.get_ncpus(), "workers"
aa = GrRib()
job = job_server.submit(aa.plotwavefunc, (band,k),(nu,pl,signal))
result = job()
the error looks like :
An error has occured during the function import
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.7/ppworker.py", line 86, in run
exec __fobj
File "<string>", line 127, in <module>
ImportError: Error importing numpy: you should not try to import numpy from
its source directory; please exit the numpy source tree, and relaunch
your python intepreter from there.
An error has occured during the function import
Traceback (most recent call last):
File "/usr/lib/pymodules/python2.7/ppworker.py", line 86, in run
exec __fobj
File "<string>", line 1, in <module>
File "/usr/share/pyshared/matplotlib/__init__.py", line 135, in <module>
from matplotlib.rcsetup import (defaultParams,
File "/usr/share/pyshared/matplotlib/rcsetup.py", line 19, in <module>
from matplotlib.colors import is_color_like
File "/usr/share/pyshared/matplotlib/colors.py", line 52, in <module>
import numpy as np
File "/usr/share/pyshared/numpy/__init__.py", line 136, in <module>
import add_newdocs
File "/usr/share/pyshared/numpy/add_newdocs.py", line 9, in <module>
from numpy.lib import add_newdoc
File "/usr/share/pyshared/numpy/lib/__init__.py", line 4, in <module>
from type_check import *
File "/usr/share/pyshared/numpy/lib/type_check.py", line 8, in <module>
import numpy.core.numeric as _nx
File "/usr/share/pyshared/numpy/core/__init__.py", line 5, in <module>
import multiarray
ImportError: No module named multiarray
Can anyone help me out? I understand it as I will have to change the directory PP is looking for pyshared objects.
I could be wrong, but I believe the way you submit the job is wrong. As seen on the documentation (http://www.parallelpython.com/content/view/15/30/#QUICKSMP), the third argument (not including "self") are dependent functions. Also, the modules have to be strings. I'm assuming the "band" and "k" are dependent functions since I do not see their declarations:
job = job_server.submit(aa.plotwavefunc, depfuncs = (band,k), modules = ("numpy","pylab","signal"))