I was going through this question : How do I return the definition of a class in python?
But I am unable to display the class definition. I am getting the below error:
>>> class A:
... pass
...
>>> import inspect
>>> source_text = inspect.getsource(A)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\**\Python\Python36\lib\inspect.py", line 968, in getsource
lines, lnum = getsourcelines(object)
File "C:\Users\**\Python\Python36\lib\inspect.py", line 955, in getsourcelines
lines, lnum = findsource(object)
File "C:\Users\**\Python\Python36\lib\inspect.py", line 768, in findsource
file = getsourcefile(object)
File "C:\Users\**\Python\Python36\lib\inspect.py", line 684, in getsourcefile
filename = getfile(object)
File "C:\Users\**\Python\Python36\lib\inspect.py", line 654, in getfile
raise TypeError('{!r} is a built-in class'.format(object))
TypeError: <module '__main__' (<_frozen_importlib_external.SourceFileLoader object at 0x0000026A79293F60>)> is a built-in class
>>>
Can someone please advise what am I doing wrong here? Thanks.
The inspect.getsource() function only works if there is a text file available to load the source code.
You typed the definition of the class into the interactive interpreter, which doesn't keep the original source around when compiling that source into in-memory class and code objects.
Put your class definition into a module, import the module, and then use inspect.getsource().
inspect.getsource() works by first finding the module for a given object (for classes, by looking at the ClassObj.__module__ attribute for the module name, then getting the module via sys.modules[modulename]) then seeing if the module has a __file__ attribute from which a readable source file can be determined. If there is such a filename and it can be read, then the inspect module reads that file to then search for the class ClassName: line and giving you all lines from that point on with same or deeper indentation. The interactive interpreter executes everything in the __main__ module and there is no __file__ attribute for the interpreter, so any attempts at loading source code for objects defined there will simply fail.
If you just wanted to know what members the class defines, use dir() or help() on the object instead. You don't need to see the full source code for that information.
Related
I am having a super weird error, the situation is the following:
I am writing some code in a package called hsltools. The Module is called em_combiner1_4. I recently changed the version from 1_3, and also changed all imports accordingly.
If I run my code now, I get the following error:
Traceback (most recent call last):
File "C:\Users\...\git_home\hsltools\hsltools\tests\emc_tester\EMC_tester.py", line 85, in <module>
combine_2d()
File "C:\Users\...\git_home\hsltools\hsltools\tests\emc_tester\EMC_tester.py", line 48, in combine_2d
elec_field2d = emc.EM(import_path+"2DelecMap.ef2.npz")
File "c:\users\...\git_home\hsltools\hsltools\em_combiner1_4.py", line 441, in __init__
load_npz()
File "c:\users\...\git_home\hsltools\hsltools\em_combiner1_4.py", line 251, in load_npz
self.em_type = data["metadata"][1]
File "C:\Users\...\Anaconda3\lib\site-packages\numpy\lib\npyio.py", line 254, in __getitem__
return format.read_array(bytes,
File "C:\Users\...\Anaconda3\lib\site-packages\numpy\lib\format.py", line 744, in read_array
array = pickle.load(fp, **pickle_kwargs)
ModuleNotFoundError: No module named 'hsltools.em_combiner1_3
The thing I don't get is the location of the error: ...\numpy\lib\format.py.
There is absolutaly no import there, and anyway, numpy would not import my module.
I am wondering if I am generally misunderstanding some basic concept, or if the error is wrong.
It's happening while deserializing pickled objects. Pickle stores the type information including the module name. When loading a pickled object, the module is imported to construct the final object.
Your pickle file contains an object that is part of hsltools.em_combiner1_3. So that is the module pickle tries to find.
I tried this code-snippet inside a script named test.py :
from inspect import *
def f1(p,r):
"""Return f1 score from p,r"""
return 2*p*r/(p+r)
print(getsourcelines(f1))
If I run this from terminal with python3 test.py, it outputs the following :
(['def f1(p,r):\n', '\t"""Return f1 score from p,r"""\n', '\treturn 2*p*r/(p+r)\n'], 3)
But, if I run the same whole script line by line inside python shell, it throws a OSError. This is what I tried in python shell along with the error :
>>> from inspect import *
>>>
>>> def f1(p,r):
... """Return f1 score from p,r"""
... return 2*p*r/(p+r)
...
>>> print(getsourcelines(f1))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.6/inspect.py", line 955, in getsourcelines
lines, lnum = findsource(object)
File "/usr/lib/python3.6/inspect.py", line 786, in findsource
raise OSError('could not get source code')
OSError: could not get source code
>>>
Why does inspect.getsourcelines(f1) throws error inside python shell, but not when it is run form the file? Is there any other way to get the source lines of a function declared inside a python shell?
This is the expected behaviour. inspect has only a limited support for builtin objects (not loaded from files).
It is explicit in other functions like getsourcefile where the doc says:
This will fail with a TypeError if the object is a built-in module, class, or function.
Either if less explicit, the doc for getsourcelines says (emphasize mine):
The source code is returned as a list of the lines corresponding to the object and the line number indicates where in the original source file the first line of code was found. An OSError is raised if the source code cannot be retrieved.
In the current versions, getsourcelines try to locate the function in the current source file. As it cannot get the current source file for the functions declared outside of a file, it raises an exception.
The underlying cause, is that when python is started in interactive mode, the main module is a builtin module and has no __file__ attribute.
Why can't I get the definition of Callable from module collections in the following code?
How can I get the definition of a class in a module? Thanks.
>>> from collections import Callable
>>> inspect.getsource(Callable)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.5/inspect.py", line 944, in getsource
lines, lnum = getsourcelines(object)
File "/usr/lib/python3.5/inspect.py", line 931, in getsourcelines
lines, lnum = findsource(object)
File "/usr/lib/python3.5/inspect.py", line 788, in findsource
raise OSError('could not find class definition')
OSError: could not find class definition
>>> inspect.getsourcelines(Callable)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.5/inspect.py", line 931, in getsourcelines
lines, lnum = findsource(object)
File "/usr/lib/python3.5/inspect.py", line 788, in findsource
raise OSError('could not find class definition')
OSError: could not find class definition
>>> inspect.getmodule(Callable)
<module 'collections.abc' from '/usr/lib/python3.5/collections/abc.py'>
>>> inspect.getfile(Callable)
'/usr/lib/python3.5/collections/abc.py'
>>> inspect.getsourcefile(Callable)
'/usr/lib/python3.5/collections/abc.py'
In general, this is easily done with inspect.getsource which accepts a module, a class, a method, a function, a traceback, a frame,
or a code object. The source they represent should of course be written in Python else an error is raised.
In this specific case, you just happen to be unlucky because while Callable is defined in _collections_abc the Callable.__module__ name is callections.abc:
>>> Callable.__module__
'collections.abc'
This throws getsource off because it won't look in _collections_abc that contains Callables definition but, instead, in collections.abc which merely imports all definitions from _collections_abc:
>>> print(getsource(collections.abc))
from _collections_abc import *
from _collections_abc import __all__
Normally, getsource doesn't have an issue finding the source, for example, on itself:
>>> print(getsource(getsource))
def getsource(object):
"""Return the text of the source code for an object.
The argument may be a module, class, method, function, traceback, frame,
or code object. The source code is returned as a single string. An
OSError is raised if the source code cannot be retrieved."""
lines, lnum = getsourcelines(object)
return ''.join(lines)
In this specific case, though, it does (due to Callable.__module__ returning collections.abc.) You could replace __module__ with '_collections_abc' for a tricky way to see the source:
>>> Callable.__module__ = '_collections_abc'
>>> src = getsource(Callable)
>>> print(src)
class Callable(metaclass=ABCMeta):
__slots__ = ()
#abstractmethod
def __call__(self, *args, **kwds):
return False
#classmethod
def __subclasshook__(cls, C):
if cls is Callable:
return _check_methods(C, "__call__")
return NotImplemented
but this doesn't make me feel very comfortable.
get_source(fullname)
Return the source code for the specified module.
So you should return the whole module, as Callable is defined in the module of _collections_abc, so your code should be this:
import _collections_abc
import inspect
print(inspect.getsource(_collections_abc))
and you can see the definition of Callable in the print result.
How to get module source code by a python 'object' of that module?
class TestClass(object):
def __init__(self):
pass
def testMethod(self):
print 'abc'
return 'abc'
It's well-known that
print inspect.getsource(TestClass)
can be used to get source code of 'TestClass'.
However, the result of
ob = TestClass()
print inspect.getsource(ob)
as below, is not as expected.
Traceback (most recent call last):
File "D:\Workspaces\WS1\SomeProject\src\python\utils\ModuleUtils.py", line 154, in <module>
print inspect.getsource(ob)
File "C:\SciSoft\WinPython-64bit-2.7.10.3\python-2.7.10.amd64\lib\inspect.py", line 701, in getsource
lines, lnum = getsourcelines(object)
File "C:\SciSoft\WinPython-64bit-2.7.10.3\python-2.7.10.amd64\lib\inspect.py", line 690, in getsourcelines
lines, lnum = findsource(object)
File "C:\SciSoft\WinPython-64bit-2.7.10.3\python-2.7.10.amd64\lib\inspect.py", line 526, in findsource
file = getfile(object)
File "C:\SciSoft\WinPython-64bit-2.7.10.3\python-2.7.10.amd64\lib\inspect.py", line 420, in getfile
'function, traceback, frame, or code object'.format(object))
TypeError: <utils.TestClass.TestClass object at 0x0000000003C337B8> is not a module, class, method, function, traceback, frame, or code object
The question is:
If there is an object like 'ob' above, how to check the module source code of ob, or 'TestClass', via a method that takes 'ob' itself as a parameter?
In short, implement the following module
def getSource(obj):
###returns the result which is exactly identical to inspect.getsource(TestClass)
ob = TestClass()
###prints the result which is exactly identical to inspect.getsource(TestClass)
print getSource(ob)
(The scenario of a method like this is much more common than inspect.getsource(); for example, check the source code of an unknown, unpickled object.)
Instances don't have source code.
Use:
print inspect.getsource(type(ob))
or:
print inspect.getsource(ob.__class__)
I have a problem with pickle.load() from a file. Dump and load is done in dill_read_write.py:
dill_read_write.py
import os
import dill
from contact_geometry import ContactGeometry
def write_pickle(obj, filename):
os.chdir(os.path.abspath(os.path.join(os.path.dirname(__file__))))
filename = os.path.join(os.getcwd(), filename)
with open(filename, 'wb') as output_:
dill.dump(obj, output_)
def read_pickle(filename):
with open(filename, 'rb') as input_:
return dill.load(input_)
if __name__ == "__main__":
read_pickle("ground_.pkl")
Saving object ContactGeometry data to pickle file is done when the PyQt application (project) is running. Function write() is called in moduleC.py:
moduleC.py
from contact_geometry import ContactGeometry
from moduleA.moduleB import dill_read_write
class Foo(FooItem):
def __init__(self,...):
...
def createGeometry(self):
contact_geometry_ = ContactGeometry()
# save object to pickle file
dill_read_write.write_pickle(contact_geometry_, "object_data.pkl")
The object is saved and pickle file is created. But when I run only the file dill_read_write.py to read (load) object data from pickle file I get the following error:
Traceback (most recent call last):
File "C:\projectName\moduleA\moduleB\dill_read_write.py", line 29, in <module>
read("ground_.pkl")
File "C:\projectName\moduleA\moduleB\dill_read_write.py", line 24, in read
return dill.load(input_)
File "C:\Python27\lib\site-packages\dill-0.2.2-py2.7.egg\dill\dill.py", line 199, in load
obj = pik.load()
File "C:\Python27\Lib\pickle.py", line 858, in load
dispatch[key](self)
File "C:\Python27\Lib\pickle.py", line 1090, in load_global
klass = self.find_class(module, name)
File "C:\Python27\lib\site-packages\dill-0.2.2-py2.7.egg\dill\dill.py", line 278, in find_class
return StockUnpickler.find_class(self, module, name)
File "C:\Python27\Lib\pickle.py", line 1124, in find_class
__import__(module)
ImportError: No module named moduleA.moduleB.contact_geometry
I searched a bit and found that dill can perform better than pickle with classes but I am having problems to implement it. I've also found that I have to implement __reduce__() in class ContactGeometry in file contact_geometry.py.
contact_geometry.py
class ContactGeometry(object):
def __init__(self, ...):
...
def __reduce__(self):
return (self.__class__, (os.path.realpath(__file__))
But I am not sure what should return this method? How could I successfully load pickle file from the current situation?
Below is the project structure, if it is any help.
I'm the dill author. It's hard to tell how you are running the code, but it looks like the issue is that one way you are running the code and the module name as in #Antti Haapala's answer. His suggestions are also good ones to follow.
I'll add this… You need to make sure that (1) moduleA.moduleB.contact_geometry is on the PYTHONPATH, and (2) you are not dumping the module as __main__.moduleB.contact_geometry and trying to load it as moduleA.moduleB.contact_geometry -- dill treats __main__ as if it were a module (for the most part).
You shouldn't need to add __reduce__ methods to your classes, however.
You cannot run a python file from within a package like that; it wouldn't find the toplevel package names. I'd propose any of the following:
Write a start script in at the top level (where the main.py is), that imports and runs the read_write_dill from moduleA.moduleB
Instead in the top level directory, where the main is, you can run
that module with python -m moduleA.moduleB.dill_read_write.
Or, my preferred alternative, write a setup.py for your project and write a script for that utility.