def myfn():
class MyClass:
pass
return MyClass()
a = myfn()
b = myfn()
print(type(a) is type(b))
Here we can see that type(a) is not type(b). Is this always guaranteed to be the case? Why doesn't the interpreter optimise this since the definition of MyClass doesn't depend on any parameters passed to myfn?
The class statement, when executed always creates a new class object. Classes are not singletons. By putting the class statement in a function just lets you execute it more than once.
Class statements at the module level are executed just once because modules are executed just once, on first import.
You could bypass this by deleting the module object from the sys.modules structure; you'll note that the Foo class imported the third time is a different object after we removed the module:
>>> with open('demomodule.py', 'w') as demomodule:
... demomodule.write('class Foo: pass\n')
...
16
>>> import sys
>>> from demomodule import Foo # first import
>>> id(Foo)
140579578254536
>>> import demomodule # just another reference, module is not run again
>>> id(demomodule.Foo)
140579578254536
>>> del sys.modules['demomodule'] # removing the module object
>>> import demomodule # this causes it to be imported again
>>> id(demomodule.Foo)
140579574812488
The same can happen when you run a module as script then import the same module with import; scripts are run as the __main__ module, using import to import the script again then also creates a separate module object for the imported name:
$ echo 'class Foo: pass
> import demomodule
> print(__name__, id(Foo), id(demomodule.Foo))
> ' > demomodule.py
$ python demomodule.py
demomodule 140718182184264 140718182184264
__main__ 140718182074440 140718182184264
Python is highly dynamic in nature; applying optimisations such as caching a class object produced by a function are fraught with problems. Your function might not take any parameters, but it is not operating in a vacuum. For example, I could replace the __build_class__ hook function and insert an extra class into the bases of any class created anywhere in Python:
>>> def foo_class():
... class Foo: pass
... return Foo
...
>>> foo_class().__mro__
(<class '__main__.foo_class.<locals>.Foo'>, <class 'object'>)
>>> import builtins
>>> class Bar: pass
>>> orig_buildclass = builtins.__build_class__
>>> def my_buildclass(f, name, *bases, **kwargs):
... return orig_buildclass(f, name, *((Bar,) + bases), **kwargs)
...
>>> builtins.__build_class__ = my_buildclass
>>> foo_class().__mro__
(<class '__main__.foo_class.<locals>.Foo'>, <class '__main__.Bar'>, <class 'object'>)
Python is full of hooks like these.
Related
I have a function that uses typing.get_type_hints. I want to add a documentation test to it. However, it looks like get_type_hints fails to resolve types that are defined in a doctest.
Here is a simplified example:
import typing
def f(clazz):
"""
>>> class MyClass:
... my_field: 'MyClass'
>>> f(MyClass)
"""
typing.get_type_hints(clazz)
When running it with python3 -m doctest test.py it throws NameError: name 'MyClass' is not defined.
In order to get it to work in doctest, you would need to provide the correct evaluation scope.
Try this:
import typing
def f(clazz, globalns=None, localns=None):
"""
>>> class MyClass:
... my_field: 'MyClass'
>>> f(MyClass, globals(), locals())
"""
typing.get_type_hints(clazz, globalns, localns)
In doctest, a special set of values are used in the "eval scope" that happens with get_typing_hints.
It is looking for "test.MyClass" which doesn't actually exist otherwise.
from __future__ import annotations
import typing
def f(clazz):
"""
>>> test = 1
>>> class MyClass:
... my_field:'MyClass'
>>> f(MyClass)
"""
typing.get_type_hints(clazz)
add from __future__ import annotations at the beginning of the file, it work for me on python3.7
Structure:
package/
m1.py
m2.py
m1.py:
class A:
pass
if __name__ == '__main__':
from m2 import B
print(issubclass(B, A))
m2.py:
from m1 import A
class B(A):
pass
I don't now why I get false while I think it's obviously true when I run m1.py. My python version is python3.5.2.
Wellcome to the world of modules and namespaces!
Here is what happens:
In module m2, you import A from module m1. So you create a class m2.A as a reference to class m1.A. It happens to have the same definition as __main__.A, but they are different objects, because the main module is named __main__ and not m1. Then in module __main__ you create a class __main__.B as a reference to class m2.B
To better understand what happens here, I have added some code to m1:
...
print(issubclass(B, A))
import m1
import m2
print(A == m1.A, m1.A == m2.A)
print(B == m2.B)
print(issubclass(B, m2.A), issubclass(B, m1.A))
The output is:
False
False True
True
True True
Proving that B is indeed a subclass of m1.A but not of __main__.A.
You have to derive class A from object to get issubclass do its job:
isinstance() and issubclass() behavior differently
class A(object): ...
Here an example of python cli:
>>> class A(object): pass
>>> class B(A): pass
>>> issubclass(B,A)
True
You create 2 class objects, derived from the same class. That is: the class code is executed twice and thus results in 2 objects. And those are not the same. Example: see the output of the print statements below
# m2.py
from m1 import A
class B(A):
pass
print(A)
# m1.py
class A:
pass
if __name__ == '__main__':
from m2 import B
print(A)
print(issubclass(B, A))
# Output
#<class 'm1.A'>
#<class '__main__.A'>
#False
Also: see this answer for more information.
IN m1.py you are providing class A and in m2.py you are importing class of A. Even though it is imported from m1.py it is not the same object. You never actually import the exact object you just import the code and a sperate object is created. So instead of importing class A you just had two separate class A() code blocks would you expect them to be a subclass of each other just because they have the same name. You are not checking if the class has the same name you are checking if the class is an subclass of an actual class object (not just the text of the name).
It would be no different than importing class A from a 3rd completely different module because you are just importing code not objects.
Why does the following code not work?
>>> from mock import *
>>> class A(object):
... pass
...
>>> mock = create_autospec(A)
>>> mock.foo = Mock() # this works
>>> mock.__bar__ = Mock() # this works too
>>> mock.__subclasses__ = Mock() # this fails
AttributeError: Mock object has no attribute '__subclasses__'
I think I'm following the documentation on mocking magic methods here. The docs do remark that trying to mock a magic method which is not in the spec will not work. But why would __subclasses__ not be in the spec of an autospecced new-style class?
__subclasses__ is not part of the class spec. It is part of the metatype of the class (type here).
Python always looks up special methods on the type, never directly. If and when Python needs to call __subclasses__, it'll not do so directly, it'll use type(classobj).__subclasses__(classobj) to look up the unbound method and pass in the first argument manually. As such, adding __subclasses__ to the mock of a class is not going to be enough.
The same applies to special methods intended to operate on instances; Mock will happily take __add__ or __str__ along when using create_autospec(), and that works then for instances of the mocked class where Python will use type(mockinstance).__str__(mockinstance) to invoke the __str__ method.
If your own code calls classobj.__subclasses__() directly, you'll need to pass in a class mock that explicitly sets that method; you can't expect it to be auto-specced here.
Mocks don't like you setting any valid magic method:
>>> m = create_autospec(A)
>>> m.__add__ = Mock()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/mj/Development/venvs/stackoverflow-2.7/lib/python2.7/site-packages/mock.py", line 767, in __setattr__
raise AttributeError("Mock object has no attribute '%s'" % name)
AttributeError: Mock object has no attribute '__add__'
only non-default magic methods are allowed:
>>> m.__dummy__ = Mock()
>>> m.__dummy__
<Mock name='mock.__dummy__' id='4427608656'>
You can create a subclass of A adding in the __subclass__ method to allow you to mock it:
>>> class AMockSpec(A):
... def __subclasses__(self): pass
...
>>> m = create_autospec(AMockSpec)
>>> m.__subclasses__ = Mock()
>>> m.__subclasses__.return_value = ['SomeMockValue']
>>> m.__subclasses__()
['SomeMockValue']
The following does work (__subclasses__ is available on __class__):
>>> from mock import *
>>> class A(object):
... pass
...
>>> m = create_autospec(A)
>>> m.__class__.__subclasses__()
[]
>>> class B(A): pass
...
>>> m.__class__.__subclasses__()
[<class '__main__.B'>]
>>> m.__class__.__subclasses__ = Mock()
>>> m.__class__.__subclasses__()
<Mock name='mock()' id='4372594896'>
I have an instance of a class SomeClass that is defined within a module m. For the behavior of SomeClass, I need to access the following list:
m.__all__
How can I access this list from an instance of SomeClass?
Instances of SomeClass have the following built-in:
SomeClass.__module__
However, this is simply a string. How can I access the module itself, and its attributes?
The sys module contains a dictionary modules which maps the names of loaded modules to the modules themselves. Together with SomeClass.__module__, you can use this to access the module SomeClass was imported from.
For instance, with a module m.py like this:
# m.py
__all__ = [
"A_CONSTANT",
"SomeClass",
]
A_CONSTANT = "foo"
class SomeClass: pass
... the following works:
>>> from m import SomeClass
>>> SomeClass.__module__
'm'
>>> import sys
>>> sys.modules[SomeClass.__module__]
<module 'm' from '/path/to/m.py'>
>>> sys.modules[SomeClass.__module__].__all__
['SomeClass', 'A_CONSTANT']
>>> sys.modules[SomeClass.__module__].A_CONSTANT
'foo'
For logging purposes I want to retrieve the fully qualified class name of a Python object. (With fully qualified I mean the class name including the package and module name.)
I know about x.__class__.__name__, but is there a simple method to get the package and module?
With the following program
#!/usr/bin/env python
import foo
def fullname(o):
klass = o.__class__
module = klass.__module__
if module == 'builtins':
return klass.__qualname__ # avoid outputs like 'builtins.str'
return module + '.' + klass.__qualname__
bar = foo.Bar()
print(fullname(bar))
and Bar defined as
class Bar(object):
def __init__(self, v=42):
self.val = v
the output is
$ ./prog.py
foo.Bar
If you're still stuck on Python 2, you'll have to use __name__ instead of __qualname__, which is less informative for nested classes - a class Bar nested in a class Foo will show up as Bar instead of Foo.Bar:
def fullname(o):
klass = o.__class__
module = klass.__module__
if module == '__builtin__':
return klass.__name__ # avoid outputs like '__builtin__.str'
return module + '.' + klass.__name__
The provided answers don't deal with nested classes.
Since Python 3.3 (PEP 3155), you can use __qualname__ of the class instead of the __name__. Otherwise, a class like
class Foo:
class Bar: # this one
pass
will show up as just Bar instead of Foo.Bar.
(You'll still need to attach the __module__ to the qualname separately - __qualname__ is not intended to include module names.)
Here's one based on Greg Bacon's excellent answer, but with a couple of extra checks:
__module__ can be None (according to the docs), and also for a type like str it can be __builtin__ (which you might not want appearing in logs or whatever). The following checks for both those possibilities:
def fullname(o):
module = o.__class__.__module__
if module is None or module == str.__class__.__module__:
return o.__class__.__name__
return module + '.' + o.__class__.__name__
(There might be a better way to check for __builtin__. The above just relies on the fact that str is always available, and its module is always __builtin__)
For python3.7 I use:
".".join([obj.__module__, obj.__name__])
Getting:
package.subpackage.ClassName
Consider using the inspect module which has functions like getmodule which might be what are looking for:
>>>import inspect
>>>import xml.etree.ElementTree
>>>et = xml.etree.ElementTree.ElementTree()
>>>inspect.getmodule(et)
<module 'xml.etree.ElementTree' from
'D:\tools\python2.5.2\lib\xml\etree\ElementTree.pyc'>
Some people (e.g. https://stackoverflow.com/a/16763814/5766934) arguing that __qualname__ is better than __name__.
Here is an example that shows the difference:
$ cat dummy.py
class One:
class Two:
pass
$ python3.6
>>> import dummy
>>> print(dummy.One)
<class 'dummy.One'>
>>> print(dummy.One.Two)
<class 'dummy.One.Two'>
>>> def full_name_with_name(klass):
... return f'{klass.__module__}.{klass.__name__}'
>>> def full_name_with_qualname(klass):
... return f'{klass.__module__}.{klass.__qualname__}'
>>> print(full_name_with_name(dummy.One)) # Correct
dummy.One
>>> print(full_name_with_name(dummy.One.Two)) # Wrong
dummy.Two
>>> print(full_name_with_qualname(dummy.One)) # Correct
dummy.One
>>> print(full_name_with_qualname(dummy.One.Two)) # Correct
dummy.One.Two
Note, it also works correctly for builtins:
>>> print(full_name_with_qualname(print))
builtins.print
>>> import builtins
>>> builtins.print
<built-in function print>
__module__ would do the trick.
Try:
>>> import re
>>> print re.compile.__module__
re
This site suggests that __package__ might work for Python 3.0; However, the examples given there won't work under my Python 2.5.2 console.
This is a hack but I'm supporting 2.6 and just need something simple:
>>> from logging.handlers import MemoryHandler as MH
>>> str(MH).split("'")[1]
'logging.handlers.MemoryHandler'
Since the interest of this topic is to get fully qualified names, here is a pitfall that occurs when using relative imports along with the main module existing in the same package. E.g., with the below module setup:
$ cat /tmp/fqname/foo/__init__.py
$ cat /tmp/fqname/foo/bar.py
from baz import Baz
print Baz.__module__
$ cat /tmp/fqname/foo/baz.py
class Baz: pass
$ cat /tmp/fqname/main.py
import foo.bar
from foo.baz import Baz
print Baz.__module__
$ cat /tmp/fqname/foo/hum.py
import bar
import foo.bar
Here is the output showing the result of importing the same module differently:
$ export PYTHONPATH=/tmp/fqname
$ python /tmp/fqname/main.py
foo.baz
foo.baz
$ python /tmp/fqname/foo/bar.py
baz
$ python /tmp/fqname/foo/hum.py
baz
foo.baz
When hum imports bar using relative path, bar sees Baz.__module__ as just "baz", but in the second import that uses full name, bar sees the same as "foo.baz".
If you are persisting the fully-qualified names somewhere, it is better to avoid relative imports for those classes.
Bellow is just an improvement of Greg Bacon's answer, tested for class, instance, method, function, both builtin and user defined.
def fullname(o):
try:
# if o is a class or function, get module directly
module = o.__module__
except AttributeError:
# then get module from o's class
module = o.__class__.__module__
try:
# if o is a class or function, get name directly
name = o.__qualname__
except AttributeError:
# then get o's class name
name = o.__class__.__qualname__
# if o is a method of builtin class, then module will be None
if module == 'builtins' or module is None:
return name
return module + '.' + name
This is an adaption of the answers by Greg Bacon and MB to use the qualified class name. Note that the question did ask for the qualified class name. It was tested with Python 3.8.
def fullname(obj: object) -> str:
"""Return the full name of the given object using its module and qualified class names."""
# Ref: https://stackoverflow.com/a/66508248/
module_name, class_name = obj.__class__.__module__, obj.__class__.__qualname__
if module_name in (None, str.__class__.__module__):
return class_name
return module_name + "." + class_name
None of the answers here worked for me. In my case, I was using Python 2.7 and knew that I would only be working with newstyle object classes.
def get_qualified_python_name_from_class(model):
c = model.__class__.__mro__[0]
name = c.__module__ + "." + c.__name__
return name
My solution is:
def fullname(obj) -> str:
if type(obj).__qualname__ != "type":
# obj is instance
return ".".join(
[
obj.__class__.__module__,
obj.__class__.__qualname__,
]
)
# obj is not instance
return ".".join([obj.__module__, obj.__qualname__])
# not instance
>>> print(fullname(datetime))
"datetime.datetime"
# instance
>>> print(fullname(datetime.now())
"datetime.datetime"
# instance
>>> print(fullname(3))
"builtins.int"