In my latest series of questions, I asked about fooling around with the internals of argparse. Mostly, I want to change an attribute defined as:
class FooClass(object):
def __init__(self):
self._this_is_a_re=re.compile("foo")
Since it is "protected", I would like to check if this attribute exists and if it is a regular expression before I substitute in my own regex. e.g.:
import re
myFoo=FooClass()
attr='_this_is_a_re'
if(hasattr(myFoo,attr) and isinstance(getattr(myFoo,attr),re.RegexObject):
setattr(myFoo,attr,re.compile("bar"))
This fails with an attribute error because re has no attribute named RegexObject even though it is in the documentation. Why is RegexObject documented but not available? What am I supposed to be using there? I suppose I could say: type(a) is type(b), but that seems ugly...
It's <type '_sre.SRE_Pattern'> but I'd simply use type(re.compile('')).
BTW, it appears that even re developers don't know for sure what the exact type is, as seen here:
_pattern_type = type(sre_compile.compile("", 0))
Related
Can I identify the namedtuple type of an object from another package using isinstance() or some other call? Consider the following code for checking the type of a namedtuple:
>>> vercingatorix=namedtuple('vercingatorix','x')
>>> v=vercingatorix(1)
>>> w=v
>>> isinstance(w,vercingatorix)
True
So far, so good. But what if the namedtuple is created by another package, e.g. pkg1.pkg2.pkg3.vercingatorix, as reported by type()? I tried isinstance(w, pkg1.pkg2.pkg3.vercingatorix) but I get an AttributeError. The namedtuple is not globally accessible (e.g. isinstance(w, vercingatorix)- -- NameError).
This is a simplified case of my problem. I have a namedtuple object (representing a message) that, depending on the kind of message, has a different subclass type. I need to be able to identify the kind of message, and the simplest way to do this seems to be to look at its type. But I can't formulate a way to do this.
I can grab the class's name (which reveals the message type) and use that for comparison:
>>> w.__class__.__name__
'vercingatorix'
When the python help function is invoked with an argument of string type, it is interpreted by pydoc.Helper.help as a request for information on the topic, symbol, keyword or module identified by the value of the string. For other arguments, help on the object itself is provided, unless the object is an instance of a subclass of str. In this latter case, the pydoc.resolve function looks for a module with a name matching the value of the object and raises an exception if none is found.
To illustrate this, consider the example code:
class Extra(object):
def NewMethod(): return 'New'
Cls1 = type( 'FirstClass', (str,Extra), {'__doc__':'My new class','extra':'An extra attribute'})
inst1 = Cls1('METHODS')
help( 'METHODS' )
help( inst1 )
The first invocation of help produces information on the topic "METHODS", the 2nd produces an error message because the pydoc.resolve function is trying to find a module called "METHODS".
This means that it is difficult to provide effective documentation for user defined sub-classes of str. Would it not be possible for pydoc.resolve to use a test on the type of the object, as is done in pydoc.Helper.help, and allow instances of user defined sub-classes to be treated as other class instances?
This question follows from earlier discussion of a related question here.
The simple answer is that making user-defined subclasses of str is not the most common case—partly because the user-defined data but not the string data would be mutable. By the time you have to deal with such, it’s imagined that you know how to write help(type(x)), and using isinstance rather than type(…) is … is the correct default in general. (The other way, you’d have to use help(str(x)) if you wanted to use it, like any other string, to select a help topic, but that’s surely even rarer.)
I'd like to provide documentation (within my program) on certain dynamically created objects, but still fall back to using their class documentation. Setting __doc__ seems a suitable way to do so. However, I can't find many details in the Python help in this regard, are there any technical problems with providing documentation on an instance? For example:
class MyClass:
"""
A description of the class goes here.
"""
a = MyClass()
a.__doc__ = "A description of the object"
print( MyClass.__doc__ )
print( a.__doc__ )
__doc__ is documented as a writable attribute for functions, but not for instances of user defined classes. pydoc.help(a), for example, will only consider the __doc__ defined on the type in Python versions < 3.9.
Other protocols (including future use-cases) may reasonably bypass the special attributes defined in the instance dict, too. See Special method lookup section of the datamodel documentation, specifically:
For custom classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type, not in the object’s instance dictionary.
So, depending on the consumer of the attribute, what you intend to do may not be reliable. Avoid.
A safe and simple alternative is just to use a different attribute name of your own choosing for your own use-case, preferably not using the __dunder__ syntax convention which usually indicates a special name reserved for some specific use by the implementation and/or the stdlib.
There are some pretty obvious technical problems; the question is whether or not they matter for your use case.
Here are some major uses for docstrings that your idiom will not help with:
help(a): Type help(a) in an interactive terminal, and you get the docstring for MyClass, not the docstring for a.
Auto-generated documentation: Unless you write your own documentation generator, it's not going to understand that you've done anything special with your a value. Many doc generators do have some way to specify help for module and class constants, but I'm not aware of any that will recognize your idiom.
IDE help: Many IDEs will not only auto-complete an expression, but show the relevant docstring in a tooltip. They all do this statically, and without some special-case code designed around your idiom (which they're unlikely to have, given that it's an unusual idiom), they're almost certain to fetch the docstring for the class, not the object.
Here are some where it might help:
Source readability: As a human reading your source, I can tell the intent from the a.__doc__ = … right near the construction of a. Then again, I could tell the same intent just as easily from a Sphinx comment on the constant.
Debugging: pdb doesn't really do much with docstrings, but some GUI debuggers wrapped around it do, and most of them are probably going to show a.__doc__.
Custom dynamic use of docstrings: Obviously any code that you write that does something with a.__doc__ is going to get the instance docstring if you want it to, and therefore can do whatever it wants with it. However, keep in mind that if you want to define your own "protocol", you should use your own name, not one reserved for the implementation.
Notice that most of the same is true for using a descriptor for the docstring:
>>> class C:
... #property
... def __doc__(self):
... return('C doc')
>>> c = C()
If you type c.__doc__, you'll get 'C doc', but help(c) will treat it as an object with no docstring.
It's worth noting that making help work is one of the reasons some dynamic proxy libraries generate new classes on the fly—that is, a proxy to underlying type Spam has some new type like _SpamProxy, instead of the same GenericProxy type used for proxies to Hams and Eggseses. The former allows help(myspam) to show dynamically-generated information about Spam. But I don't know how important a reason it is; often you already need dynamic classes to, e.g., make special method lookup work, at which point adding dynamic docstrings comes for free.
I think it's preferred to keep it under the class via your doc string as it will also aid any developer that works on the code. However if you are doing something dynamic that requires this setup then I don't see any reason why not. Just understand that it adds a level of indirection that makes things less clear to others.
Remember to K.I.S.S. where applicable :)
I just stumbled over this and noticed that at least with python 3.9.5 the behavior seems to have changed.
E.g. using the above example, when I call:
help(a)
I get:
Help on MyClass in module __main__:
<__main__.MyClass object>
A description of the object
Also for reference, have a look at the pydoc implementation which shows:
def _getowndoc(obj):
"""Get the documentation string for an object if it is not
inherited from its class."""
try:
doc = object.__getattribute__(obj, '__doc__')
if doc is None:
return None
if obj is not type:
typedoc = type(obj).__doc__
if isinstance(typedoc, str) and typedoc == doc:
return None
return doc
except AttributeError:
return None
Assume that class MyClass is sometimes, but not always, defined. I have a function foo(a=None) in which argument a can be None, a string, or an object of MyClass.
My question is: If MyClass is not defined in my Python session, how can I check the type of argument a in a fashion similar to isinstance without getting a NameError?
Note on duck-typing: I am deliberately limiting the function.
I'm using Python 2.6.x and Updating is not an option. A forward-compatible solution (especially for 2.7.x) is highly appreciated.
I would suggest a different approach: polyfill the class so all code that wants to refer to it can simply do so:
try:
from foo import Bar # load the native class
except ImportError:
class Bar:
pass # implement necessary parts here
You can put this into your own module and then from mymodule import Bar everywhere it's needed. That allows all your code to use Bar regardless of whether it's defined natively or not.
Even if redefining the class isn't your preferred way to handle this, handling the ImportError is still the way to handle this situation, since you will have to import the class either way and that's where the error will occur. Instead of defining the class, you may instead want to set a class_exists = False flag or something.
If MyClass isn't defined then you have no way to reference its type.
Therefore you can have no way to verify that type(a) has the correct value.
I workarounded the problem by overriding a method in MyClass and doing nothing in it (pass). After that I no longer needed to check its type.
Different workarounds may exist for different cases. Catching the NameError could be another one.
t = 'asdfas'
print(isinstance(t, str))
try:
print(isinstance(t, MyClass))
except NameError:
print(False)
Seems to me, that such a construct may appear in future python. Like typed python, which is quite new. And in typed python we have a possibility to use future types, in apos.
I'm trying to mimic methods.grep from Ruby which simply returns a list of available methods for any object (class or instance) called upon, filtered by regexp pattern passed to grep.
Very handy for investigating objects in an interactive prompt.
def methods_grep(self, pattern):
""" returns list of object's method by a regexp pattern """
from re import search
return [meth_name for meth_name in dir(self) \
if search(pattern, meth_name)]
Because of Python's limitation not quite clear to me it unfortunately can't be simply inserted in the object class ancestor:
object.mgrep = classmethod(methods_grep)
# TypeError: can't set attributes of built-in/extension type 'object'
Is there some workaround how to inject all classes or do I have to stick with a global function like dir ?
There is a module called forbiddenfruit that enables you to patch built-in objects. It also allows you to reverse the changes. You can find it here https://pypi.python.org/pypi/forbiddenfruit/0.1.1
from forbiddenfruit import curse
curse(object, "methods_grep", classmethod(methods_grep))
Of course, using this in production code is likely a bad idea.
There is no workaround AFAIK. I find it quite annoying that you can't alter built-in classes. Personal opinion though.
One way would be to create a base object and force all your objects to inherit from it.
But I don't see the problem to be honest. You can simply use methods_grep(object, pattern), right? You don't have to insert it anywhere.