I am learning python, step-by-step. Today is about Object Oriented Programming. I know how to create and use simple classes, but something bugs me. Most of the objects I use in python do not require to call a constructor
How can this works? Or is the constructor called implicitly? Example:
>>> import xml.etree.ElementTree as etree
>>> tree = etree.parse('examples/feed.xml')
>>> root = tree.getroot()
>>> root
<Element {http://www.w3.org/2005/Atom}feed at cd1eb0>
(from http://www.diveinto.org/python3/xml.html#xml-parse)
I would have gone this way (which actually works):
>>> import xml.etree.ElementTree as etree
>>> tree = etree.ElementTree() # instanciate object
>>> tree.parse('examples/feed.xml')
I'd like to use this way of programming (do not call constructor, or at least call it implicitly) for my own project, but I can't get how it really works.
Thanks
etree.parse is a Factory function. Their purpose is mainly to be convenient ways of constructing objects (instances). As you can easily verify by looking at the source, the parse function does almost exactly as you do in your second example except it ommits a line of code or two.
In this case, what's happening is that the etree.parse() function is creating the ElementTree object for you, and returning it. That's why you don't have to call a constructor yourself; it's wrapped up in the parse function. That function creates an ElementTree instance, parses the data and modifies the new object to correctly represent the parsed information. Then it returns the object, so you can use it (in fact, if you look at the source, it does essentially what you wrote in your second example).
This is a pretty common idiom in object-oriented programming. Broadly speaking, it's called a factory function. Basically, especially for complex objects, a lot of work is required to create a useful instance of the object. So, rather than pack a lot of logic into the object's constructor, it's cleaner to make one or more factory functions to create the object and configure it as needed. This means that someone developing with the library may have several clean, simple ways to instantiate the class, even if "under the hood" that instantiation may be complex.
In the first case, you are calling a helper function from the module. Its not a class method (though internally it might create an object and then call its method). In the second case, you are instantiating an object and then calling its method.
For a class named ClassName, calling ClassName() implicity calls __init__() and returns you a new instance of ClassName.
If __init__ is not defined in ClassName, the super's __init__ will be called.
In your case, this is all inside a function:
def name(foo):
return ClassName(foo)
n = name("bar") # a function call returns a new instance
Related
I want to use a method from another class.
Neither inheritance nor delegation is a good choice (to my understanding) because the existing class is too complicated to override and too expensive to instanciate.
Note that modifying the existing class is not allowed (legacy project, you know).
I came up with a way:
class Old:
def a(self):
print('Old.a')
class Mine:
b = Old.a
and it shows
>>> Mine().b()
Old.a
>>> Mine().b
<bound method Old.a of <__main__.Mine object at 0x...>>
It seems fine.
And I tried with some more complicated cases including property modification (like self.foo = 'bar'), everything seems okay.
My question:
What is actually happening when I define methods like that?
Will that safely do the trick for my need mentioned above?
Explanation
What's happening is that you are defining a callable class property of class Mine called b. However, this works:
m = Mine()
m.b()
But this won't:
Mine.b()
Why doesn't the second way work?
When you call a function of a class, python expects the first argument to be the actual object upon which the function was called. When you do this, the self argument is automatically passed into the function behind the scenes. Since we called Mine.b() without an instantiated instance of any object, no self was passed into b().
Will this "do the trick"?
As for whether this will do the trick, that depends.
As long as Mine can behave the same way as Old, python won't complain. This is because the python interpreter does not care about the "type" of self. As long as it walks like a duck and quacks like a duck, it's a duck (see duck typing). However, can you guarantee this? What if someone goes and changes the implementation of Old.a. Most of the time, as a client of another system we have no say when the private implementation of functions change.
A simpler solution might be to pull out the functionality you are missing into a separate module. Yes, there is some code duplication but at least you can be confident the code won't change from under you.
Ultimately, if you can guarantee the behavior of Old and Mine will be similar enough for the purposes of Old.a, python really shouldn't care.
The classify_class_attrs function from the inspect module can be used to determine what kind of object each of a class's attributes is, including whether a function is an instance method, a class method, or a static method. Here is an example:
from inspect import classify_class_attrs
class Example(object):
#classmethod
def my_class_method(cls):
pass
#staticmethod
def my_static_method():
pass
def my_instance_method(self):
pass
print classify_class_attrs(Example)
This will output a list of Attribute objects for each attribute on Example, with metadata about the attribute. The relevant ones in these case are:
Attribute(name='my_class_method', kind='class method', defining_class=<class '__main__.Example'>, object=<classmethod object at 0x100535398>)
Attribute(name='my_instance_method', kind='method', defining_class=<class '__main__.Example'>, object=<unbound method Example.my_instance_method>)
Attribute(name='my_static_method', kind='static method', defining_class=<class '__main__.Example'>, object=<staticmethod object at 0x100535558>)
However, it seems that many objects in Python's standard library can't be introspected this way. I'm guessing this has something to do with the fact that many of them are implemented in C. For example, datetime.datetime.now is described with this Attribute object by inspect.classify_class_attrs:
Attribute(name='now', kind='method', defining_class=<type 'datetime.datetime'>, object=<method 'now' of 'datetime.datetime' objects>)
If we compare this to the metadata returned about the attributes on Example, you'd probably draw the conclusion that datetime.datetime.now is an instance method. But it actually behaves as a class method!
from datetime import datetime
print datetime.now() # called from the class: 2014-09-12 16:13:33.890742
print datetime.now().now() # called from a datetime instance: 2014-09-12 16:13:33.891161
Is there a reliable way to determine whether a method on a stdlib class is a static, class, or instance method?
I think you can get much of what you want, distinguishing five kinds, without relying on anything that isn't documented by inspect:
Python instance methods
Python class methods
Python static methods
Builtin instance methods
Builtin class methods or static methods
But you can't distinguish those last two from each other with using CPython-specific implementation details.
(As far as I know, only 3.x has any builtin static methods in the stdlib… but of course even in 2.x, someone could always define one in an extension module.)
The details of what's available in inspect and even what it means are a little different in each version of Python, partly because things have changed between 2.x and 3.x, partly because inspect is basically a bunch of heuristics that have gradually improved over time.
But at least for CPython 2.6 and 2.7 and 3.3-3.5, the simplest way to distinguish builtin instance methods from the other two types is isbuiltin on the method looked up from the class. For a static method or class method, this will be True; for an instance method, False. For example:
>>> inspect.isbuiltin(str.maketrans)
True
>>> inspect.isbuiltin(datetime.datetime.now)
True
>>> inspect.isbuiltin(datetime.datetime.ctime)
False
Why does this work? Well, isbuiltin will:
Return true if the object is a built-in function or a bound built-in method.
When looked up on an instance, either a regular method or a classmethod-like method is bound. But when looked up on the class, a regular method is unbound, while a classmethod-like method is bound (to the class). And of course a staticmethod-like method ends up as a plain-old function when looked up either way. So, it's a bit indirect, but it will always be correct.*
What about class methods vs. static methods?
In CPython 3.x, builtin static and class method descriptors both return the exact same type when looked up on their class, and none of the documented attributes can be used to distinguish them either. And even if this weren't true, I think the way the reference is written, it's guaranteed that no functions in inspect would be able to distinguish them.
What if we turn to the descriptors themselves? Yes, there are ways we can distinguish them… but I don't think it's something guaranteed by the language:
>>> callable(str.__dict__['maketrans'])
False
>>> callable(datetime.datetime.__dict__['now'])
True
Why does this work? Well, static methods just use a staticmethod descriptor, exactly like in Python (but wrapping a builtin function instead of a function). But class and instance methods use a special descriptor type, instead of using classmethod wrapping a (builtin) function and the (builtin) function itself, as Python class and instance methods do. These special descriptor types, classmethod_descriptor and method_descriptor, are unbound (class and instance) methods, as well as being the descriptors that bind them. There are historical/implementation reasons for this to be true, but I don't think there's anything in the language reference that requires it to be true, or even implies it.
And if you're willing to rely on implementation artifacts, isinstance(m, staticmethod) seems a lot simpler…
All that being said, are there any implementations besides CPython that have both builtin staticmethods and classmethods? If not, remember that practicality beats purity…
* What it's really testing for is whether the thing is callable without an extra argument, but that's basically the same thing as the documented "function or bound method"; either way, it's what you want.
I was sure that there'd be an answer to this question somewhere on stack overflow, but I haven't been able to find one; most of them are in regards to passing functions, and not methods, as arguments to functions.
I'm currently working with Python 2.7.5 and I'm trying to define a function like this:
def func(object, method):
object.method()
that when called like so:
some_object_instance = SomeObject()
func(some_object_instance, SomeObject.some_object_method)
using a class defined like this:
class SomeObject:
def some_object_method(self):
# do something
is basically equivalent to doing this:
some_object_instance.some_object_method()
I, however, keep getting an attribute error--something along the lines of
'SomeObject' has no attribute 'method'
I was under the impression that I could legally pass methods as arguments and have them evaluate correctly when used in the aforementioned manner. What am I missing?
That's not the way method calling works. The foo.bar syntax looks for a method named bar on the foo object. If you already have the method, just call it:
def func(object, method):
method(object)
func(some_object_instance, SomeObject.some_object_method)
SomeObject.some_object_method is what's called an "unbound method": it's a method object without a self bound into it, so you have to explicitly pass the self to it.
This might make more sense with a concrete example:
>>> s = 'ABC'
>>> s_lower = s.lower # bound method
>>> s_lower()
'abc'
>>> str_lower = str.lower # unbound method
>>> str_lower(s)
'abc'
By comparison, some_object_instance.some_object_method is a "bound method", you can just call it as-is, and some_object_instance is already "bound in" as the self argument:
def func2(method):
method()
func2(some_object_instance.some_object_method)
Unbound methods aren't explained in detail the tutorial; they're covered in the section on bound methods. So you have to go to the reference manual for documentation (in [standard type hierarchy] (https://docs.python.org/2/reference/datamodel.html#the-standard-type-hierarchy), way down in the subsection "User-defined methods"), which can be a little bit daunting for novices.
Personally, I didn't really get this until I learned how it worked under the covers. About a year ago, I wrote a blog post How methods work to try to explain it to someone else (but in Python 3.x terms, which is slightly different); it may help. To really get it, you have to get through the Descriptor HOWTO, which may take a few read-throughs and a lot of playing around in the interactive interpreter before it really clicks, but hopefully you can understand the basic concepts behind methods before getting to that point.
Since you are passing an unbound method to the function, you need to call it as:
method(object)
Or better pass the name of the method as string and then use getattr:
getattr(object, method)()
I really hope this is not a question posed by millions of newbies, but my search didn t really give me a satisfying answer.
So my question is fairly simple. Are classes basically a container for functions with its own namespace? What other functions do they have beside providing a separate namespace and holding functions while making them callable as class atributes? Im asking in a python context.
Oh and thanks for the great help most of you have been!
More importantly than functions, class instances hold data attributes, allowing you to define new data types beyond what is built into the language; and
they support inheritance and duck typing.
For example, here's a moderately useful class. Since Python files (created with open) don't remember their own name, let's make a file class that does.
class NamedFile(object):
def __init__(self, name):
self._f = f
self.name = name
def readline(self):
return self._f.readline()
Had Python not had classes, you'd probably be working with dicts instead:
def open_file(name):
return {"name": name, "f": open(name)}
Needless to say, calling myfile["f"].readline() all the time will cause your fingers to hurt at some point. You could of course introduce a function readline in a NamedFile module (namespace), but then you'd always have to use that exact function. By contrast, NamedFile instances can be used anywhere you need an object with a readline method, so it would be a plug-in replacement for file in many situation. That's called polymorphism, one of the biggest benefits of OO/class-based programming.
(Also, dict is a class, so using it violates the assumption that there are no classes :)
In most languages, classes are just pieces of code that describe how to produce an object. That's kinda true in Python too:
>>> class ObjectCreator(object):
... pass
...
>>> my_object = ObjectCreator()
>>> print my_object
<__main__.ObjectCreator object at 0x8974f2c>
But classes are more than that in Python. Classes are objects too.
Yes, objects.
As soon as you use the keyword class, Python executes it and creates an OBJECT. The instruction:
>>> class ObjectCreator(object):
... pass
...
creates in memory an object with the name ObjectCreator.
This object (the class) is itself capable of creating objects (the instances), and this is why it's a class.
But still, it's an object, and therefore:
you can assign it to a variable
you can copy it
you can add attributes to it
you can pass it as a function parameter
e.g.:
>>> print ObjectCreator # you can print a class because it's an object
<class '__main__.ObjectCreator'>
>>> def echo(o):
... print o
...
>>> echo(ObjectCreator) # you can pass a class as a parameter
<class '__main__.ObjectCreator'>
>>> print hasattr(ObjectCreator, 'new_attribute')
False
>>> ObjectCreator.new_attribute = 'foo' # you can add attributes to a class
>>> print hasattr(ObjectCreator, 'new_attribute')
True
>>> print ObjectCreator.new_attribute
foo
>>> ObjectCreatorMirror = ObjectCreator # you can assign a class to a variable
>>> print ObjectCreatorMirror.new_attribute
foo
>>> print ObjectCreatorMirror()
<__main__.ObjectCreator object at 0x8997b4c>
Classes (or objects) are used to provide encapsulation of data and operations that can be performed on that data.
They don't provide namespacing in Python per se; module imports provide the same type of stuff and a module can be entirely functional rather than object oriented.
You might gain some benefit from looking at OOP With Python, Dive into Python, Chapter 5. Objects and Object Oriented Programming or even just the Wikipedia article on object oriented programming
A class is the definition of an object. In this sense, the class provides a namespace of sorts, but that is not the true purpose of a class. The true purpose is to define what the object will 'look like' - what the object is capable of doing (methods) and what it will know (properties).
Note that my answer is intended to provide a sense of understanding on a relatively non-technical level, which is what my initial trouble was with understanding classes. I'm sure there will be many other great answers to this question; I hope this one adds to your overall understanding.
When declaring a class that inherits from a specific class:
class C(dict):
added_attribute = 0
the documentation for class C lists all the methods of dict (either through help(C) or pydoc).
Is there a way to hide the inherited methods from the automatically generated documentation (the documentation string can refer to the base class, for non-overwritten methods)? or is it impossible?
This would be useful: pydoc lists the functions defined in a module after its classes. Thus, when the classes have a very long documentation, a lot of less than useful information is printed before the new functions provided by the module are presented, which makes the documentation harder to exploit (you have to skip all the documentation for the inherited methods until you reach something specific to the module being documented).
I had the same problem and solved it on Python 2.7.6 for Windows (x86) by adding 3 lines to pydoc.py. Instructions:
make your own copy of Lib\pydoc.py
find all occurences of the inherited variable (3 times, on my count) in the file and set it to an empty list right after it is defined. For example, I got line 809:
attrs, inherited = _split_list(attrs, lambda t: t[2] is thisclass)
and wrote inherited = [] on a new line below it.
Now it doesn't print inherited methods anymore.
You can give your class a metaclass with the special method __dir__ that returns its own list of attributes. Pydoc will use this list.
Notes:
this will also affect the behaviour of dir().
in rare cases, use of metaclasses has been known to open a portal to a hell dimension.
pydoc and the help builtin don't support this, but there's no reason you couldn't write your own tool (perhaps by modifying pydoc's source) that would have the behavior you desire. Just walk the dict of classes to get the locally-defined attributes, then look for things that have doc as an attribute.