Related
I use an external pacakge which I bring in via pip.
This package has a structure as follows:
class OuterThing:
field: Innerthing
def outer_method1(self...
class InnerThing():
def inner_method1(self,...
def inner_method2(self,...
def inner_method3(self,...
I instantiate only OuterThing objects (which then internally instantiate InnerThing objects).
I want the inner thing objects to have all the normal fields and methods, just inner_method1 I need to customise a bit for my use.
What is the shortest way (i.e. the way with the least code) to do this?
I do not want to copy the whole package into my source tree if possible, just "inject" the changes at runtime (but possibly before all instantiations) to the specified method in InnerThing.
In Python, a method is just an attribute of the class object that happens to be a function having self as its first parameter. That means that you can easily replace it by your own function, provided you keep the same signature. You can even call the original method from your own one:
# after having imported InnerThing
_orig_inner_method1 = InnerThing.inner_method1
def _patched_inner_method1(self, ...):
# your own code
...
_orig_inner_method1(self, ...) # eventually call the original method
...
InnerThing.inner_method1 = _patched_inner_method1
From that point, any InnerThing object created will use your patched method.
Python allows you to Monkey Patch code simply by assigning a different function pointer to the function you're trying to replace. This can be done as long as you can grab a reference to the instance of InnerThing at runtime.
In your case, it seems like OuterThing does have a reference to InnerThing so you can do something like this:
def your_implementation_of_inner_method1(self, ...):
# Do stuff
outerThing = OuterThing()
outerThing.field.inner_method1 = your_implementation_of_inner_method1
If you want to dig deeper as to why this is possible, I recommend having a look at the Python documentation for classes. The tl;dr is that methods are actually objects and are stored as fields in instances of the class.
My questions concern instance variables that are initialized in methods outside the class constructor. This is for Python.
I'll first state what I understand:
Classes may define a constructor, and it may also define other methods.
Instance variables are generally defined/initialized within the constructor.
But instance variables can also be defined/initialized outside the constructor, e.g. in the other methods of the same class.
An example of (2) and (3) -- see self.meow and self.roar in the Cat class below:
class Cat():
def __init__(self):
self.meow = "Meow!"
def meow_bigger(self):
self.roar = "Roar!"
My questions:
Why is it best practice to initialize the instance variable within the constructor?
What general/specific mess could arise if instance variables are regularly initialized in methods other than the constructor? (E.g. Having read Mark Lutz's Tkinter guide in his Programming Python, which I thought was excellent, I noticed that the instance variable used to hold the PhotoImage objects/references were initialized in the further methods, not in the constructor. It seemed to work without issue there, but could that practice cause issues in the long run?)
In what scenarios would it be better to initialize instance variables in the other methods, rather than in the constructor?
To my knowledge, instance variables exist not when the class object is created, but after the class object is instantiated. Proceeding upon my code above, I demonstrate this:
>> c = Cat()
>> c.meow
'Meow!'
>> c.roar
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'Cat' object has no attribute 'roar'
>>> c.meow_bigger()
>>> c.roar
'Roar!'
As it were:
I cannot access the instance variable (c.roar) at first.
However, after I have called the instance method c.meow_bigger() once, I am suddenly able to access the instance variable c.roar.
Why is the above behaviour so?
Thank you for helping out with my understanding.
Why is it best practice to initialize the instance variable within the
constructor?
Clarity.
Because it makes it easy to see at a glance all of the attributes of the class. If you initialize the variables in multiple methods, it becomes difficult to understand the complete data structure without reading every line of code.
Initializing within the __init__ also makes documentation easier. With your example, you can't write "an instance of Cat has a roar attribute". Instead, you have to add a paragraph explaining that an instance of Cat might have a "roar" attribute, but only after calling the "meow_louder" method.
Clarity is king. One of the smartest programmers I ever met once told me "show me your data structures, and I can tell you how your code works without seeing any of your code". While that's a tiny bit hyperbolic, there's definitely a ring of truth to it. One of the biggest hurdles to learning a code base is understanding the data that it manipulates.
What general/specific mess could arise if instance variables are
regularly initialized in methods other than the constructor?
The most obvious one is that an object may not have an attribute available during all parts of the program, leading to having to add a lot of extra code to handle the case where the attribute is undefined.
In what scenarios would it be better to initialize instance variables
in the other methods, rather than in the constructor?
I don't think there are any.
Note: you don't necessarily have to initialize an attribute with it's final value. In your case it's acceptable to initialize roar to None. The mere fact that it has been initialized to something shows that it's a piece of data that the class maintains. It's fine if the value changes later.
Remember that class members in "pure" Python are just a dictionary. Members aren't added to an instance's dictionary until you run the function in which they are defined. Ideally this is the constructor, because that then guarantees that your members will all exist regardless of the order that your functions are called.
I believe your example above could be translated to:
class Cat():
def __init__(self):
self.__dict__['meow'] = "Meow!"
def meow_bigger(self):
self.__dict__['roar'] = "Roar!"
>>> c = Cat() # c.__dict__ = { 'meow': "Meow!" }
>>> c.meow_bigger() # c.__dict__ = { 'meow': "Meow!", 'roar': "Roar!" }
To initialize instance variables within the constructor, is - as you already pointed out - only recommended in python.
First of all, defining all instance variables within the constructor is a good way to document a class. Everybody, seeing the code, knows what kind of internal state an instance has.
Secondly, order matters. if one defines an instance variable V in a function A and there is another function B also accessing V, it is important to call A before B. Otherwise B will fail since V was never defined. Maybe, A has to be invoked before B, but then it should be ensured by an internal state, which would be an instance variable.
There are many more examples. Generally it is just a good idea to define everything in the __init__ method, and set it to None if it can not / should not be initialized at initialization.
Of course, one could use hasattr method to derive some information of the state. But, also one could check if some instance variable V is for example None, which can imply the same then.
So in my opinion, it is never a good idea to define an instance variable anywhere else as in the constructor.
Your examples state some basic properties of python. An object in Python is basically just a dictionary.
Lets use a dictionary: One can add functions and values to that dictionary and construct some kind of OOP. Using the class statement just brings everything into a clean syntax and provides extra stuff like magic methods.
In other languages all information about instance variables and functions are present before the object was initialized. Python does that at runtime. You can also add new methods to any object outside the class definition: Adding a Method to an Existing Object Instance
3.) But instance variables can also be defined/initialized outside the constructor, e.g. in the other methods of the same class.
I'd recommend providing a default state in initialization, just so its clear what the class should expect. In statically typed languages, you'd have to do this, and it's good practice in python.
Let's convey this by replacing the variable roar with a more meaningful variable like has_roared.
In this case, your meow_bigger() method now has a reason to set has_roar. You'd initialize it to false in __init__, as the cat has not roared yet upon instantiation.
class Cat():
def __init__(self):
self.meow = "Meow!"
self.has_roared = False
def meow_bigger(self):
print self.meow + "!!!"
self.has_roared = True
Now do you see why it often makes sense to initialize attributes with default values?
All that being said, why does python not enforce that we HAVE to define our variables in the __init__ method? Well, being a dynamic language, we can now do things like this.
>>> cat1 = Cat()
>>> cat2 = Cat()
>>> cat1.name = "steve"
>>> cat2.name = "sarah"
>>> print cat1.name
... "steve"
The name attribute was not defined in the __init__ method, but we're able to add it anyway. This is a more realistic use case of setting variables that aren't defaulted in __init__.
I try to provide a case where you would do so for:
3.) But instance variables can also be defined/initialized outside the constructor, e.g. in the other methods of the same class.
I agree it would be clear and organized to include instance field in the constructor, but sometimes you are inherit other class, which is created by some other people and has many instance fields and api.
But if you inherit it only for certain apis and you want to have your own instance field for your own apis, in this case, it is easier for you to just declare extra instance field in the method instead override the other's constructor without bothering to deep into the source code. This also support Adam Hughes's answer, because in this case, you will always have your defined instance because you will guarantee to call you own api first.
For instance, suppose you inherit a package's handler class for web development, you want to include a new instance field called user for handler, you would probability just declare it directly in the method--initialize without override the constructor, I saw it is more common to do so.
class BlogHandler(webapp2.RequestHandler):
def initialize(self, *a, **kw):
webapp2.RequestHandler.initialize(self, *a, **kw)
uid = self.read_cookie('user_id') #get user_id by read cookie in the browser
self.user = User.by_id(int(uid)) #run query in data base find the user and return user
These are very open questions.
Python is a very "free" language in the sense that it tries to never restrict you from doing anything, even if it looks silly. This is why you can do completely useless things such as replacing a class with a boolean (Yes you can).
The behaviour that you mention follows that same logic: if you wish to add an attribute to an object (or to a function - yes you can, too) dynamically, anywhere, not necessarily in the constructor, well... you can.
But it is not because you can that you should. The main reason for initializing attributes in the constructor is readability, which is a prerequisite for maintenance. As Bryan Oakley explains in his answer, class fields are key to understand the code as their names and types often reveal the intent better than the methods.
That being said, there is now a way to separate attribute definition from constructor initialization: pyfields. I wrote this library to be able to define the "contract" of a class in terms of attributes, while not requiring initialization in the constructor. This allows you in particular to create "mix-in classes" where attributes and methods relying on these attributes are defined, but no constructor is provided.
See this other answer for an example and details.
i think to keep it simple and understandable, better to initialize the class variables in the class constructor, so they can be directly called without the necessity of compiling of a specific class method.
class Cat():
def __init__(self,Meow,Roar):
self.meow = Meow
self.roar = Roar
def meow_bigger(self):
return self.roar
def mix(self):
return self.meow+self.roar
c=Cat("Meow!","Roar!")
print(c.meow_bigger())
print(c.mix())
Output
Roar!
Roar!
Meow!Roar!
I am trying to extend a Python library to add functionality I desire. The library provides a number of HTML form objects (such as Textbox, Dropdown, Checkbox, etc.), all derived from an Input object. I want to add an additional attribute to all of these derived objects. What is the proper way of proceeding?
I can:
Modify the original source code for the Input class, adding the attribute I want.
Create a new class that inherits Input and adds my attribute, and then create a bunch of new classes that inherit Textbox, Dropdown, and my new Input class.
Create new classes for Textbox, Dropdown, etc. with the attribute.
Solution 1 is the easiest and simplest, but is inherently wrong, whereas the other two seem like much more work and code-repetition than this task should call for.
Is there a solution I'm missing?
The 4th solution is monkey patching and might be a good idea. Before you try this, make sure this won't break anything now or in the future:
def additional_method(self, arg):
print("hello", arg)
Input.additional_method = additional_method
or for short stuff:
Input.additional_method = lambda self, arg: do_something(arg)
now all existing and future Input instances (and therefore instances of all Input subclasses) have an additional_method attached to them.
This works also for any future subclasses of Input that might not even exist yet, or you might not be aware of, at the time of adding the method, so is therefore better (i.e. more generic) than creating an alternative inheritance hierarchy, which you'll have to then maintain and keep in sync with upstream changes.
NOTE: before you downvote just because it contains the phrase "monkey patching", consider that monkey patching doesn't have to be dangerous/fragile, and is a first class (as in "respected") feature in many languages.
You can use a mixin (multiple inheritance). It's a class that just contains your extra attribute, and add this class to the parent class of a subclass of Textbox, Dropdown, Checkbox... like this:
from ... import TextBox
class Mixin:
def __init__(self):
self.new_attribude = ...
class TextBox_with_new_attribute(Mixin, TextBox):
pass
But, it depends tightly on your goals...
Edit: base on #Veedrac comment, in case of third party library.
If there are a lot of classes you could dynamically apply a mixin:
for cls in list_of_classes_to_patch:
patched = type(cls.__name__, (my_mixin, cls), {})
setattr(place_to_export, cls.__name__, patched)
place_to_export can be defined as import this_module as place_to_export.
I've been reading lots of previous SO discussions of factory functions, etc. and still don't know what the best (pythonic) approach is to this particular situation. I'll admit up front that i am imposing a somewhat artificial constraint on the problem in that i want my solution to work without modifying the module i am trying to extend: i could make modifications to it, but let's assume that it must remain as-is because i'm trying to understand best practice in this situation.
I'm working with the http://pypi.python.org/pypi/icalendar module, which handles parsing from and serializing to the Icalendar spec (hereafter ical). It parses the text into a hierarchy of dictionary-like "component" objects, where every "component" is an instance of a trivial derived class implementing the different valid ical types (VCALENDAR, VEVENT, etc.) and they are all spit out by a recursive factory from the common parent class:
class Component(...):
#classmethod
def from_ical(cls, ...)
I have created a 'CalendarFile' class that extends the ical 'Calendar' class, including in it generator function of its own:
class CalendarFile(Calendar):
#classmethod
def from_file(cls, ics):
which opens a file (ics) and passes it on:
instance = cls.from_ical(f.read())
It initializes and modifies some other things in instance and then returns it. The problem is that instance ends up being a Calendar object instead of a CalendarFile object, in spite of cls being CalendarFile. Short of going into the factory function of the ical module and fiddling around in there, is there any way to essentially "recast" that object as a 'CalendarFile'?
The alternatives (again without modifying the original module) that I have considered are:make the CalendarFile class a has-a Calendar class (each instance creates its own internal instance of a Calendar object), but that seems methodically stilted.
fiddle with the returned object to give it the methods it needs (i know there's a term for creating a customized object but it escapes me).
make the additional methods into functions and just have them work with instances of Calendar.
or perhaps the answer is that i shouldn't be trying to subclass from a module in the first place, and this type of code belongs in the module itself.
Again i'm trying to understand what the "best" approach is and also learn if i'm missing any alternatives. Thanks.
Normally, I would expect an alternative constructor defined as a classmethod to simply call the class's standard constructor, transforming the arguments that it receives into valid arguments to the standard constructor.
>>> class Toy(object):
... def __init__(self, x):
... self.x = abs(x)
... def __repr__(self):
... return 'Toy({})'.format(self.x)
... #classmethod
... def from_string(cls, s):
... return cls(int(s))
...
>>> Toy.from_string('5')
Toy(5)
In most cases, I would strongly recommend something like this approach; this is the gold standard for alternative constructors.
But this is a special case.
I've now looked over the source, and I think the best way to add a new class is to edit the module directly; otherwise, scrap inheritance and take option one (your "has-a" option). The different classes are all slightly differentiated versions of the same container class -- they shouldn't really even be separate classes. But if you want to add a new class in the idiom of the code as it it is written, you have to add a new class to the module itself. Furthermore, from_iter is deceptively named; it's not really a constructor at all. I think it should be a standalone function. It builds a whole tree of components linked together, and the code that builds the individual components is buried in a chain of calls to various factory functions that also should be standalone functions but aren't. IMO much of that code ought to live in __init__ where it would be useful to you for subclassing, but it doesn't.
Indeed, none of the subclasses of Component even add any methods. By adding methods to your subclass of Calendar, you're completely disregarding the actual idiom of the code. I don't like its idiom very much but by disregarding that idiom, you're making it even worse. If you don't want to modify the original module, then forget about inheritance here and give your object a has-a relationship to Calendar objects. Don't modify __class__; establish your own OO structure that follows standard OO practices.
This is probably a basic question but I am new to programming. I am working with a third party python code and it provides a class with event and event delegates. The syntax for the events and event delegates are follows:
public Delegate Sub RequestEventDelegate (request As MDNPRequest, _
response as MDNPResponseParser)
public Event RequestEvent As MDNPRequest.RequestEventDelegate
I wrote the following code to subcribe to the event but is not working. I do not know what I am
doing wrong.
Mreq = MDNPRequest()
Mreq.RequestEvent += Mreq.RequestEventDelegate(handleResponseEvent)
def handleResponseEvent (request, response):
print ' event fired'
I am adding the two lines of code to the end of a function that opens up the communication channel. I also tested adding the two lines of code to a function that send a poll on the communication channel. In the second scenario the event fires and every time I execute the polling function. Does this defeat the purpose of event subscription?
I think that my problem maybe due to different functions creating instances of the same class. I would like to consolidate some of the functions into a class using the outline shown below. Method1 creates an instance 'a' of a class1 that I would like the other methods in myClass to use. I tried using a class variable which I set to a class1 instance but this is not working. I reference the class variable using the class name for example myClass.variable.somemethod from class1 but I get "Object reference not set to an instance of an object" error. What is the best approach so that all methods in myClass can have access to a? Eventually I would like to call myClass from another module.
from file1 import *
myClass:
class_variable = class1() # class1 from file1
def __init__(self)
...
def Method1(self, argument list):
# this method instantiates a
...
a = class1()
def Method2 (self):
...
a.class1method1
...
def Method3 (self):
...
a.class1method2
...
If this is actually your code:
Mreq.RequestEvent += Mreq.RequestEventDelegate(handleResponseEvent)
def handleRequestEvent (request, response):
print ' event fired'
… handleResponseEvent is not the same thing as handleRequestEvent.
As a side note, you almost never need to create an explicit delegate. It's sometimes a useful optimization, but it's one more thing you can get wrong, and one more thing that can disguise useful debugging information when you do, so it's usually simpler to write the code without it first, and only add wrap it as a delegate after it's working, if you find yourself creating a whole lot of them and want to save some memory.
From your later edits, I suspect that you're missing the fundamentals of how classes work in Python. You may want to read through the tutorial chapter, or maybe search for a friendlier/more detailed tutorial.
In particular:
I would like to consolidate some of the functions into a class using the outline shown below. Method1 creates an instance 'a' of a class1 that I would like the other methods in myClass to use. I tried using a class variable which I set to a class1 instance but this is not working.
That's not the way to do it. Class attributes, like your class_variable, are created at class creation time (that is, generally, as soon as you import the module or run the script), not instance creation time. If you want something created when instances of your class are created, you use instance attributes, not class attributes, and you set them in the __init__ method. In your case, you don't want the instance created until Method1 is called on an instance—again, that means you use an instance attribute; you just do it inside Method1 rather than __init__.
Also, class attributes are shared by all instances of the class; instance attributes, each instance has its own one. Thing about dogs: each dog has its own tail, there's not one tail shared by all dogs, so tail is an instance attribute. Often, in simple scripts, you don't notice the difference, because you only happen to ever create one instance of the class. But if you can't figure out the difference practically, think about it conceptually (like the Dog example)—and if you still can't figure it out, you almost always want an instance attribute.
I reference the class variable using the class name for example myClass.variable.somemethod from class1 but I get "Object reference not set to an instance of an object" error.
Most likely this is because class1 is a COM/interop or .NET class, and you're trying to create and use it before doing any of the relevant setup, which is only happening because you're trying to do it as soon as you import the module/run the script. If so, if you create it when you actually intended to, there won't be a problem.
What is the best approach so that all methods in myClass can have access to a?
Create an instance attribute in Method1, like this:
def Method1(self, argument list):
# this method instantiates a
...
self.a = class1()
And then use it the same way:
def Method2 (self):
...
self.a.class1method1()
...
Just doing a = whatever just creates a local variable that goes away at the end of the method. Even if it happens to have the same name as a class attribute, instance attribute, or global, you're still creating a new local variable, not modifying the thing you want to modify. Unlike some other languages, Python requires you to be explicit about what you're trying to overwrite—self.a for an instance attribute, myClass.a for a class attribute, etc.—so you don't do it by accident.
Also, note the parentheses at the end of that last expression. If you want to call a function or method, you need parentheses; otherwise, you're just referencing the method itself as a value.
Eventually I would like to call myClass from another module.
I'm not sure what you mean by "class myClass". When you call a class, that constructs a new instance of the class. You can then call that instance's methods the same way you would any other object. It doesn't matter what module it was defined in (except that you obviously have to write my_instance = mymodule.MyClass()).
Look at how you use the standard library; it's exactly the same. For example, if you import csv, you can construct a DictWriter by writing my_writer = csv.DictWriter(my_file). And then you call its methods by writing my_writer.writerow(my_row). Once you've constructed it, it doesn't matter what module it came from.
One more thing:
You've tried to define a class like this:
myClass:
You obviously can't do that; you need the class keyword. But also, in Python 2.x, you always want to give base classes, using object if you don't need anything else. Otherwise, you get an old-style class, which causes all kinds of weird quirks and limitations that you don't want to learn about and have to debug. So:
class myClass(object):