We Keep Coding

How to change one class in an external package for my use?

I use an external pacakge which I bring in via pip.
This package has a structure as follows:
class OuterThing:
field: Innerthing
def outer_method1(self...
class InnerThing():
def inner_method1(self,...
def inner_method2(self,...
def inner_method3(self,...
I instantiate only OuterThing objects (which then internally instantiate InnerThing objects).
I want the inner thing objects to have all the normal fields and methods, just inner_method1 I need to customise a bit for my use.
What is the shortest way (i.e. the way with the least code) to do this?
I do not want to copy the whole package into my source tree if possible, just "inject" the changes at runtime (but possibly before all instantiations) to the specified method in InnerThing.

In Python, a method is just an attribute of the class object that happens to be a function having self as its first parameter. That means that you can easily replace it by your own function, provided you keep the same signature. You can even call the original method from your own one:
# after having imported InnerThing
_orig_inner_method1 = InnerThing.inner_method1
def _patched_inner_method1(self, ...):
# your own code
...
_orig_inner_method1(self, ...) # eventually call the original method
...
InnerThing.inner_method1 = _patched_inner_method1
From that point, any InnerThing object created will use your patched method.

Python allows you to Monkey Patch code simply by assigning a different function pointer to the function you're trying to replace. This can be done as long as you can grab a reference to the instance of InnerThing at runtime.
In your case, it seems like OuterThing does have a reference to InnerThing so you can do something like this:
def your_implementation_of_inner_method1(self, ...):
# Do stuff
outerThing = OuterThing()
outerThing.field.inner_method1 = your_implementation_of_inner_method1
If you want to dig deeper as to why this is possible, I recommend having a look at the Python documentation for classes. The tl;dr is that methods are actually objects and are stored as fields in instances of the class.

Instance variables in methods outside the constructor (Python) -- why and how?

My questions concern instance variables that are initialized in methods outside the class constructor. This is for Python.
I'll first state what I understand:
Classes may define a constructor, and it may also define other methods.
Instance variables are generally defined/initialized within the constructor.
But instance variables can also be defined/initialized outside the constructor, e.g. in the other methods of the same class.
An example of (2) and (3) -- see self.meow and self.roar in the Cat class below:
class Cat():
def __init__(self):
self.meow = "Meow!"
def meow_bigger(self):
self.roar = "Roar!"
My questions:
Why is it best practice to initialize the instance variable within the constructor?
What general/specific mess could arise if instance variables are regularly initialized in methods other than the constructor? (E.g. Having read Mark Lutz's Tkinter guide in his Programming Python, which I thought was excellent, I noticed that the instance variable used to hold the PhotoImage objects/references were initialized in the further methods, not in the constructor. It seemed to work without issue there, but could that practice cause issues in the long run?)
In what scenarios would it be better to initialize instance variables in the other methods, rather than in the constructor?
To my knowledge, instance variables exist not when the class object is created, but after the class object is instantiated. Proceeding upon my code above, I demonstrate this:
>> c = Cat()
>> c.meow
'Meow!'
>> c.roar
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'Cat' object has no attribute 'roar'
>>> c.meow_bigger()
>>> c.roar
'Roar!'
As it were:
I cannot access the instance variable (c.roar) at first.
However, after I have called the instance method c.meow_bigger() once, I am suddenly able to access the instance variable c.roar.
Why is the above behaviour so?
Thank you for helping out with my understanding.

Why is it best practice to initialize the instance variable within the
constructor?
Clarity.
Because it makes it easy to see at a glance all of the attributes of the class. If you initialize the variables in multiple methods, it becomes difficult to understand the complete data structure without reading every line of code.
Initializing within the __init__ also makes documentation easier. With your example, you can't write "an instance of Cat has a roar attribute". Instead, you have to add a paragraph explaining that an instance of Cat might have a "roar" attribute, but only after calling the "meow_louder" method.
Clarity is king. One of the smartest programmers I ever met once told me "show me your data structures, and I can tell you how your code works without seeing any of your code". While that's a tiny bit hyperbolic, there's definitely a ring of truth to it. One of the biggest hurdles to learning a code base is understanding the data that it manipulates.
What general/specific mess could arise if instance variables are
regularly initialized in methods other than the constructor?
The most obvious one is that an object may not have an attribute available during all parts of the program, leading to having to add a lot of extra code to handle the case where the attribute is undefined.
In what scenarios would it be better to initialize instance variables
in the other methods, rather than in the constructor?
I don't think there are any.
Note: you don't necessarily have to initialize an attribute with it's final value. In your case it's acceptable to initialize roar to None. The mere fact that it has been initialized to something shows that it's a piece of data that the class maintains. It's fine if the value changes later.

Remember that class members in "pure" Python are just a dictionary. Members aren't added to an instance's dictionary until you run the function in which they are defined. Ideally this is the constructor, because that then guarantees that your members will all exist regardless of the order that your functions are called.
I believe your example above could be translated to:
class Cat():
def __init__(self):
self.__dict__['meow'] = "Meow!"
def meow_bigger(self):
self.__dict__['roar'] = "Roar!"
>>> c = Cat() # c.__dict__ = { 'meow': "Meow!" }
>>> c.meow_bigger() # c.__dict__ = { 'meow': "Meow!", 'roar': "Roar!" }

To initialize instance variables within the constructor, is - as you already pointed out - only recommended in python.
First of all, defining all instance variables within the constructor is a good way to document a class. Everybody, seeing the code, knows what kind of internal state an instance has.
Secondly, order matters. if one defines an instance variable V in a function A and there is another function B also accessing V, it is important to call A before B. Otherwise B will fail since V was never defined. Maybe, A has to be invoked before B, but then it should be ensured by an internal state, which would be an instance variable.
There are many more examples. Generally it is just a good idea to define everything in the __init__ method, and set it to None if it can not / should not be initialized at initialization.
Of course, one could use hasattr method to derive some information of the state. But, also one could check if some instance variable V is for example None, which can imply the same then.
So in my opinion, it is never a good idea to define an instance variable anywhere else as in the constructor.
Your examples state some basic properties of python. An object in Python is basically just a dictionary.
Lets use a dictionary: One can add functions and values to that dictionary and construct some kind of OOP. Using the class statement just brings everything into a clean syntax and provides extra stuff like magic methods.
In other languages all information about instance variables and functions are present before the object was initialized. Python does that at runtime. You can also add new methods to any object outside the class definition: Adding a Method to an Existing Object Instance

3.) But instance variables can also be defined/initialized outside the constructor, e.g. in the other methods of the same class.
I'd recommend providing a default state in initialization, just so its clear what the class should expect. In statically typed languages, you'd have to do this, and it's good practice in python.
Let's convey this by replacing the variable roar with a more meaningful variable like has_roared.
In this case, your meow_bigger() method now has a reason to set has_roar. You'd initialize it to false in __init__, as the cat has not roared yet upon instantiation.
class Cat():
def __init__(self):
self.meow = "Meow!"
self.has_roared = False
def meow_bigger(self):
print self.meow + "!!!"
self.has_roared = True
Now do you see why it often makes sense to initialize attributes with default values?
All that being said, why does python not enforce that we HAVE to define our variables in the __init__ method? Well, being a dynamic language, we can now do things like this.
>>> cat1 = Cat()
>>> cat2 = Cat()
>>> cat1.name = "steve"
>>> cat2.name = "sarah"
>>> print cat1.name
... "steve"
The name attribute was not defined in the __init__ method, but we're able to add it anyway. This is a more realistic use case of setting variables that aren't defaulted in __init__.

I try to provide a case where you would do so for:
3.) But instance variables can also be defined/initialized outside the constructor, e.g. in the other methods of the same class.
I agree it would be clear and organized to include instance field in the constructor, but sometimes you are inherit other class, which is created by some other people and has many instance fields and api.
But if you inherit it only for certain apis and you want to have your own instance field for your own apis, in this case, it is easier for you to just declare extra instance field in the method instead override the other's constructor without bothering to deep into the source code. This also support Adam Hughes's answer, because in this case, you will always have your defined instance because you will guarantee to call you own api first.
For instance, suppose you inherit a package's handler class for web development, you want to include a new instance field called user for handler, you would probability just declare it directly in the method--initialize without override the constructor, I saw it is more common to do so.
class BlogHandler(webapp2.RequestHandler):
def initialize(self, *a, **kw):
webapp2.RequestHandler.initialize(self, *a, **kw)
uid = self.read_cookie('user_id') #get user_id by read cookie in the browser
self.user = User.by_id(int(uid)) #run query in data base find the user and return user

These are very open questions.
Python is a very "free" language in the sense that it tries to never restrict you from doing anything, even if it looks silly. This is why you can do completely useless things such as replacing a class with a boolean (Yes you can).
The behaviour that you mention follows that same logic: if you wish to add an attribute to an object (or to a function - yes you can, too) dynamically, anywhere, not necessarily in the constructor, well... you can.
But it is not because you can that you should. The main reason for initializing attributes in the constructor is readability, which is a prerequisite for maintenance. As Bryan Oakley explains in his answer, class fields are key to understand the code as their names and types often reveal the intent better than the methods.
That being said, there is now a way to separate attribute definition from constructor initialization: pyfields. I wrote this library to be able to define the "contract" of a class in terms of attributes, while not requiring initialization in the constructor. This allows you in particular to create "mix-in classes" where attributes and methods relying on these attributes are defined, but no constructor is provided.
See this other answer for an example and details.

i think to keep it simple and understandable, better to initialize the class variables in the class constructor, so they can be directly called without the necessity of compiling of a specific class method.
class Cat():
def __init__(self,Meow,Roar):
self.meow = Meow
self.roar = Roar
def meow_bigger(self):
return self.roar
def mix(self):
return self.meow+self.roar
c=Cat("Meow!","Roar!")
print(c.meow_bigger())
print(c.mix())
Output
Roar!
Roar!
Meow!Roar!

Python deleting class objects

I'm looking for a way to delete instances of a class, the methods I see to safely do this are like the one proposed by Clint Miller here. For what I'm trying to do, this does not seem like a viable option.
My question is, is the method below safe for getting rid of class instances, and if not, how (if at all) can it be modified to make it so?
class myClass:
_reg = []
def __init__(self, *args):
self.args = args
self._reg.append(self)
...
def close(self):
self._reg.pop(self._reg.index(self))
Instances of myClass are only ever created like this: myClass(arg1, arg2...) as opposed to this: a = myClass(arg1, arg2...) or through a function like myFunc. When the instance is no longer needed, its close function is called.
def myFunc(*args):
...
myClass(newArg1, newArg2...)
P.S You can assume that no new references will be made to items in myClass._reg and thanks in advance.
Edit: The registry exists as a way of keeping track of instances of myClass while they exist. It's not just an overcomplicated way of trying to immediately delete them (I probably wouldn't have created them were that the case).
The amount of time each instance needs to exist and perform functions that have been omitted for brevity (and relevance) varies a lot.
As for why I need to make sure unused instances are deleted new instances need to be created at unspecified times, and some of the intended functions of each instance will begin to clash with one another in ways that will be hard to deal with if they're not deleted or otherwise disabled. Besides that, the list of instances will almost certainly become extremely long as this happens which could cause other problems.

How to access calls to self from python class functions

Concretely, I have a user-defined class of type
class Foo(object):
def __init__(self, bar):
self.bar = bar
def bind(self):
val = self.bar
do_something(val)
I need to:
1) be able to call on the class (not an instance of the class) to recover all the self.xxx attributes defined within the class.
For an instance of a class, this can be done by doing a f = Foo('') and then f.__dict__. Is there a way of doing it for a class, and not an instance? If yes, how? I would expect Foo.__dict__ to return {'bar': None} but it doesn't work this way.
2) be able to access all the self.xxx parameters called from a particular function of a class. For instance I would like to do Foo.bind.__selfparams__ and recieve in return ['bar']. Is there a way of doing this?

This is something that is quite hard to do in a dynamic language, assuming I understand correctly what you're trying to do. Essentially this means going over all the instances in existence for the class and then collecting all the set attributes on those instances. While not infeasible, I would question the practicality of such approach both from a design as well as performance points of view.
More specifically, you're talking of "all the self.xxx attributes defined within the class"—but these things are not defined at all, not at least in a single place—they more like "evolve" as more and more instances of the class are brought to life. Now, I'm not saying all your instances are setting different attributes, but they might, and in order to have a reliable generic solution, you'd literally have to keep track of anything the instances might have done to themselves. So unless you have a static analysis approach in mind, I don't see a clean and efficient way of achieving it (and actually even static analysis is of no help generally speaking in a dynamic language).
A trivial example to prove my point:
class Foo(object):
def __init__(self):
# statically analysable
self.bla = 3
# still, but more difficult
if SOME_CONSTANT > 123:
self.x = 123
else:
self.y = 321
def do_something(self):
import random
setattr(self, "attr%s" % random.randint(1, 100), "hello, world of dynamic languages!")
foo = Foo()
foo2 = Foo()
# only `bla`, `x`, and `y` attrs in existence so far
foo2.do_something()
# now there's an attribute with a random name out there
# in order to detect it, we'd have to get all instances of Foo existence at the moment, and individually inspect every attribute on them.
And, even if you were to iterate all instances in existence, you'd only be getting a snapshot of what you're interested, not all possible attributes.

This is not possible. The class doesn't have those attributes, just functions that set them. Ergo, there is nothing to retrieve and this is impossible.
This is only possible with deep AST inspection. Foo.bar.func_code would normally have the attributes you want under co_freevars but you're looking up the attributes on self, so they are not free variables. You would have to decompile the bytecode from func_code.co_code to AST and then walk said AST.
This is a bad idea. Whatever you're doing, find a different way of doing it.

To do this, you need some way to find all the instances of your class. One way to do this is just to have the class itself keep track of its instances. Unfortunately, keeping a reference to every instance in the class means that those instances can never be garbage-collected. Fortunately, Python has weakref, which will keep a reference to an object but does not count as a reference to Python's memory management, so the instances can be garbage-collected as per usual.
A good place to update the list of instances is in your __init__() method. You could also do it in __new__() if you find the separation of concerns a little cleaner.
import weakref
class Foo(object):
_instances = []
def __init__(self, value):
self.value = value
cls = type(self)
type(self)._instances.append(weakref.ref(self,
type(self)._instances.remove))
#classmethod
def iterinstances(cls):
"Returns an iterator over all instances of the class."
return (ref() for ref in cls._instances)
#classmethod
def iterattrs(cls, attr, default=None):
"Returns an iterator over a named attribute of all instances of the class."
return (getattr(ref(), attr, default) for ref in cls._instances)
Now you can do this:
f1, f2, f3 = Foo(1), Foo(2), Foo(3)
for v in Foo.iterattrs("value"):
print v, # prints 1 2 3
I am, for the record, with those who think this is generally a bad idea and/or not really what you want. In particular, instances may live longer than you expect depending on where you pass them and what that code does with them, so you may not always have the instances you think you have. (Some of this may even happen implicitly.) It is generally better to be explicit about this: rather than having the various instances of your class be stored in random variables all over your code (and libraries), have their primary repository be a list or other container, and access them from there. Then you can easily iterate over them and get whatever attributes you want. However, there may be use cases for something like this and it's possible to code it up, so I did.

Python events and delegates

This is probably a basic question but I am new to programming. I am working with a third party python code and it provides a class with event and event delegates. The syntax for the events and event delegates are follows:
public Delegate Sub RequestEventDelegate (request As MDNPRequest, _
response as MDNPResponseParser)
public Event RequestEvent As MDNPRequest.RequestEventDelegate
I wrote the following code to subcribe to the event but is not working. I do not know what I am
doing wrong.
Mreq = MDNPRequest()
Mreq.RequestEvent += Mreq.RequestEventDelegate(handleResponseEvent)
def handleResponseEvent (request, response):
print ' event fired'
I am adding the two lines of code to the end of a function that opens up the communication channel. I also tested adding the two lines of code to a function that send a poll on the communication channel. In the second scenario the event fires and every time I execute the polling function. Does this defeat the purpose of event subscription?
I think that my problem maybe due to different functions creating instances of the same class. I would like to consolidate some of the functions into a class using the outline shown below. Method1 creates an instance 'a' of a class1 that I would like the other methods in myClass to use. I tried using a class variable which I set to a class1 instance but this is not working. I reference the class variable using the class name for example myClass.variable.somemethod from class1 but I get "Object reference not set to an instance of an object" error. What is the best approach so that all methods in myClass can have access to a? Eventually I would like to call myClass from another module.
from file1 import *
myClass:
class_variable = class1() # class1 from file1
def __init__(self)
...
def Method1(self, argument list):
# this method instantiates a
...
a = class1()
def Method2 (self):
...
a.class1method1
...
def Method3 (self):
...
a.class1method2
...

If this is actually your code:
Mreq.RequestEvent += Mreq.RequestEventDelegate(handleResponseEvent)
def handleRequestEvent (request, response):
print ' event fired'
… handleResponseEvent is not the same thing as handleRequestEvent.
As a side note, you almost never need to create an explicit delegate. It's sometimes a useful optimization, but it's one more thing you can get wrong, and one more thing that can disguise useful debugging information when you do, so it's usually simpler to write the code without it first, and only add wrap it as a delegate after it's working, if you find yourself creating a whole lot of them and want to save some memory.

From your later edits, I suspect that you're missing the fundamentals of how classes work in Python. You may want to read through the tutorial chapter, or maybe search for a friendlier/more detailed tutorial.
In particular:
I would like to consolidate some of the functions into a class using the outline shown below. Method1 creates an instance 'a' of a class1 that I would like the other methods in myClass to use. I tried using a class variable which I set to a class1 instance but this is not working.
That's not the way to do it. Class attributes, like your class_variable, are created at class creation time (that is, generally, as soon as you import the module or run the script), not instance creation time. If you want something created when instances of your class are created, you use instance attributes, not class attributes, and you set them in the __init__ method. In your case, you don't want the instance created until Method1 is called on an instance—again, that means you use an instance attribute; you just do it inside Method1 rather than __init__.
Also, class attributes are shared by all instances of the class; instance attributes, each instance has its own one. Thing about dogs: each dog has its own tail, there's not one tail shared by all dogs, so tail is an instance attribute. Often, in simple scripts, you don't notice the difference, because you only happen to ever create one instance of the class. But if you can't figure out the difference practically, think about it conceptually (like the Dog example)—and if you still can't figure it out, you almost always want an instance attribute.
I reference the class variable using the class name for example myClass.variable.somemethod from class1 but I get "Object reference not set to an instance of an object" error.
Most likely this is because class1 is a COM/interop or .NET class, and you're trying to create and use it before doing any of the relevant setup, which is only happening because you're trying to do it as soon as you import the module/run the script. If so, if you create it when you actually intended to, there won't be a problem.
What is the best approach so that all methods in myClass can have access to a?
Create an instance attribute in Method1, like this:
def Method1(self, argument list):
# this method instantiates a
...
self.a = class1()
And then use it the same way:
def Method2 (self):
...
self.a.class1method1()
...
Just doing a = whatever just creates a local variable that goes away at the end of the method. Even if it happens to have the same name as a class attribute, instance attribute, or global, you're still creating a new local variable, not modifying the thing you want to modify. Unlike some other languages, Python requires you to be explicit about what you're trying to overwrite—self.a for an instance attribute, myClass.a for a class attribute, etc.—so you don't do it by accident.
Also, note the parentheses at the end of that last expression. If you want to call a function or method, you need parentheses; otherwise, you're just referencing the method itself as a value.
Eventually I would like to call myClass from another module.
I'm not sure what you mean by "class myClass". When you call a class, that constructs a new instance of the class. You can then call that instance's methods the same way you would any other object. It doesn't matter what module it was defined in (except that you obviously have to write my_instance = mymodule.MyClass()).
Look at how you use the standard library; it's exactly the same. For example, if you import csv, you can construct a DictWriter by writing my_writer = csv.DictWriter(my_file). And then you call its methods by writing my_writer.writerow(my_row). Once you've constructed it, it doesn't matter what module it came from.
One more thing:
You've tried to define a class like this:
myClass:
You obviously can't do that; you need the class keyword. But also, in Python 2.x, you always want to give base classes, using object if you don't need anything else. Otherwise, you get an old-style class, which causes all kinds of weird quirks and limitations that you don't want to learn about and have to debug. So:
class myClass(object):