I have a python file say
python_file_a.py
def load_content():
dir = "/down/model/"
model = Model(model_dir=dir)
return model
model = load_content()
def invoke(req):
return model.execute(req)
test_python_file_a.py
#patch("module.python_file_a.load_content")
#patch("module.python_file_a.model", Mock(spec=Model))
def test_invoke():
from module.python_file_a import model, invoke
model.execute = Mock(return_value="Some response")
invoke("some request")
This is still trying to load the actual model from the path "/down/model/" in the test. What is the correct way of patching so that the load_content function is mocked in the test?
Without knowing more about what your code does or how it's used it's hard to say exactly, but in this case the correct approach--and in many cases--is to not hard-code values as local variables in functions. Change your load_content() function to take an argument like:
def load_content(dirname):
...
or even give it a default value like
def load_content(dirname="/default/path"):
pass
For the test don't use the model instance instantiated at module level (arguably you should not be doing this in the first place, but again it depends on what you're trying to do).
Update: Upon closer inspect the problem really seems to stem from you instantiating a module-global instance at import time. Maybe try to avoid doing that and use lazy instantiation instead, like:
model = None
then if you really must write a function that accesses the global variable:
def invoke():
global model
if model is None:
model = load_content()
Alternatively you can use a PEP 562 module-level __getattr__ function.
Or write a class instead of putting everything at module-level.
class ModelInvoker:
def __init__(self, dirname='/path/to/content'):
self.dirname = dirname
#functools.cached_property
def model(self):
return load_content(self.dirname)
def invoke(self, req):
return model.execute(req)
Many other approaches to this depending on your use case. But finding some form of encapsulation is what you need if you want to be able to easily mock and replace parts of some code, and not execute code unnecessarily at import time.
Related
I want to create a subclass of a class of an existing package (whose source code I don't want to/cannot change). The objects of the class are initialized just using a string and then populated later on using all kind of add functions. A minimal example could look like this (without any add functions):
import copy
class Origin(object):
def __init__(self, name):
self.name = name
self.dummy_list = [1, 2, 'a']
self.dummy_stuff = {'a': [12, 'yt']}
def make_copy(self):
return copy.deepcopy(self)
def dummy_function(self):
return len(self.dummy_list)
I want to create a subclass in such a way that I can initialize its instances using an instance of Origin. A straightforward way would be
class BasedOnOrigin(Origin):
def __init__(self, origin_instance, new_prop):
Origin.__init__(self, origin_instance.name)
self.dummy_list = copy.deepcopy(origin_instance.dummy_list)
self.dummy_stuff = copy.deepcopy(origin_instance.dummy_stuff)
self.new_prop = new_prop
The annoying thing there is, that I need to copy all kind of things which I need to know about in advance.
Another option would be
class BasedOnOrigin2(Origin):
def __init__(self, origin_instance, new_prop):
Origin.__init__(self, origin_instance.name)
self = origin_instance.make_copy()
self.new_prop = new_prop
but the self = part looks rather non-standard and new_prop is not set, so I would need an extra function for this.
Is there a standard way of doing this?
An alternative to the above would be to add the additional functions to existing instances using e.g.
from functools import partial
def add_function(obj, func):
setattr(obj, func.__name__, partial(func, obj))
but this can be annoying if there are (i) a lot of functions to add and (ii) a lot of instances to which one wants to add functions.
but the self = part looks rather non-standard and new_prop is not set
self is just a plain local variable, so rebinding it only effects the local scope indeed.
Is there a standard way of doing this?
From what you describe it looks like your real problem is that you have instances of class created by another lib that you don't want / cannot modify and what you really want is to add new methods (and eventually override some methods) to those objects, but cannot since you can tell this lib to use your own class instead.
If the point is purely and simply "replace" the original class with your own version of it (so all instances of the original class are impacted by the change), the canonical solution is to monkeypatch the original class:
from otherlib import TheClass
def patch_the_class():
# we do this in a function to avoid
# polluting the global namespace
# add a new method
def newmethod(self):
# code here
TheClass.newmethod = newmethod
# override an existing method
# keep a reference to the original so
# we can still use it:
_original = TheClass.some_method
def mymethod(self, arg):
something = _original(self, arg)
# additional stuff here
return something
TheClass.some_method = mymethod
patch_the_class()
Just make sure this is executed before any use of the patched class and you're done.
The pro of this solution (wrt/ patching each instance individually) is a lesser cost and the assurance that no one will ever forget to patch an instance.
Now note that monkeypatches are to be considered as either a temporary workaround or a last-resort hack. If the lib you are patching is OSS, you can modify it to either improve the original class or implement some way to make the concrete class to use configurable and contribute it back.
I think the best approach is defining a function that will extend original origin instance without copying it e.g.
def exdend(*origin_instances):
def my_function_one(self):
pass
def my_function_two(self):
pass
for origin_instance in origin_instances:
setattr(origin_instance, my_function_one.__name__, partial(my_function_one, origin_instance))
setattr(origin_instance, my_function_two.__name__, partial(my_function_two, origin_instance))
return origin_instances
I have a file called file_parsers.py and it contains the following class:
class FileParser():
def __init__(self, file_text):
self.file_text = file_text
def do_something(self):
my_value = func_with_no_state()
I'm not sure what questions to ask when deciding whether func_with_no_state() should be inside the class or outside of the class as a file-level function?
Also, is it easier to stub this function when it is at a file-level or inside the class?
So... Does any other class use func_with_no_state? If not, it should be hidden within FileParser. If something else does use it, you have a bigger question. If OtherClass uses func_with_no_state pretty frequently (on par with FileParser) then it would be a good idea to keep func_with_no_state outside so that both classes can use it. But if FileParser is by far the main user, then OtherClass could just pull the function from FileParser's definition.
here is a part of my code :
class projet(object):
def nameCouche(self):
valLissage = float(ui.valLissage.displayText())
return (valLissage)
valCouche = nameCouche() # asks for a positional argument but 'self' doesnt work
def choixTraitement(self):
ui.okLissage.clicked.connect(p.goLissage)
def goLissage(self, valCouche):
if ui.chkboxLissage.isChecked():
print(valCouche) # result is False
os.system(r'"C:\Program Files\FME\fme.exe" D:\Stelios\..... --MAX_NUM_POINTS {0}'.format(valCouche))
So I would like to use valCouche in goLissage method but it doesnt work.
I thought that valCouche would have the argument of valLissage but instead it gives False as a value.
I've tried different alternatives but still doesnt work.
You've got multiple problems here.
First, if you write this in the middle of a class definition:
valCouche = nameCouche()
... you're creating a class attribute, which is shared by all instances, not a normal instance attribute.
Also, you're running this at class definition time. That means there is no self yet--there aren't any instances yet to be self--so you can't call a method like nameCouche, because you don't have anything to call it on.
What you want to do is call the method at instance initialization time, on the instance being initialized, and store the return value in an instance attribute:
def __init__(self):
self.valCouche = self.nameCouche()
Then, when you want to access this value in another method later, you have to access it as self.valCouche.
If you make those changes, it will work. But your object model still doesn't make much sense. Why is nameCouche a method when it doesn't have anything to do with the object, and doesn't access any of its attributes? Maybe it makes sense as a #staticmethod, but really, I think it makes more sense just as a plain function outside the class. In fact, none of the code you've written seems to have anything to do with the class.
This kind of cram-everything-into-the-class design is often a sign that you're trying to write Java code in Python, and haven't yet really understood how Python does OO. You might want to read a good tutorial on Python classes. But briefly: if you're writing a class just to have somewhere to dump a bunch of vaguely-related functions, what you want is a module, not a class. If you have some reason to have instances of that class, and the functions all act on the data of each instance, then you want a class.
You have to declare variabile in the __init__ method (constructor) and then use it in your code
ex:
class projet(object):
def __init__(self):
self.valCouche = ''
def nameCouche(self):
valLissage = float(ui.valLissage.displayText())
return (valLissage)
def choixTraitement(self):
ui.okLissage.clicked.connect(p.goLissage)
def goLissage(self, valCouche):
if ui.chkboxLissage.isChecked():
self.valCouche = self.nameCouche()
print(self.valCouche) # result is False
os.system(r'"C:\Program Files\FME\fme.exe" D:\Stelios\..... --MAX_NUM_POINTS {0}'.format(self.valCouche))
you have to define an initialization function: def__init__(self)
defining valCouche as an instance attribute make it accessible on all the method so we have the following
class projet(object):
def __init__(self):
self.valCouche = ''
def nameCouche(self):
self.valCouche = float(ui.valLissage.displayText())
#staticmethod #here there is no need for self so it is a method of class
def choixTraitement():
ui.okLissage.clicked.connect(p.goLissage)
def goLissage(self):
if ui.chkboxLissage.isChecked():
print(self.valCouche) # result is False
os.system(r'"C:\Program Files\FME\fme.exe" D:\Stelios\..... --MAX_NUM_POINTS {0}'.format(self.valCouche))
Not sure if this is a dupe or not. Here it goes.
I need to write some Python code that looks like:
class TestClass:
def test_case(self):
def get_categories(self):
return [“abc”,”bcd”]
# do the test here
and then have a test engine class that scans all these test classes, loads all the test_case functions and for each invokes get_categories to find out if the test belongs t the group of interest for the specific run.
The problem is that get_categories is not seen as an attribute of test_case, and even if I manually assign it
class TestClass:
def test_case(self):
def get_categories(self):
return [“abc”,”bcd”]
# do the test here
test_case.get_categories = get_categories
this is only going to happen when test_case first runs, too late for me.
The reason why this function can’t go on the class (or at least why I want it to be also available at the per-function level) is that a TestClass can have multiple test cases.
Since this is an already existing testing infrastructure, and the categories mechanism works (other than the categories-on-function scenario, which is of lesser importance), a rewrite is not in the plans.
Language tricks dearly appreciated.
Nested functions don't become attributes any more than any other assignment.
I suspect your test infrastructure is doing some severely weird things if this isn't supported (and uses old-style classes!), but you could just do this:
class TestClass:
def test_case(self):
# ...
def _get_categories(self):
return [...]
test_case.get_categories = _get_categories
del _get_categories
Class bodies are executable code like any other block.
What you need is nested classes. Functions aren't made to do what you are trying to do, so you have to move up a notch. Function attributes are mainly used as markup, whereas classes can have anything you want.
class TestClass(object):
class TestCase(object):
#classmethod
def get_categories(cls):
return ['abc', 'efg']
Note that I used #classmethod so that you could use it without instantiating TestCase(); modify if you want to do test_case = TestCase().
Context
I'm trying to have some "plugins" (I'm not sure this is the correct definition for this) to my code. By "plugin", I mean a module which defines a model (this is a scientific code) in such a way that its existence is enough to use it anywhere else in the code.
Of course these plugins must follow a template which uses some modules/function/classes defined in my code. Here is a small snippet for the relevant part of my code:
# [In the code]
class AllModels():
def __init__(self):
"""
Init.
"""
self.count = 0
def register(self, name, model):
"""
Adds a model to the code
"""
setattr(self, name, model)
self.count += 1
return
class Model():
def __init__(self, **kwargs):
"""
Some constants that defines a model
"""
self.a = kwargs.get("a", None)
self.b = kwargs.get("b", None)
# and so on...
def function1(self, *args, **kwargs):
"""
A function that all models will have, but which needs:
- to have a default behavior (when the instance is created)
- to be redefinable by the "plugin" (ie. the model)
"""
# default code for the default behavior
return
instance = AllModels()
and here is the relevant part of the "plugin":
# [in the plugin file]
from code import Model, instance
newmodel = Model(a="a name", b="some other stuff")
def function1(*args, **kwargs):
"""
Work to do by this model
"""
# some specific model-dependent work
return
instance.register(newmodel)
Additional information and requirements
function1 has exactly the same signature for any model plugin, but
is usually doing a different job for each.
I'd like a default behavior for the function1 so that if it is not
defined by the plugin, I'll still be able to do something (try
different possibilities, and/or raise a warning/error).
In the plugin, function1 may use some other functions that are only defined in this plugin. I'm stating this because the code is running with the multiprocessing module, and I need the instance instance of AllModels to be able to call function1 in child processes. instance is defined in the parent process, as well as the model plugins, but will be used in different child processes (no modification is done to it though).
it would be awesome that function1, when "redefined" by the plugin, be able to access the attributes of the Model instance (ie. self).
Problem
I've read many different sources of python documentation and some SO question. I only see two/three possible solutions to this problem:
1) not declaring function1 method in Model class, but just set it as an attribute when the plugin creates a new instance of it.
# [in the plugin file]
def function1(*args, **kwargs):
# ....
return
newmodel.function1 = function1
and then call it whenever needed. In that case the attribute function1 in the object Model would be initiate to None probably. One caveat of that is that there is no "default behaviour" for function1 (it has to be dealt in the code, eg. testing if instance.function1 is None: ...), and an even bigger one is that I can't access self this way...
2) using somehow the python decorators. I've never used this, and the documentation I've read is not that simple (I mean not straight forward due to the huge number of possibilities on its usage). But it seems to be a good solution. However I'm worried about its performance impact (I've read that it could slow down the execution of the decorated function/method). If this solution is the best option, then I'd like to know how to use it (a quick snippet maybe), and if it is possible to use attributes of the class Model:
# [in the plugin file]
#mydecorator
def function1(self, *args, **kwargs):
"""
I'm not sure I can use *self*, but it would be great since some attributes of self are used for some other function similar to *function1*...
"""
# some stuff using *self*, eg.:
x = self.var **2 + 3.4
# where self.var has been defined before, eg.: newmodel.var = 100.
3) using the module types and its MethodType... I'm not sure that is relevant in my case... but I may be wrong.
As you can probably see after this long question, I'm not very familiar with such python features, and my understanding of decorators is really poor now. While keeping reading some documentation, I thought that might be worth to ask the question here since I'm not sure of the direction to take in order to treat my problem.
Solution
The beauty of the answer of Senderle is that it is really simple and obvious... And having missed it is a shame. Sorry for polluting SO with that question.
Well, unless I'm mistaken, you want to subclass Model. This is sort of like creating an instance of Model and replacing its function1 attribute with a function defined in the plugin module (i.e. your option 1); but it's much cleaner, and takes care of all the details for you:
# [in the plugin file]
from code import Model, instance
class MyModel(Model):
def function1(*args, **kwargs):
"""
Work to do by this model
"""
# some specific model-dependent work
return
newmodel = MyModel(a="a name", b="some other stuff")
instance.register(newmodel)
This way, all the other methods (functions "attached" to a Model instance) are inherited from Model; they will behave in just the same way, but function1 will be overridden, and will follow your customized function1 definition.
Could you write a dummy function1() function in the Model class and raise a NotImplementedError? That way, if anyone tries to inherit from Model without implementing function1(), they'll get an exception when they try to run the code. If you're running the code for them, you can catch that error and return a helpful error message to the user.
For example:
class Model:
#Your code
def function1():
raise NotImplementedError("You need to implement function1
when you inherit from Model")
Then, you can do the following when you run the code:
try:
modelObj.function1()
except NotImplementedError as e:
#Perform error handling here
EDIT: The official Python documentation for NotImplementedError states: "In user defined base classes, abstract methods should raise this exception when they require derived classes to override the method." That does seem to fit the requirements here.
What you are trying to do canbe done in pretty straightforward ways - just using Objected Oriented techniques and taking advantage that in Python functions are also normal objects -
One simple thing to do is just having your "model" class to accept the "function1" as a parameter, and store it as an object member.
Some code like this, with minimal changes to your code - though much more interesting things are certainly possible:
# [In the code]
class AllModels():
def __init__(self):
"""
Init.
"""
self.count = 0
def register(self, name, **kwargs):
"""
Adds a model to the code
"""
model = Model(**kwargs)
setattr(self, name, model)
self.count += 1
return
class Model():
def __init__(self, **kwargs):
"""
Some constants that defines a model
"""
self.a = kwargs.get("a", None)
self.b = kwargs.get("b", None)
if "function1" in kwargs:
self.real_function1 = kwargs["function1"]
self.function1.__doc__ = kwargs["function1"].__doc__
# and so on...
def function1(self, *args, **kwargs):
"""
A function that all models will have, but which needs:
- to have a default behavior (when the instance is created)
- to be redefinable by the "plugin" (ie. the model)
"""
if self.real_function1:
return self.real_function1(self, *args, **kwargs)
# default code for the default behavior
return
instance = AllModels()
and
# [in the plugin file]
from code import Model, instance
newmodel = Model(a="a name", b="some other stuff")
def function1(self, *args, **kwargs):
"""
Work to do by this model
"""
# some specific model-dependent work
return
instance.register("name", function1 = function1, a="a name", b="some other stuff")