Hi I am new to Disco and integrating existing code to it. Is Disco capable to call map/reduce function as a function within a class instead of global function? The following code might explain more clearly.
class Segmenter(object):
def map_fun(line, params):
....
def reduce_fun(iter, params):
....
def disco_mp(self):
job = Job().run(input=["raw://word_to_segment_......"],
map=map_fun,
reduce=reduce_fun)
...
The result of execution is
NameError: global name 'map_fun' is not defined
But if I change map_fun, reduce_fun into global function, it would work fine as expected.
However I still have to find a way to make it work as class functions, is there any way to
do it ?
Thanks,
Chandler
You need static method, you can do this with decorator:
class Segmenter(Job):
map = staticmethod(map_fun)
reduce = staticmethod(reduce_fun)
#staticmethod
def map_fun(line, params):
....
#staticmethod
def reduce_fun(iter, params):
....
def disco_mp(self):
job = self.run(input=["raw://word_to_segment_......"])
Note that you will not have access to self in both map_fun and reduce_fun, and this is why params exists. Also note that Job.run is now self.run and Segmenter extends Job.
It looks like you want to use self.map_fun and self.reduce_fun. Methods of objects can't be accessed by their bare names in Python; you have to use self. You will also need to provide a self argument to those methods. You should read the Python tutorial to familiarize yourself with the basics of classes and methods in Python.
(Also, why is the title of your question unrelated to the actual question?)
Related
I have a python file say
python_file_a.py
def load_content():
dir = "/down/model/"
model = Model(model_dir=dir)
return model
model = load_content()
def invoke(req):
return model.execute(req)
test_python_file_a.py
#patch("module.python_file_a.load_content")
#patch("module.python_file_a.model", Mock(spec=Model))
def test_invoke():
from module.python_file_a import model, invoke
model.execute = Mock(return_value="Some response")
invoke("some request")
This is still trying to load the actual model from the path "/down/model/" in the test. What is the correct way of patching so that the load_content function is mocked in the test?
Without knowing more about what your code does or how it's used it's hard to say exactly, but in this case the correct approach--and in many cases--is to not hard-code values as local variables in functions. Change your load_content() function to take an argument like:
def load_content(dirname):
...
or even give it a default value like
def load_content(dirname="/default/path"):
pass
For the test don't use the model instance instantiated at module level (arguably you should not be doing this in the first place, but again it depends on what you're trying to do).
Update: Upon closer inspect the problem really seems to stem from you instantiating a module-global instance at import time. Maybe try to avoid doing that and use lazy instantiation instead, like:
model = None
then if you really must write a function that accesses the global variable:
def invoke():
global model
if model is None:
model = load_content()
Alternatively you can use a PEP 562 module-level __getattr__ function.
Or write a class instead of putting everything at module-level.
class ModelInvoker:
def __init__(self, dirname='/path/to/content'):
self.dirname = dirname
#functools.cached_property
def model(self):
return load_content(self.dirname)
def invoke(self, req):
return model.execute(req)
Many other approaches to this depending on your use case. But finding some form of encapsulation is what you need if you want to be able to easily mock and replace parts of some code, and not execute code unnecessarily at import time.
I am new to opp programming.I wanted to know what to do with a function that is inside the class but does not use self value
For example
class example:
def __init__(self,n):
self.number=n
def get_t(self,t):
return t*t
def main(self):
b=1
k=self.get_t(b)
From the example the function get_t has nothing to do with self value.
So I wanted to know where to place the function get_t or may be how to restructure the class.
Thank you for your consideration
What you're looking for are static methods. To declare a method static do it like this
#staticmethod
def foo():
pass
Nothing. Just let it be, Python won't complain about it and there's nothing fundamentally wrong about methods that doesn't use its instance. If your linter complains about it, you can shut up that warning. These kind of helper functions are often intended to be private methods that aren't intended to be used externally, you may want to prefix the name with underscore to indicate that.
Convert it into a free function. Python is an OOP language, but it's also a mixed paradigm language, unlike Java, for example, you can actually create a function outside of a class declaration. Pythonic code does not necessarily means putting everything into classes, and often a free function is perfectly suitable place for functions that doesn't involve a particular object instance.
def get_t(t):
return t*t
class example:
def main(self):
b=1
k=self.get_t(b)
If you want to be able to call it from the class by doing Example.get_t(blah) without having to have an instance, then you can either use the staticmethod or classmethod decorator. I suggest using classmethod which can do everything that staticmethod can do while the reverse isn't true and it's easier to make classmethod work correctly when you need to override it in a multi inheritance situation. staticmethod has a very tiny performance advantage, but you're microoptimizing if that's your concern.
class example:
#classmethod
def get_t(cls, t):
return t*t
def main(self):
b=1
k=self.get_t(b)
If get_t() is only being called from one method, you can put it as an inner function of that method:
class example:
def main(self):
def get_t(t):
return t * t
b=1
k=self.get_t(b)
With regards to naming, get_xxx is usually a code smell in python. The get_ prefix indicates that the method is likely a getter, and pythonic code usually don't use getters/setters, because the language supports property. What you have on here though, isn't actually a getter but rather a computation method, so it shouldn't be prefixed with get_. A better name might be calculate_t(t) or square(t).
Case 1: If self is there:-
class example:
def get_t(self,t):
return t*t
Then You can not access get_t function directly from class example like example.get_t(t=2) ,it will give you error. But you can access now by creating an object of class like q = example() and then q.get_t(t=2) , it will give you your desired result.
Case 2 : If self is not there:-
class example:
def get_t(t):
return t*t
Now You can directly access get_t function by class example like example.get_t(t=2) ,it will give you your desired result. But now you cannot use get_t function by creating object like q = example() then q.get_t(t=2) it will give you error.
Conclusion :- It all depends on your use case. But when you struck in this type of ambiguity use #staticmethod like given below:-
class example:
#staticmethod
def get_t(t):
return t*t
I hope it may help you.
I have a file called file_parsers.py and it contains the following class:
class FileParser():
def __init__(self, file_text):
self.file_text = file_text
def do_something(self):
my_value = func_with_no_state()
I'm not sure what questions to ask when deciding whether func_with_no_state() should be inside the class or outside of the class as a file-level function?
Also, is it easier to stub this function when it is at a file-level or inside the class?
So... Does any other class use func_with_no_state? If not, it should be hidden within FileParser. If something else does use it, you have a bigger question. If OtherClass uses func_with_no_state pretty frequently (on par with FileParser) then it would be a good idea to keep func_with_no_state outside so that both classes can use it. But if FileParser is by far the main user, then OtherClass could just pull the function from FileParser's definition.
here is a part of my code :
class projet(object):
def nameCouche(self):
valLissage = float(ui.valLissage.displayText())
return (valLissage)
valCouche = nameCouche() # asks for a positional argument but 'self' doesnt work
def choixTraitement(self):
ui.okLissage.clicked.connect(p.goLissage)
def goLissage(self, valCouche):
if ui.chkboxLissage.isChecked():
print(valCouche) # result is False
os.system(r'"C:\Program Files\FME\fme.exe" D:\Stelios\..... --MAX_NUM_POINTS {0}'.format(valCouche))
So I would like to use valCouche in goLissage method but it doesnt work.
I thought that valCouche would have the argument of valLissage but instead it gives False as a value.
I've tried different alternatives but still doesnt work.
You've got multiple problems here.
First, if you write this in the middle of a class definition:
valCouche = nameCouche()
... you're creating a class attribute, which is shared by all instances, not a normal instance attribute.
Also, you're running this at class definition time. That means there is no self yet--there aren't any instances yet to be self--so you can't call a method like nameCouche, because you don't have anything to call it on.
What you want to do is call the method at instance initialization time, on the instance being initialized, and store the return value in an instance attribute:
def __init__(self):
self.valCouche = self.nameCouche()
Then, when you want to access this value in another method later, you have to access it as self.valCouche.
If you make those changes, it will work. But your object model still doesn't make much sense. Why is nameCouche a method when it doesn't have anything to do with the object, and doesn't access any of its attributes? Maybe it makes sense as a #staticmethod, but really, I think it makes more sense just as a plain function outside the class. In fact, none of the code you've written seems to have anything to do with the class.
This kind of cram-everything-into-the-class design is often a sign that you're trying to write Java code in Python, and haven't yet really understood how Python does OO. You might want to read a good tutorial on Python classes. But briefly: if you're writing a class just to have somewhere to dump a bunch of vaguely-related functions, what you want is a module, not a class. If you have some reason to have instances of that class, and the functions all act on the data of each instance, then you want a class.
You have to declare variabile in the __init__ method (constructor) and then use it in your code
ex:
class projet(object):
def __init__(self):
self.valCouche = ''
def nameCouche(self):
valLissage = float(ui.valLissage.displayText())
return (valLissage)
def choixTraitement(self):
ui.okLissage.clicked.connect(p.goLissage)
def goLissage(self, valCouche):
if ui.chkboxLissage.isChecked():
self.valCouche = self.nameCouche()
print(self.valCouche) # result is False
os.system(r'"C:\Program Files\FME\fme.exe" D:\Stelios\..... --MAX_NUM_POINTS {0}'.format(self.valCouche))
you have to define an initialization function: def__init__(self)
defining valCouche as an instance attribute make it accessible on all the method so we have the following
class projet(object):
def __init__(self):
self.valCouche = ''
def nameCouche(self):
self.valCouche = float(ui.valLissage.displayText())
#staticmethod #here there is no need for self so it is a method of class
def choixTraitement():
ui.okLissage.clicked.connect(p.goLissage)
def goLissage(self):
if ui.chkboxLissage.isChecked():
print(self.valCouche) # result is False
os.system(r'"C:\Program Files\FME\fme.exe" D:\Stelios\..... --MAX_NUM_POINTS {0}'.format(self.valCouche))
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 3 years ago.
Improve this question
I have this code:
class SomeClass:
#classmethod
def func1(cls,arg1):
#---Do Something---
#classmethod
def func2(cls,arg1):
#---Do Something---
# A 'function map' that has function name as its keys and the above function
# objects as values
func_map={'func1':func1,'func2':func2}
#classmethod
def func3(cls,arg1):
# following is a dict(created by reading a config file) that
# contains func names as keys and boolean as values that tells
# the program whether or not to run that function
global funcList
for func in funcList:
if funcList[func]==True:
cls.func_map[func](arg1) #TROUBLING PART!!!
if _name__='main'
SomeClass.func3('Argumentus-Primus')
When I run this I keep getting the error:
Exception TypeError: "'classmethod' object is not callable"
I am unable to figure out what is wrong with this and would appreciate your help.
You can't create references to classmethods until the class has been defined. You'll have to move it out of the class definition. However using a global function map to decide what gets run is really awkward. If you described what you are trying to do with this, we could probably suggest a better solution.
class SomeClass(object):
#classmethod
def func1(cls, arg1):
print("Called func1({})".format(arg1))
#classmethod
def func2(cls, arg1):
print("Call func2({})".format(arg1))
#classmethod
def func3(cls, arg1):
for fnName,do in funcList.iteritems():
if do:
try:
cls.func_map[fnName](arg1)
except KeyError:
print("Don't know function '{}'".format(fnName))
# can't create function map until class has been created
SomeClass.func_map = {
'func1': SomeClass.func1,
'func2': SomeClass.func2
}
if __name__=='__main__':
funcList = {'func1':True, 'func2':False}
SomeClass.func3('Argumentus-Primus')
I discovered something tonight that will be helpful here: We can unwrap magic staticmethod and classmethod objects via: getattr(func, '__func__')
How did I find this information? Using JetBrains' PyCharm (I don't know about other Python IDEs), I viewed the source code for #staticmethod and #classmethod. Both classes define the attribute __func__.
"The rest is left as an exercise for the reader."
All other answers suggest to add some code outside the class SomeClass definition. It may be ok in some cases, but in my case it was very inconvenient. I really wanted to keep the func_map inside the class.
I suggest the following approach. Use not a class variable, but one more classmethod:
class SomeClass:
# ...
#classmethod
def get_func_map(cls):
return {'func1': cls.func1, 'func2': cls.func2}
#classmethod
def func3(cls, arg1):
# .....
cls.get_func_map()[func_name](arg1)
Of course you should modify this code so that a new dictionary not be constructed each time you call the get_func_map method. It's easy, I did not do myself it to keep the example small and clear.
Tested on python 3.6
Add self as an argument for each method within the class.
Also
if _name__='main'
SomeClass.func3('Argumentus-Primus')
should look like this:
if __name__=='__main__':
SomeClass.func3('Argumentus-Primus')
and should not be within the body of the class.
You may need to try a static method.
#staticmethod
def function():...
Static methods do not pass the class as an implicit first argument.
Here's a bad way to do it:
def func3(cls,arg1):
global funcList
for func in funcList:
if funcList[func]==True:
eval(f'SomeClass.{func}')(arg1)
Only works if func is the name of the function. That being said, do not use this method, because you're taking user input. It would be very easy to inject nasty code in the call. That being said, this does work.