I have come across a python project that commonly calls external functions from class methods and passes the class instance and some other parameters to the external function.
The method used is shown in method_one below and I have never come across this implementation before. Using locals to get both the local method parameters and the self class instance seems strange to say the least. The code then relies upon the dictionary keys being named correctly i.e. the same as the parameters of the external function (some_function).
To me, the obvious, simpler direct alternative is method_two but even over that I would prefer either
making some_function a method of ExampleClass1 so it has direct access to self, or
passing only the required attributes of the ExampleClass1 instance to some_function.
Example code:
class ExampleClass1(object):
def __init__(self, something):
self.something = something
def method_one(self, param_1, param_2):
all_params = locals()
all_params['example_self'] = all_params.pop('self')
some_function(**all_params)
def method_two(self, param_1, param_2):
some_function(self, param_1, param_2)
def some_function(example_self, param_1, param_2):
print(example_self.something, param_1, param_2)
e = ExampleClass1("do")
e.method_one(1, "a")
e.method_two(2, "b")
So,
Is there any reason to be using method_one that I'm not aware of?
How would you offer advice on the best practice for this situation?
Passing self as a parameter to external functions is a totally standard practice. I'm a little unclear why the call to locals() is used and why keys are being shuffled around, but that's a separate matter. In general, I find that if you're using locals(), then 9 times out of 10 the code you're writing can be simpler. Notable exception being metaprogramming, which is another topic.
One example that I use this for is when you want to separate out code into several modules rather than have one large class with a bunch of methods. There's a lot of ways to organize code, but one approach that I use is to segregate functions to other modules based on their domain, and then pass self to those functions for their use.
Concrete example: a server object accepting requests can have the routes handling those requests live elsewhere, and then delegate the actual business logic to the external route functions. If those routes need the server object, though, then you may want to pass self (being the server) to them. You could make an argument they should just be methods then, but that's a matter of code style and depends a lot on exact use case.
In general, passing self around isn't a bad practice when used appropriately.
Related
Why is this decorator strategy considered bad? (..or is it!?)
class User(object):
def __init__(self):
self.thing = 5
def __atomic_rate_change(fn):
def wrapper(self,*args,**kwargs):
print "start magic"
self.thing += 1
fn(self,*args,**kwargs)
print "end magic"
return wrapper
#__atomic_rate_change
def foo(self,add):
print self.__atomic_rate_change # <bound method User.__atomic_rate_change of <__main__.User object at 0x6ffffe1ef50>>
self.thing += add
print "normal call {0}".format(self.thing)
test = User()
test.foo(1)
This works. But, according to resource below, it's bad practice. Reasons would be that:
[...] there is major flaw in this approach: atomic_rating_change becomes an
instance method of the User class. That doesn’t make any sense. More
than this, it doesn’t even work as a method: if you call it, the
decorated parameter will be used as self.
https://medium.com/#vadimpushtaev/decorator-inside-python-class-1e74d23107f6
I don't understand why it's a problem/wrong/bad that atomic_rate_change is a instance method. I'm only intending the decorator to be used within the class. Perhaps in this case it's okay?
Stylistically, placing function definitions into the class definition which are not methods are kind of out of place (imho it can even be unpythonic). Flat is better than nested, so it is probably better to declare the function outside of the class. This way when the reader is looking at your class, there won't be the confusion of why there is a method which does not take self as an argument (because the function is declared like a method when it is merely a decorator, though this is slightly different if the function is a #staticmethod).
If you're worried about it being used outside of the class, prefix it with an _ and then from my_package import * won't import it. It can still be used in that module, but it won't be used outside unless explicitly imported.
Practically, the author is referring to the occasional odd behavior of scoping (similar to the debates in Javascript on whether to use function() { ... or () => { ... based on how things are scoped.) If you're not careful and accidentally have logic involving self in the wrong part of your decorator, you could have scoping issues.
The only advantages I can see to using functions inside of the classes are possibly because it is closer to the methods (but that introduces unnecessary nesting, potential scoping problems, and cognitive load of realizing that's a decorator and not a method), and better hiding of the function if it's name startswith _ or __.
TL;DR Stylistic/Pythonicity concerns, and potential scoping issues.
When using Pycharm, It often points out an error, saying:
Unresolved reference 'name'. This inspection detects names that should
resolve but don't. Due to dynamic dispatch and duck typing, this is
possible in a limited but useful number of cases. Top-level and
class-level items are supported better than instance items.
I've snooped around about this, but most questions and information I find is about preventing the message from being shown. what I want to know is:
What is dynamic dispatch/duck typing?
What are (or an example of) these "useful number of cases"?
Python uses a duck typing convention. This means that you do not have to specify what type a name is. Unlike in Java, for example, where you must specify explicitly that variable may be type int or Object. Essentially, type checking is done at runtime.
"If it walks like a duck and it quacks like a duck, then it must be a duck."
In Python everything will seem to work until you use try to manipulate an object in a way that it is not designed to. Basically, an object may not have a certain method or attribute that another might, and you won't find this out until Python throws an error upon trying it.
Dynamic Dispatch is the practice of the compiler or environment choosing which version of a polymorphic function to use at runtime. If you have multiple implementations of a method, you can use them in different ways despite the methods having the same or similar properties/attributes. Here's an example:
class Foo:
def flush():
pass
class Bar:
def flush():
pass
Both classes have a flush() method but the correct name is chosen at runtime.
Python is not the best example of this process since methods can take multiple parameters and don't have to be reimplemented. Java is a better example, but I'm not fluent enough in it to provide a correct example.
The warning means that you're using a variable that PyCharm doesn't recognise, but due to Python's dynamic nature it can't be sure if it's right or you're right.
For example you may have the following code:
class myClass():
def myfunc(self):
print(self.name)
PyCharm will probably complain that self.name can't be resolved. However, you may use the class like this:
my_class = myClass()
my_class.name = "Alastair"
my_class.myfunc()
which is perfectly valid (albeit brittle).
The message goes on to say that it's more confident about attribute and methods that are less ambiguous. For example:
class myClass():
my_instance_var = "Al"
def myfunc(self):
print(self.my_instance_var)
As my_instance_var is defined in the source code (a class attribute), PyCharm can be confident it exists.
(Don't use class attributes unless you know what you're doing!)
I'm relatively new to Python.
When I did C/C++ programming, I used the internal classes quite often. For example, in some_file.cc, we may implement a class in the anonymous namespace to prevent it from being used outside. This is useful as a helper class specific to that file.
Then, how we can do a similar thing in Python?
class MyClassOuter:
def __init__(self,...):
class MyClassInner:
def __init__(self,...):
pass
self.my_class = MyClassInner(..)
would only have MyClassInner available inside the __init__ function of MyClassOuter
likewise you could put it inside a function
def my_class_factory(arg1,arg2,...):
class MyClass:
def __init__(self,arg1,arg2,...):
print "OK??"
return MyClass(arg1,arg2,...)
Python code doesn't have any such equivalent for an anonymous namespace, or static linkage for functions. There are a few ways you can get what you're looking for
Prefix with _. Names beginning with an underscore are understood
to be for internal use to that python file and are not exported by
from * imports. it's as simple as class _MyClass.
Use __all__: If a python file contains a list a list of strings
named __all__, the functions and classes named within are
understood to be local to that python file and are not exported by
from *.
Use local classes/functions. This would be done the same way you've
done so with C++ classes.
None these gets exactly what you want, but privacy and restricting in this way are just not part of the language (much like how there's no private data member equivalent). Pydoc is also well aware of these conventions and will provide informative documentation for the intended-to-be-public functions and classes.
I am studying python. I am trying to understand how to design a library that exposes a public api. I want avoid to expose internal methods that could change in future. I am looking for a simple and pythonic way to do it.
I have a library that contains a bunch of classes. Some methods of those classes are used internally among classes. I don't want to expose those methods to the client code.
Suppose that my library (f.e. mylib) contains a class C with two methods a C.public() method thought to be used from client code and C.internal() method used to do some work into the library code.
I want to commit myself to the public api (C.public()) but I am expecting to change the C.internal() method in future, for example adding or removing parameters.
The following code illustrates my question:
mylib/c.py:
class C:
def public(self):
pass
def internal(self):
pass
mylib/f.py:
class F:
def build():
c = C()
c.internal()
return c
mylib/__init__.py:
from mylib.c import C
from mylib.f import F
client/client.py:
import mylib
f = mylib.F()
c = f.build()
c.public()
c.internal() # I wish to hide this from client code
I have thought the following solutions:
document only public api, warning user in documentation to don't use private library api. Live in peace hoping that clients will use only public api. If the next library version breaks client code is the client fault:).
use some form of naming convention, f.e. prefix each method with "_", (it is reserved for protected methods and raises a warning into ide), perhaps I can use other prefixes.
use objects composition to hide internal methods.
For example the library could return to the clients only PC object that
embeds C objects.
mylib/pc.py:
class PC:
def __init__(self, c):
self.__c__
def public(self):
self.__cc__.public()
But this looks a little contrived.
Any suggestion is appreciated :-)
Update
It was suggested that this question is duplicated of Does Python have “private” variables in classes?
It is similar question but I is a bit different about scope. My scope is a library not a single class. I am wondering if there is some convention about marking (or forcing) which are the public methods/classes/functions of a library. For example I use the __init__.py to export the public classes or functions. I am wondering if there is some convention about exporting class methods or if i can rely only on documentation.
I know I can use "_" prefix for marking protected methods. As best as I know protected method are method that can be used in class hierarchy.
I have found a question about marking public method with a decorator #api Sphinx Public API documentation but it was about 3 years ago. There is commonly accepted solution, so if someone are reading my code understand what are methods intended to be library public api, and methods intended to be used internally in the library?
Hope I have clarified my questions.
Thanks all!
You cannot really hide methods and attributes of objects. If you want to be sure that your internal methods are not exposed, wrapping is the way to go:
class PublicC:
def __init__(self):
self._c = C()
def public(self):
self._c.public()
Double underscore as a prefix is usually discouraged as far as I know to prevent collision with python internals.
What is discouraged are __myvar__ names with double-underscore prefix+suffix ...this naming style is used by many python internals and should be avoided -- Anentropic
If you prefer subclassing, you could overwrite internal methods and raise Errors:
class PublicC(C):
def internal(self):
raise Exception('This is a private method')
If you want to use some python magic, you can have a look at __getattribute__. Here you can check what your user is trying to retrieve (a function or an attribute) and raise AttributeError if the client wants to go for an internal/blacklisted method.
class C(object):
def public(self):
print "i am a public method"
def internal(self):
print "i should not be exposed"
class PublicC(C):
blacklist = ['internal']
def __getattribute__(self, name):
if name in PublicC.blacklist:
raise AttributeError("{} is internal".format(name))
else:
return super(C, self).__getattribute__(name)
c = PublicC()
c.public()
c.internal()
# --- output ---
i am a public method
Traceback (most recent call last):
File "covering.py", line 19, in <module>
c.internal()
File "covering.py", line 13, in __getattribute__
raise AttributeError("{} is internal".format(name))
AttributeError: internal is internal
I assume this causes the least code overhead but also requires some maintenance. You could also reverse the check and whitelist methods.
...
whitelist = ['public']
def __getattribute__(self, name):
if name not in PublicC.whitelist:
...
This might be better for your case since the whitelist will probably not change as often as the blacklist.
Eventually, it is up to you. As you said yourself: It's all about documentation.
Another remark:
Maybe you also want to reconsider your class structure. You already have a factory class F for C. Let F have all the internal methods.
class F:
def build(self):
c = C()
self._internal(c)
return c
def _internal(self, c):
# work with c
In this case you do not have to wrap or subclass anything. If there are no hard design constraints to render this impossible, I would recommend this approach.
I have thought the following solutions:
document only public api, warning user in documentation to don't use
private library api. Live in peace hoping that clients will use only
public api. If the next library version breaks client code is the
client fault:).
use some form of naming convention, f.e. prefix each method with "_",
(it is reserved for protected methods and raises a warning into ide),
perhaps I can use other prefixes.
use objects composition to hide internal methods. For example the
library could return to the clients only PC object that embeds C
objects.
You got it pretty right with the first two points.
The Pythonic way is to name internal methods starting with single underscore '_', this way all Python developers know that this method is there, but it's use is discouraged and won't use it. (Until they decide to do some monkey-patching, but you shouldn't care for this scenario.) For newbie developers you might want to mention explicitly about not using methods starting with underscore. Also, just don't provide public documentation for your "private" methods, use it for internal reference only.
You might want to take a look at "name mangling", but it's less common.
Hiding internals with object composition or methods like __getattribute__ and etc. is generally discouraged in Python.
You might want to look at source code of some popular libraries to see how they manage this, e.g. Django, Twisted, etc.
I'm new to python, and I've been reading that using global to pass variables to other functions is considered noobie, as well as a bad practice. I would like to move away from using global variables, but I'm not sure what to do instead.
Right now I have a UI I've created in wxPython as its own separate class, and I have another class that loads settings from a .ini file. Since the settings in the UI should match those in the .ini, how do I pass around those values? I could using something like: Settings = Settings() and then define the variables as something like self.settings1, but then I would have to make Settings a global variable to pass it to my UI class (which it wouldn't be if I assign in it main()).
So what is the correct and pythonic way to pass around these variables?
Edit: Here is the code that I'm working with, and I'm trying to get it to work like Alex Martelli's example. The following code is saved in Settings.py:
import ConfigParser
class _Settings():
#property
def enableautodownload(self): return self._enableautodownload
def __init__(self):
self.config = ConfigParser.ConfigParser()
self.config.readfp(open('settings.ini'))
self._enableautodownload=self.config.getboolean('DLSettings', 'enableautodownload')
settings = _Settings()
Whenever I try to refer to Settings.settings.enableautodownload from another file I get: AttributeError: 'module' object has no attribute 'settings'. What am I doing wrong?
Edit 2: Never mind about the issue, I retyped the code and it works now, so it must have been a simple spelling or syntax error.
The alternatives to global variables are many -- mostly:
explicit arguments to functions, classes called to create one of their instance, etc (this is usually the clearest, since it makes the dependency most explicit, when feasible and not too repetitious);
instance variables of an object, when the functions that need access to those values are methods on that same object (that's OK too, and a reasonable way to use OOP);
"accessor functions" that provide the values (or an object which has attributes or properties for the values).
Each of these (esp. the first and third ones) is particularly useful for values whose names must not be re-bound by all and sundry, but only accessed. The really big problem with global is that it provides a "covert communication channel" (not in the cryptographic sense, but in the literal one: apparently separate functions can actually be depending on each other, influencing each other, via global values that are not "obvious" from the functions' signatures -- this makes the code hard to test, debug, maintain, and understand).
For your specific problem, if you never use the global statement, but rather access the settings in a "read-only" way from everywhere (and you can ensure that more fully by making said object's attributes be read-only properties!), then having the "read-only" accesses be performed on a single, made-once-then-not-changed, module-level instance, is not too bad. I.e., in some module foo.py:
class _Settings(object):
#property
def one(self): return self._one
#property
def two(self): return self._two
def __init__(self, one, two):
self._one, self._two = one, two
settings = _Settings(23, 45)
and from everywhere else, import foo then just access foo.settings.one and foo.settings.two as needed. Note that I've named the class with a single leading underscore (just like the two instance attributes that underlie the read-only properties) to suggest that it's not meant to be used from "outside" the module -- only the settings object is supposed to be (there's no enforcement -- but any user violating such requested privacy is most obviously the only party responsible for whatever mayhem may ensue;-).