I have written a coloured_output module which holds a class named ColouredMsg.
To use it I only have to create an instance of ColouredMsg:
from coloured_output import ColouredMsg
cm = ColouredMsg()
then use it this way:
warning_msg = cm.warn(*parameters)
error_msg = cm.error(*parameters)
The thing is I am planning to use this class for console logs, in a software:
cm = ColouredMsg()
print cm.warn(*parameters)
As one might guess, this call will be often made, in almost every module. So at this point, I'd like to know which practice is best. Either I stick with this way, and I will have to import the module coloured_output and to declare an instance of ColouredMsg at the beginning of each module, or I can make a general function in the coloured_output module, like this:
def warn(*parameters):
cm = ColouredMsg()
return cm.warn(*parameters)
then in my software:
import coloured_output as co
# ...
co.warn(*parameters)
But here I would have to create an instance of ColouredMsg everytime I want to print a coloured message, which would be heavy and should probably be avoided.
Which leaves me with a third choice, I could create a general instance of ColouredMsg in my coloured_output module, which would look like:
class ColouredMsg():
def warn(*parameters):
message = do_whatever_is_needed(*parameters)
return message
cm = ColouredMsg()
def warn(*parameters):
return cm.warn(*parameters)
So I have two choices: either declaring a general instance in each of my software modules, or declaring a general instance in the coloured_output module. Or something else I have not thought about.
I would embrace any suggestion.
Daniel's suggestion in the comments is good.
But if you really want to do this yourself, I recommend using a hidden instance in the module, rather than creating a new temporary instance in each function call. The "hidden instance" approach is used in the standard library, eg, that's what the random module does for the standard random functions.
Related
I want to define a bunch of config variables that can be imported in all the modules in my project. The values of those variables will be constant during runtime but are not known before runtime; they depend on the input. Usually I'd define a dict in my top module which would be passed to all functions and classes from other modules; however, I was thinking it may be cleaner to simply create a blank config.py module which would be dynamically filled with config variables by the top module:
# top.py
import config
config.x = x
# config.py
x = None
# other.py
import config
print(config.x)
I like this approach because I don't have to save the parameters as attributes of classes in my other modules; which makes sense to me because parameters do not describe classes themselves.
This works but is it considered bad practice?
The question as such may be disputed. But I would generally say yes, it's "bad practice" because scope and impact of change is really getting blurred. Note the use case you're describing really is not about sharing configuration, but about different parts of the program functions, objects, modules exchanging data and as such it's a bit of a variation on (meta)global variable).
Reading common configuration values could be fine, but changing them along the way... you may lose track of what happened where and also in which order as modules get imported / values get modified. For instance assume the config.py and two modules m1.py:
import config
print(config.x)
config.x=1
and m2.py:
import config
print(config.x)
config.x=2
and a main.py that just does:
import m1
import m2
import config
print(config.x)
or:
import m2
import m1
import config
print(config.x)
The state in which you find config in each module and really any other (incl. main.py here) depends on order in which imports have occurred and who assigned what value when. Even for a program entirely under your control, this may get confusing (and source of mistakes) rather quickly.
For runtime data and passing information between objects and modules (and your example is really that and not configuration that is predefined and shared between modules) I would suggest you look into describing the information perhaps in a custom state (config) object and pass it around through appropriate interface. But really just a function / method argument may be all that is needed. The exact form depends on what exactly you're trying to achieve and what your overall design is.
In your example, other.py behaves differently when called or imported before top.py which may still seem obvious and manageable in a minimal example, but really is not a very sound design. Anyone reading the code (incl. future you) should be able to follow its logic and this IMO breaks its flow.
The most trivial (and procedural) example of what for what you've described and now I hopefully have a better grasp of would be other.py recreating your current behavior:
def do_stuff(value):
print(value) # We did something useful here
if __name__ == "__main__":
do_stuff(None) # Could also use config with defaults
And your top.py presumably being the entry point and orchestrating importing and execution doing:
import other
x = get_the_value()
other.do_stuff(x)
You can of course introduce an interface to configure do_stuff perhaps a dict or a custom class even with default implementation in config.py:
class Params:
def __init__(self, x=None):
self.x = x
and your other.py:
def do_stuff(params=config.Params()):
print(params.x) # We did something useful here
And on your top.py you can use:
params = config.Params(get_the_value())
other.do_stuff(params)
But you could also have any use case specific source of value(s):
class TopParams:
def __init__(self, url):
self.x = get_value_from_url(url)
params = TopParams("https://example.com/value-source")
other.do_stuff(params)
x could even be a property which you retrieve every time you access it... or lazily when needed and then cached... Again, it really then is a matter of what you need to do.
"Is it bad practice to modify attributes of one module from another module?"
that it is considered as bad practice - violation of the law of demeter, which means in fact "talk to friends, not to strangers".
Objects should expose behaviour and functions, but should HIDE the data.
DataStructures should EXPOSE data, but should not have any methods (which are exposed). The law of demeter does not apply to such DataStructures. OOP Purists might cover such DataStructures with setters and getters, but it really adds no value in Python.
there is a lot of literature about that like : https://en.wikipedia.org/wiki/Law_of_Demeter
and of course, a must to read: "Clean Code", by Robert C. Martin (Uncle Bob), check it out on Youtube also.
For procedural programming it is perfectly normal to keep data in a DataStructure which does not have any (exposed) methods.
The procedures in the program work with that data. Consider to use the module attrs, see : https://www.attrs.org/en/stable/ for easy creation of such classes.
my prefered method for keeping config is (here without using attrs):
# conf_xy.py
"""
config is code - so why use damned parsers, textfiles, xml, yaml, toml and all that
if You just can use testable code as config that can deliver the correct types, etc.
as well as hinting in Your favorite IDE ?
Here, for demonstration without using attrs package - usually I use attrs (read the docs)
"""
class ConfXY(object):
def __init__(self) -> None:
self.x: int = 1
self.z: float = get_z_from_input()
...
conf_xy=ConfXY()
# other.py
from conf_xy import conf_xy
...
y = conf_xy.x * 2
...
If I have a module gobalcfgs containing
class Cfgs:
...
cfg = Cfgs()
is there a way to share this across multiple modules in a project.
I have a main module that import utility modules. So, the import direction is
main <-- utility_1.py
main <-- utility_2.py
However, some global variables and classed get defined in main and are used in the utility modules. Currently I have them as variables in the function call, but it would be nicer to have a global structure that can be accessed from all modules instead of sending the information over a chain of functions.
so, instead of having something like this in main
cfg.sethost(host)
data = utility.getData(cfg)
in utility
def getData(cfg):
getDataFromX(cfg):
def getDataFromX(cfg):
con = openConnection(cfg):
def openConnection(cfg):
con = mycon(cfg.host)
Having cfg being some sort of global and then have in main
from globalcfgs import cfg
cfg.setHost(host)
data = utility.getData()
and in utility
from globalcfgs import cfg
def openConnection():
con = mycon(cfg.host)
This is not exactly a real life example I have but supposed to illustrate that I need to ship information over a long chain of functions.
Is it possible to avoid this?
Firstly, don't call a class globals, as that shadows the built-in globals() function. In any case, Python style is that classes use CamelCase - with initial capitals - and functions use snake_case.
That apart though, what you have is fine. You can import cfg in any module that uses it; the only thing is that you have the import statement the wrong way round, it should be from globalstuff import cfg.
I am writing a Python wrapper for a C library using the cffi.
The C library has to be initialized and shut down. Also, the cffi needs some place to save the state returned from ffi.dlopen().
I can see two paths here:
Either I wrap this whole stateful business in a class like this
class wrapper(object):
def __init__(self):
self.c = ffi.dlopen("mylibrary")
self.c.initialize()
def __del__(self):
self.c.terminate()
Or I provide two global functions that hide the state in a global variable
def initialize():
global __library
__library = ffi.dlopen("mylibrary")
__library.initialize()
def terminate():
__library.terminate()
del __library
The first path is somewhat cumbersome in that it requires the user to always create an object that really serves no other purpose other than managing the library state. On the other hand, it makes sure that terminate() is actually called every time.
The second path seems to result in a somewhat easier API. However, it exposes some hidden global state, which might be a bad thing. Also, if the user forgets to call terminate(), the C library is not unloaded correctly (which is not a big problem on the C side).
Which one of these paths would be more pythonic?
Exposing a wrapper object only makes sense in python if the library actually supports something like multiple instances in one application. If it doesn't support that or it's not really relevant go for kindall's suggestion and just initialize the library when imported and add an atexit handler for cleanup.
Adding wrappers around a stateless api or even an api without support for keeping different sets of state is not really pythonic and would raise expectations that different instances have some kind of isolation.
Example code:
import atexit
# Normal library initialization
__library = ffi.dlopen("mylibrary")
__library.initialize()
# Private library cleanup function
def __terminate():
__library.terminate()
# register function to be called on clean interpreter termination
atexit.register(__terminate)
For more details about atexit this question has some more details, as has the python documentation of course.
Is it possible to overload the from/import statement in Python?
For example, assuming jvm_object is an instance of class JVM, is it possible to write this code:
class JVM(object):
def import_func(self, cls):
return something...
jvm = JVM()
# would invoke JVM.import_func
from jvm import Foo
This post demonstrates how to use functionality introduced in PEP-302 to import modules over the web. I post it as an example of how to customize the import statement rather than as suggested usage ;)
It's hard to find something which isn't possible in a dynamic language like Python, but do we really need to abuse everything? Anyway, here it is:
from types import ModuleType
import sys
class JVM(ModuleType):
Foo = 3
sys.modules['JVM'] = JVM
from JVM import Foo
print Foo
But one pattern I've seen in several libraries/projects is some kind of a _make_module() function, which creates a ModuleType dynamically and initializes everything in it. After that, the current Module is replaced by the new module (using the assignment to sys.modules) and the _make_module() function gets deleted. The advantage of that, is that you can loop over the module and even add objects to the module inside that loop, which is quite useful sometimes (but use it with caution!).
Im a coding a library including textual feedback that I need to translate.
I put the following lines in a _config.py module that I import everywhere in my app :
import gettext, os, sys
pathname = os.path.dirname(sys.argv[0])
localdir = os.path.abspath(pathname) + "/locale"
gettext.install("messages", localdir)
I have the *.mo files in ./locale/lang_LANG/LC_MESSAGES and I apply the _() function to all the strings that need to be translated.
Now I just added a feature for the user, supposedly a programmer, to be able to create his own messages. I don't want him to care about the underlying implementation, so I want him to be able to make it something straightforward like :
lib_object.message = "My message"
I used properties to make it clean, but what if my user whats to translate his own code (that uses mine) and does something like :
import gettext, os, sys
pathname = os.path.dirname(sys.argv[0])
localdir = os.path.abspath(pathname) + "/locale"
gettext.install("user_app", localdir)
lib_object.message = _("My message")
Is it a problem ? What can I do to avoid troubles without bothering my user ?
You can use the class based gettext api to isolate message catalogs. This is also what is recommended in the python gettext documentation.
The drawback is that you, or the other dev, will have to use the gettext method or define the _() method in the local scope, bound to the specific gettext class. An example of a class with its own string catalog:
import gettext
class MyClass(object):
def __init__(self, locale_for_instance):
self.lang = gettext.translation("appname", localedir, \
locale=locale_for_instance)
def some_method(self, arg):
return self.lang.gettext("You called some method")
def other_method(self, arg): # does the same thing
_ = self.lang.gettext
return _("You called some method")
You could stick the code for adding the _() in a decorator, so all the methods that need it is prefixed with something like #with_local_gettext
(Note, I've not tested the above could but It Should Work Just Fine(tm) )
If the goal is to not bother your user (and he's not very good) I guess you could use the class based approach in your code and let the user use the global one.
You can only gettext.install() once. In general it's useless for library work -- gettext.install() will only do the right thing if the module calling it is in charge of the whole program, since it will only provide you with one catalog to load from. Library code should do something akin to what Mailman does: have their own wrapper for gettext() that passes the right arguments for this module, then imports that as '_' in each module that wants to use it.