I am struggling to figure out how to handle importing dependencies that are used in multiple files.
Let's say I want to import an external API for example, and two classes depend on this import. Putting the import into the 'index' file, as an attempt to make it global does not work. I can import it in each class file fine, but that seems to be a violation of DRY, as well as setting myself up for failure later on.
So is there a way to import once, in a single file that is globally accessible? What I experimented with was creating an index.py, foo.py (for the foo class) and bar.py (for the bar class):
Index:
from example import API
import foo
import bar
foo()
bar()
foo.py:
class foo:
... (try to put the example API to use)
bar.py: (same as foo.py really, just here to make the case for using the same dependency in two different places)
This failed to work, as the classes appeared to not be able to access exampleAPI. What is the correct way to do this, or am I looking at it wrong? Thanks!
In general, you should import each module you need in each of your own modules that needs to use it. You don't need to worry about duplication, since each module has its own global namespace. Furthermore, modules are cached (in the sys.modules dictionary) so you don't need to worry about extra work being done to load the module multiple times.
That said, there can be some exceptions. For instance, if the specific source of an API is considered "private" information (e.g. because it's an implementation detail or because it might be configurable and not always come from the same place all the time), it might make sense to import it into some namespace where all other users will look for it.
On the other hand, your example suggests you may be splitting up your code more than you should. Unlike some other languages (such as Java), in Python it's neither required nor recommended for each class to live in its own file. Instead, you should divide your code up into modules dictated by how closely they interact with each other. Closely related classes should be part of the same module, while pieces that don't interact at all might make more sense in separate modules (especially if some other code might need one part but not the other). It may not be inappropriate for your whole program to be in a single module! Obviously, some of this is a matter of style and taste, so there's not a single best option for every programmer in every situation.
For your example code, if you want to keep separate modules, I'd suggest something like this:
index.py:
from foo import Foo # no need to import API here if you're not using it directly
from bar import Bar
foo = Foo() # create an instance of the foo class
result = foo.some_method() # call methods on it
bar = Bar(foo) # you can also pass your instances around to other classes
foo.py:
from example import API
class Foo:
def some_method(self):
return API.whatever() # use the API in some way
bar.py:
from example import API # don't worry about importing the API more than ocne
class Bar:
def __init__(self, foo):
self.foo = foo
def blah(self):
API.something_else(self.foo.some_method())
Note that I changed some names around. Python's convention is to use CapitalizedNames for classes, and lowercase_names_with_underscores (sometimes known as "snake case") for modules, variables and methods. Your original code seemed to have some confusion between the modules name foo and bar and the classes within them with the same names. Using different styles for the different names can help avoiding that confusion.
Related
I would like to convert a singleton-object programmatically into a Python module so that I can use the methods of this singleton-object directly by importing them via the module instead of accessing them as object attributes. By "programmatically" I mean that I do not want to have to copy-paste the class methods explicitly into a module file. I need some sort of a workaround that allows me to import the object methods into to global scope of another module.
I would really appreciate if someone could help me on this one.
Here is a basic example that should illustrate my problem:
mymodule.py
class MyClass:
"""This is my custom class"""
def my_method(self):
return "myValue"
singleton = MyClass()
main_as_is.py
from mymodule import MyClass
myobject = MyClass()
print(myobject.my_method())
main_to_be.py
from mymodule import my_method # or from mymodule.singleton import my_method
print(my_method())
You can use the same strategy that the standard random module uses. All the functions in that module are actually methods of a "private" instance of the Random class. That's convenient for most common uses of the module, although sometimes it's useful to create your own instances of Random so that you can have multiple independent random streams.
I've adapted your code to illustrate that technique. I named the class and its instance with a single leading underscore, since that's the usual convention in Python to signify a private name, but bear in mind it's simply a convention, Python doesn't do anything to enforce this privacy.
mymodule.py
class _MyClass:
""" This is my custom class """
def my_method(self):
return "myValue"
_myclass = _MyClass()
my_method = _myclass.my_method
main_to_be.py
from mymodule import my_method
print(my_method())
output
myValue
BTW, the from mymodule import method1, method2 syntax is ok if you only import a small number of names, or it's clear from the name which module it's from (like math module functions and constants), and you don't import from many modules. Otherwise it's better to use this sort of syntax
import mymodule as mm
# Call a method from the module
mm.method1()
That way it's obvious which names are local, and which ones are imported and where they're imported from. Sure, it's a little more typing, but it makes the code a whole lot more readable. And it eliminates the possibility of name collisions.
FWIW, here's a way to automate adding all of the _myclass methods without explicitly listing them (but remember "explicit is better than implicit"). At the end of "mymodule.py", in place of my_method = _myclass.my_method, add this:
globals().update({k: getattr(_myclass, k) for k in _MyClass.__dict__
if not k.startswith('__')})
I'm not comfortable with recommending this, since it directly injects items into the globals() dict. Note that that code will add all class attributes, not just methods.
In your question you talk about singleton objects. We don't normally use singletons in Python, and many programmers in various OOP languages consider them to be an anti-pattern. See https://stackoverflow.com/questions/12755539/why-is-singleton-considered-an-anti-pattern for details. For this application there is absolutely no need at all to use a singleton. If you only want a single instance of _MyClass then simply don't create another instance of it, just use the instance that mymodule creates for you. But if your boss insists that you must use a singleton, please see the example code here.
If I create a package named foo that imports bar, why is bar visible under foo as foo.bar when I import foo in another module? Is there a way to prevent this; to keep bar hidden so as not to clutter the namespace?
Import bar wherever you use it, rather than globally
If bar is being used in a function, import as
def func():
import bar
....
Or even,
if __name__ == '__main__':
import bar
my_main(bar)
Or if you love classes,
class Fubar():
def __init__(self):
import bar
self.bar = bar
Imports in Python are really just another form of name assignment. There is really no difference between an object that has been imported into foo and one that has been defined in foo - they are both visible internally and externally in exactly the same way. So no, there is no way to prevent this.
I don't really see how this is cluttering the namespace, though. You've still only imported one name, foo, into your other module.
TL;DR: Python Imports create named bindings for pieces of code so they can be referenced and used.
An import is essentially binding a piece of code to a name. So the namespace should always reflect what has been imported. If you hide that you may end up causing unexpected problems for someone else or yourself.
If you are importing the wrong modules, importing modules you don't use, or have a ton of imports because you have 10 classes in one file you should consider fixing the underlying issue(s). Not trying to hide it by messing with how modules are imported.
Does it matter where modules are loaded in a code?
Or should they all be declared at the top, since during load time the external modules will have to be loaded regardless of where they are declared in the code...?
Example:
from os import popen
try:
popen('echo hi')
doSomethingIllegal;
except:
import logging #Module called only when needed?
logging.exception("Record to logger)
or is this optimized by the compiler the same way as:
from os import popen
import logging #Module will always loaded regardless
try:
popen('echo hi')
doSomethingIllegal;
except:
logging.exception("Record to logger)
This indicates it may make a difference:
"import statements can be executed just about anywhere. It's often useful to place them inside functions to restrict their visibility and/or reduce initial startup time. Although Python's interpreter is optimized to not import the same module multiple times, repeatedly executing an import statement can seriously affect performance in some circumstances."
These two OS questions, local import statements? and import always at top of module? discuss this at length.
Finally, if you are curious about your specific case you could profile/benchmark your two alternatives in your environment.
I prefer to put all of my import statements at the top of the source file, following stylistic conventions and for consistency (also it would make changes easier later w/o having to hunt through the source file looking for import statements scattered throughout)
The general rule of thumb is that imports should be at the top of the file, as that makes code easier to follow, and that makes it easier to figure out what a module will need without having to go through all the code.
The Python style guide covers some basic guidelines for how imports should look: http://www.python.org/dev/peps/pep-0008/#imports
In practice, though, there are times when it makes sense to import from within a particular function. This comes up with imports that would be circular:
# Module 1
from module2 import B
class A(object):
def do_something(self):
my_b = B()
...
# Module 2
from module1 import A
class B(object):
def do_something(self):
my_a = A()
...
That won't work as is, but you could get around the circularity by moving the import:
# Module 1
from module2 import B
class A(object):
def do_something(self):
my_b = B()
...
# Module 2
class B(object):
def do_something(self):
from module1 import A
my_a = A()
...
Ideally, you would design the classes such that this would never come up, and maybe even include them in the same module. In that toy example, having each import the other really doesn't make sense. However, in practice, there are some cases where it makes more sense to include an import for one method within the method itself, rather than throwing everything into the same module, or extracting the method in question out to some other object.
But, unless you have good reason to deviate, I say go with the top-of-the-module convention.
I have an application consisting of a base app that brings in several modules. The base app reads a parameter file into a configuration hash, and I want to share it across all my modules.
Currently, I am passing a 'parent' object down to modules, and then those modules are doing stuff like self.parent.config to obtain the configuration.
However, as there are several levels to the module hierarchy, I find myself doing things like self.parent.parent.config, which is starting to look bad.
What are some better patterns for sharing a config object across an application and it's modules? I am thinking about having a 'config' module which basically creates a global config variable that can be set by the base app, then imported and accessed by other modules, but I am not sure if using globals like that is a bad practice for other reasons.
I am reasonably new to Python so be nice =)
You could just:
import config
and have a global config module
excerpts from my comments:
You can always add special rules for odd situations by just saying oddValue if isOddSituation() else config.normalValue.
If you want to have configuration modules be hierarchically subclassable (like my other answer describes), then you can represent a config as a class, or you can use the copy module and make a shallow copy and modify it, or you can use a "config dictionary", e.g.:
import config as baseConfig
config = dict(baseConfig, overriddenValue=etc)
It doesn't really matter too much which scope you're in.
Answering old question:
Just use dependency injection as suggested by #Reed Copsey here. E.g.
class MyClass:
def __init__(myConfig):
self.myConfig = myConfig
...
def foo():
self.myConfig.getConfig(key)
...
self.myConfig.setConfig(key,val)
...
...
# myConfig is your configuration management Module/Class
obj = SomeClass(myConfig)
I think by 'module', you are actually referring to a 'class/object'. An object is an instance of a class, for example:
class MyClass(object):
def __init__(self, ...):
...
...
myObject = MyClass()
A module is a .py file you import, like so:
import mymodule
It seems unlikely that all the classes you instantiate would want to have access to a global configuration. However if you really need everything in your application to have access to some global parameters, you can put them in your own config module:
myParam1 = 1
myParam2 = 2
and then from any module or any object or anywhere really, as long as you did import config, you could just say print(config.myParam1)
Alternatively if you want a large hierarchy of objects to all share access to the same property, you don't need to refer to it via manually setting a self.parent. As long as you use inheritance, you can do stuff like:
class Parent(object):
def __init__(self, theConfig):
self.theConfig = theConfig
class Child(Parent):
...
def method(self,...):
print(self.theConfig)
Take a look at this. It could help you:
https://gist.github.com/dgarana/c052a3287629dd7c0b0c9d7921081e9d
I'm new to python, and I've been reading that using global to pass variables to other functions is considered noobie, as well as a bad practice. I would like to move away from using global variables, but I'm not sure what to do instead.
Right now I have a UI I've created in wxPython as its own separate class, and I have another class that loads settings from a .ini file. Since the settings in the UI should match those in the .ini, how do I pass around those values? I could using something like: Settings = Settings() and then define the variables as something like self.settings1, but then I would have to make Settings a global variable to pass it to my UI class (which it wouldn't be if I assign in it main()).
So what is the correct and pythonic way to pass around these variables?
Edit: Here is the code that I'm working with, and I'm trying to get it to work like Alex Martelli's example. The following code is saved in Settings.py:
import ConfigParser
class _Settings():
#property
def enableautodownload(self): return self._enableautodownload
def __init__(self):
self.config = ConfigParser.ConfigParser()
self.config.readfp(open('settings.ini'))
self._enableautodownload=self.config.getboolean('DLSettings', 'enableautodownload')
settings = _Settings()
Whenever I try to refer to Settings.settings.enableautodownload from another file I get: AttributeError: 'module' object has no attribute 'settings'. What am I doing wrong?
Edit 2: Never mind about the issue, I retyped the code and it works now, so it must have been a simple spelling or syntax error.
The alternatives to global variables are many -- mostly:
explicit arguments to functions, classes called to create one of their instance, etc (this is usually the clearest, since it makes the dependency most explicit, when feasible and not too repetitious);
instance variables of an object, when the functions that need access to those values are methods on that same object (that's OK too, and a reasonable way to use OOP);
"accessor functions" that provide the values (or an object which has attributes or properties for the values).
Each of these (esp. the first and third ones) is particularly useful for values whose names must not be re-bound by all and sundry, but only accessed. The really big problem with global is that it provides a "covert communication channel" (not in the cryptographic sense, but in the literal one: apparently separate functions can actually be depending on each other, influencing each other, via global values that are not "obvious" from the functions' signatures -- this makes the code hard to test, debug, maintain, and understand).
For your specific problem, if you never use the global statement, but rather access the settings in a "read-only" way from everywhere (and you can ensure that more fully by making said object's attributes be read-only properties!), then having the "read-only" accesses be performed on a single, made-once-then-not-changed, module-level instance, is not too bad. I.e., in some module foo.py:
class _Settings(object):
#property
def one(self): return self._one
#property
def two(self): return self._two
def __init__(self, one, two):
self._one, self._two = one, two
settings = _Settings(23, 45)
and from everywhere else, import foo then just access foo.settings.one and foo.settings.two as needed. Note that I've named the class with a single leading underscore (just like the two instance attributes that underlie the read-only properties) to suggest that it's not meant to be used from "outside" the module -- only the settings object is supposed to be (there's no enforcement -- but any user violating such requested privacy is most obviously the only party responsible for whatever mayhem may ensue;-).