We Keep Coding

Python - global class that can be reused

I have a config var in the main class that I use a lot and I also pass it to some other instances that pass it to other classes, and so on.
I need a solution that can be used globally, without the need of passing an instance of the config file, and that I can modify as needed (not const).
something that only requires import anywhere and that the data will be the same and up to date.
In Java, I know that I can Autowired and I know that behind the scenes I'll use the exact same instance.
What solution does Python have?
example
util.py
config = None
setConfig(self, config):
self.config = config
getConfig(self):
return self.config
file1.py
....
import util
util.setConfig(4)
file2.py
import util
util.getConfig()
file3.py
import util
util.setConfig(8)
file4.py
import util
util.getConfig()
main.py
file1()
file2() # ==4
file3()
file4() # ==8
file2() # ==8
Thanks

Pytest mock: how to patch an import statement inside a function [duplicate]

Is it possible to mock a module in python using unittest.mock? I have a module named config, while running tests I want to mock it by another module test_config. how can I do that ? Thanks.
config.py:
CONF_VAR1 = "VAR1"
CONF_VAR2 = "VAR2"
test_config.py:
CONF_VAR1 = "test_VAR1"
CONF_VAR2 = "test_VAR2"
All other modules read config variables from the config module. While running tests I want them to read config variables from test_config module instead.

If you're always accessing the variables in config.py like this:
import config
...
config.VAR1
You can replace the config module imported by whatever module you're actually trying to test. So, if you're testing a module called foo, and it imports and uses config, you can say:
from mock import patch
import foo
import config_test
....
with patch('foo.config', new=config_test):
foo.whatever()
But this isn't actually replacing the module globally, it's only replacing it within the foo module's namespace. So you would need to patch it everywhere it's imported. It also wouldn't work if foo does this instead of import config:
from config import VAR1
You can also mess with sys.modules to do this:
import config_test
import sys
sys.modules["config"] = config_test
# import modules that uses "import config" here, and they'll actually get config_test
But generally it's not a good idea to mess with sys.modules, and I don't think this case is any different. I would favor all of the other suggestions made over it.

foo.py:
import config
VAR1 = config.CONF_VAR1
def bar():
return VAR1
test.py:
import unittest
import unittest.mock as mock
import test_config
class Test(unittest.TestCase):
def test_one(self):
with mock.patch.dict('sys.modules', config=test_config):
import foo
self.assertEqual(foo.bar(), 'test_VAR1')
As you can see, the patch works even for code executed during import foo.

If you want to mock an entire module just mock the import where the module is used.
myfile.py
import urllib
test_myfile.py
import mock
import unittest
class MyTest(unittest.TestCase):
#mock.patch('myfile.urllib')
def test_thing(self, urllib):
urllib.whatever.return_value = 4

Consider this following setup
configuration.py:
import os
class Config(object):
CONF_VAR1 = "VAR1"
CONF_VAR2 = "VAR2"
class TestConfig(object):
CONF_VAR1 = "test_VAR1"
CONF_VAR2 = "test_VAR2"
if os.getenv("TEST"):
config = TestConfig
else:
config = Config
now everywhere else in your code you can use:
from configuration import config
print config.CONF_VAR1, config.CONF_VAR2
And when you want to mock your coniguration file just set the environment variable "TEST".
Extra credit:
If you have lots of configuration variables that are shared between your testing and non-testing code, then you can derive TestConfig from Config and simply overwrite the variables that need changing:
class Config(object):
CONF_VAR1 = "VAR1"
CONF_VAR2 = "VAR2"
CONF_VAR3 = "VAR3"
class TestConfig(Config):
CONF_VAR2 = "test_VAR2"
# CONF_VAR1, CONF_VAR3 remain unchanged

If your application ("app.py" say) looks like
import config
print config.var1, config.var2
And gives the output:
$ python app.py
VAR1 VAR2
You can use mock.patch to patch the individual config variables:
from mock import patch
with patch('config.var1', 'test_VAR1'):
import app
This results in:
$ python mockimport.py
test_VAR1 VAR2
Though I'm not sure if this is possible at the module level.

How to use a 'global' variable in python?

In my problem I have a python code that is being started by a user, like:
# file main.py
import sys
import mymodule
config = sys.argv[1]
which imports another module containing functions, classes etc. like
# module.py
def check_config():
# do something with the content of 'config'
class Module(object):
# does something with the content of 'config'
How can I access the value of 'config' from within module.py? Should I use a 'global' variable here? Or is there a more sophisticated, pythonic way to solve this problem?
Also, I do not want to define an argument 'config' to each function and class I use in other modules...
Further remark: main.py imports other modules, not the other way around...

Instead of trying to wrangle global into performing this you should pass config as a parameter.
file main.py
import sys
import mymodule
config = sys.argv[1]
checked = mymodule.check_config(config)
mod = mymodule.Module(config)
module.py
def check_config(config):
# do something with the content of 'config'
class Module(object):
# does something with the content of 'config'
def __init__(self, config):
# initialise with config
Always avoid usingglobal when you can. If you need to modify config just have a module function return it.
config = change_config(config)
module.py
def change_config(config):
...
return config
However, an alternative method is to define a value within module.py which will store this information that holds nothing by default. Then as soon as file main.py has imported module.py and the config data is ready, you could assign the data to module.py's config name. Like this:
file main.py
import sys
import mymodule
config = sys.argv[1]
mymodule.config = config
mymodule.test_config()
mymodule.check_config()
mymodule.Module()
module.py
config = None
def test_config():
print config
# this will refer to the value supplied from file main.py
Note however, that the values in the module and main file will not be joined. If you reassign config in file main.py for any reason you have to pass that value to the module again. However if you pass a mutable value like a dict or list then you can modify it in file main.py and the values will be shared.

I don't recommend using a global variable, but here's the design you should use if you do. config needs to be defined in mymodule; after you import the module, you can set the value of mymodule.config the way you are currently setting config.
# file main.py
import sys
import mymodule
mymodule.config = sys.argv[1]
# module.py
# The exact value doesn't matter, as long as we create the name.
# None is good as it conveys the lack of a value; it's part of your
# module's contract, presumably, that a proper value must be assigned
# before you can use the rest of the module.
config = None
def check_config():
# do something with the content of 'config'
class Module(object):
# does something with the content of 'config'

A global variable is almost never the answer. Just allow the functions and classes in your "library" (module.py or mymodule.py, you seem to use both) to accept arguments. So:
mymodule.py
def check_config(configuration):
pass
class Module(object):
def __init__(self, configuration):
self.config = configuration
class ConfigError(Exception):
pass
Then when you want to use them in your "application" code:
main.py
import sys
import mymodule
config = sys.argv[1]
if mymodule.check_config(config):
myobject = mymodule.Module(config)
else:
raise mymodule.ConfigError('Unrecognized configuration format.')

Could you describe that you app should to do? Because now it's not clear, why you want it.
Maybe environment variable could help you?
Btw, you can read config file in one place (module), and import all stuff you need from it.
config.py
import os
if os.environ['sys'] == 'load_1':
import load_1 as load
i = 12
else:
import load_2 as load
i = 13
main.py
import config
config.load("some_data")
print config.i

Best practice for importing modules that contain imports

I tried to break up one large file into three smaller files for organization purposes and now nothing works. I'm sure the problem is with my import statements. Main.py imports Members.py and Members imports from RouteTools. However, even after including
import routeTools
...routeTools.tool()
In Members.py, it doesn't work after
Main.py
import Members
...Members.stuff()
I even tried putting
import routeTools
at the top of Main.py as well. What am I doing wrong or is there a better way to organize one file into multiple modules? Thanks.
Edit: "nothing works" = NameError: global name 'tool' is not defined when running routeTools.tool() from Members.py after it has been imported into Main.py
Here is my code:
import routetools
class Member(object):
def __init__(self, fields, scouts, id):
...
self.routes = [routeTools.Route(s) for s in self.scouts ]
...
And this is called in Main.py:
import Members
import routeTools
...
member = Members.Member(self.fields, self.scouts, i)
routeTools contains:
class Route(object):
...

How do I call a plugin module that's loaded?

Either it's lack of sleep but I feel silly that I can't get this. I have a plugin, I see it get loaded but I can't instantiate it in my main file:
from transformers.FOMIBaseClass import find_plugins, register
find_plugins()
Here's my FOMIBaseClass:
from PluginBase import MountPoint
import sys
import os
class FOMIBaseClass(object):
__metaclass__ = MountPoint
def __init__(self):
pass
def init_plugins(self):
pass
def find_plugins():
plugin_dir = os.path.dirname(os.path.realpath(__file__))
plugin_files = [x[:-3] for x in os.listdir(plugin_dir) if x.endswith("Transformer.py")]
sys.path.insert(0, plugin_dir)
for plugin in plugin_files:
mod = __import__(plugin)
Here's my MountPoint:
class MountPoint(type):
def __init__(cls,name,bases,attrs):
if not hasattr(cls,'plugins'):
cls.plugins = []
else:
cls.plugins.append(cls)
I see it being loaded:
# /Users/carlos/Desktop/ws_working_folder/python/transformers/SctyDistTransformer.pyc matches /Users/carlos/Desktop/ws_working_folder/python/transformers/SctyDistTransformer.py
import SctyDistTransformer # precompiled from /Users/carlos/Desktop/ws_working_folder/python/transformers/SctyDistTransformer.pyc
But, for the life of me, I can't instantiate the 'SctyDistTransformer' module from the main file. I know I'm missing something trivial. Basically, I want to employ a class loading plugin.

To dymically load Python modules from arbitrary folders use imp module:
http://docs.python.org/library/imp.html
Specifically the code should look like:
mod = imp.load_source("MyModule", "MyModule.py")
clz = getattr(mod, "MyClassName")
Also if you are building serious plug-in architecture I recommend using Python eggs and entry points:
http://wiki.pylonshq.com/display/pylonscookbook/Using+Entry+Points+to+Write+Plugins
https://github.com/miohtama/vvv/blob/master/vvv/main.py#L104