My application has a structure similar to this one:
myapp.py
basemod.py
[pkg1]
__init__.py
mod1.py
[pkg2]
__init__.py
mod2.py
myapp.py:
import pkg1
import pkg2
if __name__ == '__main__':
pkg1.main()
pkg2.main()
basemod.py:
import pkg1
def get_msg():
return pkg1.msg
pkg1/__init__.py:
import mod1
msg = None
def main():
global msg
mod1.set_bar()
msg = mod1.bar
pkg1/mod1.py:
bar = None
def set_bar():
global bar
bar = 'Hello World'
pkg2/__init__.py:
import mod2
def main():
mod2.print_foo()
pkg2/mod2.py:
import basemod
foo = basemod.get_msg()
def print_foo():
print(foo)
If I run myapp.py I get:
None
While in my mind I'd expect:
Hello World
My goal is to keep the two packages completely independent from each other, and only communicating through basemod.py, which is a sort of API to pkg1.
I'm starting to think that I have not completely understood how imports among packages work, what am I doing wrong?
Thank you!
Took me a while to read through all that code, but it looks like your problem is in pkg2/mod2.py. The line foo = basemod.get_msg() is executed the first time that file is imported, and never again. So by the time you change the value of mod1.bar, this has already executed, and foo is None.
The solution should simply be to move that line into the print_foo function, so it is only executed when that function is called - which is after the code that sets the relevant value.
Related
I have created two python modules to demonstrate a common problem i seem to have developing python apps.
test_config.py
import do_something
class TestConfig:
_conf = {
"test_case": 'Goodbye',
}
#staticmethod
def get_config(name):
return TestConfig._conf[name]
#staticmethod
def set(name, value):
TestConfig._conf[name] = value
if __name__ == "__main__":
print(TestConfig.get_config('test_case')) # prints Goodbye
TestConfig.set('test_case', 'Hello')
print(TestConfig.get_config('test_case')) # Prints Hello
do_something.say_something() # prints Goodbye?
# Why does do_something print Goodbye- expected Hello
do_something.py
from test_config import TestConfig
def say_something():
text = TestConfig.get_config('test_case')
print(text)
Could someone explain why do_something.say_something() prints out Goodbye and not Hello? And maybe the best way to create a global config class to access global variables. I keep running into similar problems as well as circular imports. I need to set some config variables at runtime.
Let's put this one line:
print(id(TestConfig))
like this:
import do_something
class TestConfig:
_conf = {
"test_case": 'Goodbye',
}
#staticmethod
def get_config(name):
return TestConfig._conf[name]
#staticmethod
def set(name, value):
TestConfig._conf[name] = value
print(id(TestConfig))
if __name__ == "__main__":
print(TestConfig.get_config('test_case')) # prints Goodbye
TestConfig.set('test_case', 'Hello')
print(TestConfig.get_config('test_case')) # Prints Hello
do_something.say_something() # prints Goodbye?
and this
from test_config import TestConfig
print(id(TestConfig))
def say_something():
text = TestConfig.get_config('test_case')
print(text)
We only put 2 lines, so it should print out 2 of same ID right?
1700517329968
1700517329968
1700517331888
Goodbye
Hello
Goodbye
It is in fact called 3 times, and 3rd one actually has different id! So it has different TestConfig.
Logic is:
Main script(test_config.py) imports do_something.py
do_something.py imports test_config.py as module!
(if __name__ == "__main__": is now false so that block doesn't run)
Honestly I am not 100% sure how this doesn't count as circular import, but maybe import to test_config.py didn't import do_something again because it is already imported in namespace or it has some circular import resolving logic.
Suggested fix is, having a separate configuration python script or json/yaml any data format. This kind of circular import is not a good solution!
This question already has answers here:
How to change a module variable from another module?
(3 answers)
Closed 1 year ago.
So I have read that the way to share globals across files is to create a module which holds globals and import this in all python files needing to access the global. However it doesn't seem to work as expected for me (Python 3.6+)
Simple directory structure:
run.py
mypack/
-- globals.py
-- stuff.py
-- __init__.py
I have a global var in globals.py which I need to modify in main file (run.py) and finally print out while exiting the program. It does not seem to work:
__init__.py:
from .stuff import *
globals.py:
test = 'FAIL'
stuff.py:
import atexit
from .globals import test
def cya():
print ("EXIT: test = " + test)
atexit.register(cya)
def hi():
print('HI')
run.py:
import mypack
import mypack.globals as globals
mypack.hi()
globals.test = 'PASS'
print ("MAIN: test = " + globals.test)
Output on script execution:
HI
MAIN: test = PASS
EXIT: test = FAIL
Clearly the exit routine (cya) did not show the correct value of global value that was modified in run.py. Not sure what I am doing wrong.
the python documentation might help you on this one.
https://docs.python.org/3/faq/programming.html#how-do-i-share-global-variables-across-modules
Thanks to #PeterTrcka for pointing out the issue. Also thanks to #buran for indicating globals is a bad name for module since its inbuilt function. Here is the working solution:
directory structure:
run.py
mypack/
-- universal.py
-- stuff.py
-- __init__.py
__init__.py:
from .stuff import *
universal.py:
class GlobalVars:
test = 'FAIL'
stuff.py:
import atexit
from .universal import GlobalVars
def cya():
print ("EXIT: test = " + GlobalVars.test)
atexit.register(cya)
def hi():
print('HI')
run.py:
import mypack
from mypack.universal import GlobalVars
mypack.hi()
GlobalVars.test = 'PASS'
print ("MAIN: test = " + GlobalVars.test)
Output on script execution:
HI
MAIN: test = PASS
EXIT: test = PASS
Issue was: at each import all variables will be reinitialized thier values. Use singleton object:
universal.py
import logging
class GlobVars:
_instances = {}
def __new__(cls, logger, counter_start=0):
if cls not in cls._instances:
print("Creating Instance")
cls._instances[cls] = super(GlobVars, cls).__new__(cls)
return cls._instances[cls]
def __init__(self, logger, counter_start=0):
self.logger = logger
self.counter = counter_start
glob_vars = GlobVars(logger=logging.getLogger("basic_logger"))
run.py
from universal import glob_vars
glob_vars.logger.info("One logger rulles them all")
I have a module I need to test that calls a function on import but I cannot call this function for various reasons. So I am mocking this function but even mocking it calls import.
For example I am testing mod1.py that looks like this:
import os
def bar():
return 'foo'
def dont_call():
os.listdir("C:\\tmp")
dont_call()
And my test looks something like this:
import mock
#mock.patch("mod1.dont_call")
def test_mod1(mock_dont_call):
import mod1
assert mod1.bar()=='foo'
if __name__=="__main__":
test_mod1()
The problem is os.listdir is called.
I cannot change mod1 so what can I do?
I am using python2.7.
To put this in context I am testing a module that opens a database connection on import which I do not agree with but I can see the reasoning behind it. Unfortunately I cannot access this database on my QA machine.
If you want code to 'not' be executed on import put them inside the following condition:
In mod1.py, do the following:
if __name__=="__main__":
dont_call()
This is because, by default when you import a python module, all the code in it gets executed. By adding the above condition, you are explicitly stating that dont_call() is to be called only when the file it run as a script and not when it is imported in other modules.
The workaround I found was to mock what dont_call was calling giving me something like this:
import mock
#mock.patch("os.listdir")
def test_mod1(mock_dont_call):
import mod1
assert mod1.bar()=='foo'
if __name__=="__main__":
test_mod1()
Check your dir
$tree.
test_shot/
├── mod1.py
├── __pycache__
│ └── mod1.cpython-310.pyc
└── test.py
Below code works fine for me.
mod1.py
import os
def bar():
return 'foo'
def dont_call():
os.listdir(".")
def call_this():
print('called this')
call_this()
dont_call()
test.py
import mock
#mock.patch("mod1.dont_call")
def test_mod1(mock_dont_call):
import mod1
assert mod1.bar()=='foo'
if __name__=="__main__":
test_mod1()
Here is output:
$cd test_shot
$python3 test.py
called this
I have two python scripts in different locations. I try to import the functions of the second one in such a way that they are integrated with the globals of the first one. That works fine. However when I call functions in the first script from the second one they cant be found.
foo.py
def run():
glob = {}
loc = {}
execfile("/path/to/bar.py", glob, loc)
currmodule = globals()
currmodule["func_in_bar"] = glob["func_in_bar"]
func_in_bar()
def func_in_foo_A():
print("fooA")
def func_in_foo_B():
print("fooB")
if __name__ == "__main__":
run()
bar.py
def func_in_bar():
func_in_foo_A()
func_in_foo_B()
When foo is run it fails with: NameError: global name 'func_in_foo_A' is not defined
In bar.py you need to add an import foo and then reference foo.func_in_foo(), etc.
Alternatively, use the form: from foo import func_in_foo
I have the following files in my directory:
foo/
foo.py
foolib/
__init__.py
bar.py
Within __init__.py:
__all__ = ["bar"]
Within bar.py:
class Bar:
def __init__(self):
None
def hello(self):
print("Hello World")
return
def hi():
print("Hi World")
Now if I have the following code within foo.py:
from foolib import *
bar.hi()
foobar = Bar()
foobar.hello()
"Hi World" prints, but I get a NameError for Bar(). If I explicitly import the module:
from foolib.bar import *
I get the expected output "Hello World".
Is there a way for me to import classes from the modules, without explicitly calling them? I feel like I am missing something in the __init__ file. Either that or I am flagrantly violating some Python best practice.
To import the class you must import the class, somewhere. When you do from foolib import *, because of your __init__.py this imports the module bar. It doesn't allow you to access anything inside that module.
If you want to automatically access everything in bar from the foolib package without having to import bar, you could put this in __init__.py:
from bar import *
This makes everything in bar available directly in foolib.