This maybe a very naive question and perhaps it's best asked with an example:
module1.py
import module2
def new_func():
print(var_string)
module2.new_func = new_func
module2.func()
module2.new_func()
module2.py
var_string = "i'm the global string!"
def func():
print(var_string)
result
> python module1.py
i'm the global string!
Traceback (most recent call last):
File "module1.py", line 8, in <module>
module2.new_func()
File "module1.py", line 4, in new_func
print(var_string)
NameError: name 'var_string' is not defined
So my question is this:
Is it possible to insert a function into a module and have it's global namespace update accordingly?
Related: global variable defined in main script can't be accessed by a function defined in a different module
Note that I am aware that sharing global variables is a bad idea and I'm also aware that a configuration module would be a good compromise to that but also please note that this is not what I am trying to achieve.
You may think it is useful, but very little python code is written this way and I think most python programmers would be confused with code that does this. Modifying a module after it's been imported (monkeypatching) is usually looked down upon because it's very easy to do the wrong thing and cause strange bugs.
You made an analogy comparing it to overriding / extending methods on a class, but if this is really what you want to do, why not just use a class? The features of classes make it much safer and easier to do this kind of thing.
Your code will work if you do this:
from module2 import var_string
#or..
from module2 import *
But I'm not sure if that's the solution you're looking for. Either way, I personally wouldn't try to make this code work, it's fighting against the way python code is normally written. If you have a practical example of code that you think would be improved by dynamically modifying modules, I would like to see it. It's a little hard to see the benefit with the example code you gave.
I'm not understand what you want, and what this string must do "module2.new_func = new_func", because you dont have the function new_funcin module2.
But if you want to reseting variable in each modules, you cant use like this :
Module1 :
import module2
def new_func():
print(var_string)
new_class=module2.MyStuff()
var_string=new_class.func()
new_func()
Module2:
class MyStuff:
def __init__(self):
self.var_string = "i'm the global string!"
def func(self):
print(self.var_string)
return self.var_string
Related
As i was going through python basics and introduction one thing that really confuses me is namespaces. I cant seem to understand how they work.
For instance in Cs50 course they brush over the concept but i wanted to get clearer understanding of it if possible because it seems very important to grasp. For example this code:
import cs50
x = get_int("x: ")
y = get_int("y: ")
print(x + y)
Causes this error:
python calculator.py
Traceback (most recent call last):
File "/workspaces/20377622/calculator.py", line 3, in
x = get_int("x: ")
NameError: name 'get_int' is not defined
What is wonder is why when cs50.get_int() is written instead interpreter doesn't throw an error? Is it because cs50 library is seen as its own namespace structure and . goes into that structure to get_int location? What does . operator do exactly here in terms of namespaces ?
You import cs50, so you have a name "cs50", you can use
cs50.get_int()
but namespaces has no name get_int.
You can use from cs50 import get_int to import name "get_int"
To answer this question, let's talk about modules.
In Python, "module" is used to refer to two things. First, a piece of code, usually a .py file. Second, the object that is created for the namespace of that code when it is run.
When you do import foo, a couple of things happen.
Python checks if foo has already been imported. If so, it skips to the last step.
Python looks up where it can find foo, for example if there is a foo.py in the right place.
Python creates a fresh namespace object and stores a reference to it in sys.modules['foo']
Python runs the code that it found, using that namespace object as its "global" namespace.
Python goes back to the importing module, and adds a global variable foo in the namespace of importing module that points to sys.modules['foo'].
You can then access any global variable bar that was created in the foo.py module by using foo.bar.
You could also use from cs50 import get_int which works like this:
import cs50
get_int = cs50.get_int
... except that the name cs50 is not assigned to.
If you're asking why it works that way: this way different modules can define the same name for classes, functions or constants, and they won't conflict with each other.
Now, if you know you're only using get_int from cs50, and you don't have any other get_int either in your main module or that you imported, the from ... import syntax is very useful to not have to write cs50.get_int every time you want to use it.
I've run into a bit of a wall importing modules in a Python script. I'll do my best to describe the error, why I run into it, and why I'm tying this particular approach to solve my problem (which I will describe in a second):
Let's suppose I have a module in which I've defined some utility functions/classes, which refer to entities defined in the namespace into which this auxiliary module will be imported (let "a" be such an entity):
module1:
def f():
print a
And then I have the main program, where "a" is defined, into which I want to import those utilities:
import module1
a=3
module1.f()
Executing the program will trigger the following error:
Traceback (most recent call last):
File "Z:\Python\main.py", line 10, in <module>
module1.f()
File "Z:\Python\module1.py", line 3, in f
print a
NameError: global name 'a' is not defined
Similar questions have been asked in the past (two days ago, d'uh) and several solutions have been suggested, however I don't really think these fit my requirements. Here's my particular context:
I'm trying to make a Python program which connects to a MySQL database server and displays/modifies data with a GUI. For cleanliness sake, I've defined the bunch of auxiliary/utility MySQL-related functions in a separate file. However they all have a common variable, which I had originally defined inside the utilities module, and which is the cursor object from MySQLdb module.
I later realised that the cursor object (which is used to communicate with the db server) should be defined in the main module, so that both the main module and anything that is imported into it can access that object.
End result would be something like this:
utilities_module.py:
def utility_1(args):
code which references a variable named "cur"
def utility_n(args):
etcetera
And my main module:
program.py:
import MySQLdb, Tkinter
db=MySQLdb.connect(#blahblah) ; cur=db.cursor() #cur is defined!
from utilities_module import *
And then, as soon as I try to call any of the utilities functions, it triggers the aforementioned "global name not defined" error.
A particular suggestion was to have a "from program import cur" statement in the utilities file, such as this:
utilities_module.py:
from program import cur
#rest of function definitions
program.py:
import Tkinter, MySQLdb
db=MySQLdb.connect(#blahblah) ; cur=db.cursor() #cur is defined!
from utilities_module import *
But that's cyclic import or something like that and, bottom line, it crashes too. So my question is:
How in hell can I make the "cur" object, defined in the main module, visible to those auxiliary functions which are imported into it?
Thanks for your time and my deepest apologies if the solution has been posted elsewhere. I just can't find the answer myself and I've got no more tricks in my book.
Globals in Python are global to a module, not across all modules. (Many people are confused by this, because in, say, C, a global is the same across all implementation files unless you explicitly make it static.)
There are different ways to solve this, depending on your actual use case.
Before even going down this path, ask yourself whether this really needs to be global. Maybe you really want a class, with f as an instance method, rather than just a free function? Then you could do something like this:
import module1
thingy1 = module1.Thingy(a=3)
thingy1.f()
If you really do want a global, but it's just there to be used by module1, set it in that module.
import module1
module1.a=3
module1.f()
On the other hand, if a is shared by a whole lot of modules, put it somewhere else, and have everyone import it:
import shared_stuff
import module1
shared_stuff.a = 3
module1.f()
… and, in module1.py:
import shared_stuff
def f():
print shared_stuff.a
Don't use a from import unless the variable is intended to be a constant. from shared_stuff import a would create a new a variable initialized to whatever shared_stuff.a referred to at the time of the import, and this new a variable would not be affected by assignments to shared_stuff.a.
Or, in the rare case that you really do need it to be truly global everywhere, like a builtin, add it to the builtin module. The exact details differ between Python 2.x and 3.x. In 3.x, it works like this:
import builtins
import module1
builtins.a = 3
module1.f()
As a workaround, you could consider setting environment variables in the outer layer, like this.
main.py:
import os
os.environ['MYVAL'] = str(myintvariable)
mymodule.py:
import os
myval = None
if 'MYVAL' in os.environ:
myval = os.environ['MYVAL']
As an extra precaution, handle the case when MYVAL is not defined inside the module.
This post is just an observation for Python behaviour I encountered. Maybe the advices you read above don't work for you if you made the same thing I did below.
Namely, I have a module which contains global/shared variables (as suggested above):
#sharedstuff.py
globaltimes_randomnode=[]
globalist_randomnode=[]
Then I had the main module which imports the shared stuff with:
import sharedstuff as shared
and some other modules that actually populated these arrays. These are called by the main module. When exiting these other modules I can clearly see that the arrays are populated. But when reading them back in the main module, they were empty. This was rather strange for me (well, I am new to Python). However, when I change the way I import the sharedstuff.py in the main module to:
from globals import *
it worked (the arrays were populated).
Just sayin'
A function uses the globals of the module it's defined in. Instead of setting a = 3, for example, you should be setting module1.a = 3. So, if you want cur available as a global in utilities_module, set utilities_module.cur.
A better solution: don't use globals. Pass the variables you need into the functions that need it, or create a class to bundle all the data together, and pass it when initializing the instance.
The easiest solution to this particular problem would have been to add another function within the module that would have stored the cursor in a variable global to the module. Then all the other functions could use it as well.
module1:
cursor = None
def setCursor(cur):
global cursor
cursor = cur
def method(some, args):
global cursor
do_stuff(cursor, some, args)
main program:
import module1
cursor = get_a_cursor()
module1.setCursor(cursor)
module1.method()
Since globals are module specific, you can add the following function to all imported modules, and then use it to:
Add singular variables (in dictionary format) as globals for those
Transfer your main module globals to it
.
addglobals = lambda x: globals().update(x)
Then all you need to pass on current globals is:
import module
module.addglobals(globals())
Since I haven't seen it in the answers above, I thought I would add my simple workaround, which is just to add a global_dict argument to the function requiring the calling module's globals, and then pass the dict into the function when calling; e.g:
# external_module
def imported_function(global_dict=None):
print(global_dict["a"])
# calling_module
a = 12
from external_module import imported_function
imported_function(global_dict=globals())
>>> 12
The OOP way of doing this would be to make your module a class instead of a set of unbound methods. Then you could use __init__ or a setter method to set the variables from the caller for use in the module methods.
Update
To test the theory, I created a module and put it on pypi. It all worked perfectly.
pip install superglobals
Short answer
This works fine in Python 2 or 3:
import inspect
def superglobals():
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals
save as superglobals.py and employ in another module thusly:
from superglobals import *
superglobals()['var'] = value
Extended Answer
You can add some extra functions to make things more attractive.
def superglobals():
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals
def getglobal(key, default=None):
"""
getglobal(key[, default]) -> value
Return the value for key if key is in the global dictionary, else default.
"""
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals.get(key, default)
def setglobal(key, value):
_globals = superglobals()
_globals[key] = value
def defaultglobal(key, value):
"""
defaultglobal(key, value)
Set the value of global variable `key` if it is not otherwise st
"""
_globals = superglobals()
if key not in _globals:
_globals[key] = value
Then use thusly:
from superglobals import *
setglobal('test', 123)
defaultglobal('test', 456)
assert(getglobal('test') == 123)
Justification
The "python purity league" answers that litter this question are perfectly correct, but in some environments (such as IDAPython) which is basically single threaded with a large globally instantiated API, it just doesn't matter as much.
It's still bad form and a bad practice to encourage, but sometimes it's just easier. Especially when the code you are writing isn't going to have a very long life.
I am trying to import a module from inside a function and have it be available to my whole file the same way it would be if I imported outside any functions and before all the other code. The reason it is in a function is because I don't have much control over the structure of the script. Is this possible without resorting to things like hacking __builtin__ or passing what I need all around my code?
How about something like globals()["os"] = __import__("os")?
I guess this could be wrapped in a generic function if you wanted since the module name is a string.
Seeing your new comments, I want to emphasize that this sounds unnecessary. You're actually modifying the script more by importing within a function than by importing at the top of the script in the normal way. Still, in the spirit of answering the question asked, I'm leaving my previous answer.
I'm honestly not certain this is the correct way to do this, but a quick check confirms that if you declare the module name as global within the function before importing, it is imported into the global namespace.
>>> def import_re():
... global re
... import re
...
>>> re
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 're' is not defined
>>> import_re()
>>> re
<module 're' from '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/re.pyc'>
Don't do this unless you really have to -- and then write it in big red letters, so to speak.
I am in a project where we are starting refactoring some massive code base. One problem that immediately sprang up is that each file imports a lot of other files. How do I in an elegant way mock this in my unit test without having to alter the actual code so I can start to write unit-tests?
As an example: The file with the functions I want to test, imports ten other files which is part of our software and not python core libs.
I want to be able to run the unit tests as separately as possible and for now I am only going to test functions that does not depend on things from the files that are being imported.
Thanks for all the answers.
I didn't really know what I wanted to do from the start but now I think I know.
Problem was that some imports was only possible when the whole application was running because of some third-party auto-magic. So I had to make some stubs for these modules in a directory which I pointed out with sys.path
Now I can import the file which contains the functions I want to write tests for in my unit-test file without complaints about missing modules.
If you want to import a module while at the same time ensuring that it doesn't import anything, you can replace the __import__ builtin function.
For example, use this class:
class ImportWrapper(object):
def __init__(self, real_import):
self.real_import = real_import
def wrapper(self, wantedModules):
def inner(moduleName, *args, **kwargs):
if moduleName in wantedModules:
print "IMPORTING MODULE", moduleName
self.real_import(*args, **kwargs)
else:
print "NOT IMPORTING MODULE", moduleName
return inner
def mock_import(self, moduleName, wantedModules):
__builtins__.__import__ = self.wrapper(wantedModules)
try:
__import__(moduleName, globals(), locals(), [], -1)
finally:
__builtins__.__import__ = self.real_import
And in your test code, instead of writing import myModule, write:
wrapper = ImportWrapper(__import__)
wrapper.mock_import('myModule', [])
The second argument to mock_import is a list of module names you do want to import in inner module.
This example can be modified further to e.g. import other module than desired instead of just not importing it, or even mocking the module object with some custom object of your own.
If you really want to muck around with the python import mechanism, take a look at the ihooks module. It provides tools for changing the behavior of the __import__ built-in. But it's not clear from your question why you need to do this.
"imports a lot of other files"? Imports a lot of other files that are part of your customized code base? Or imports a lot of other files that are part of the Python distribution? Or imports a lot of other open source project files?
If your imports don't work, you have a "simple" PYTHONPATH problem. Get all of your various project directories onto a PYTHONPATH that you can use for testing. We have a rather complex path, in Windows we manage it like this
#set Part1=c:\blah\blah\blah
#set Part2=c:\some\other\path
#set that=g:\shared\stuff
set PYTHONPATH=%part1%;%part2%;%that%
We keep each piece of the path separate so that we (a) know where things come from and (b) can manage change when we move things around.
Since the PYTHONPATH is searched in order, we can control what gets used by adjusting the order on the path.
Once you have "everything", it becomes a question of trust.
Either
you trust something (i.e., the Python code base) and just import it.
Or
You don't trust something (i.e., your own code) and you
test it separately and
mock it for stand-alone testing.
Would you test the Python libraries? If so, you've got a lot of work. If not, then, you should perhaps only mock out the things you're actually going to test.
No difficult manipulation is necessary if you want a quick-and-dirty fix before your unit-tests.
If the unit tests are in the same file as the code you wish to test, simply delete unwanted module from the globals() dictionary.
Here is a rather lengthy example: suppose you have a module impp.py with contents:
value = 5
Now, in your test file you can write:
>>> import impp
>>> print globals().keys()
>>> def printVal():
>>> print impp.value
['printVal', '__builtins__', '__file__', 'impp', '__name__', '__doc__']
Note that impp is among the globals, because it was imported. Calling the function printVal that uses impp module still works:
>>> printVal()
5
But now, if you remove impp key from globals()...
>>> del globals()['impp']
>>> print globals().keys()
['printVal', '__builtins__', '__file__', '__name__', '__doc__']
...and try to call printVal(), you'll get:
>>> printVal()
Traceback (most recent call last):
File "test_imp.py", line 13, in <module>
printVal()
File "test_imp.py", line 5, in printVal
print impp.value
NameError: global name 'impp' is not defined
...which is probably exactly what you're trying to achieve.
To use it in your unit-tests, you can delete the globals just before running the test suite, e.g. in __main__:
if __name__ == '__main__':
del globals()['impp']
unittest.main()
In your comment above, you say you want to convince python that certain modules have already been imported. This still seems like a strange goal, but if that's really what you want to do, in principle you can sneak around behind the import mechanism's back, and change sys.modules. Not sure how this'd work for package imports, but should be fine for absolute imports.
I have a file called main.py and a file called classes.py
main.py contains the application and what's happening while class.py contains some classes.
main.py has the following code
main.py
import classes
def addItem(text):
print text
myClass = classes.ExampleClass()
And then we have classes.py
classes.py
class ExampleClass (object):
def __init__(self):
addItem('bob')
Surprisingly enough that's not the actual code I am using because I've stripped out anything that'd get in the way of you seeing what I want to do. I want to be able to call a method that's defined in main.py from a class within classes.py. How do I do this?
Thanks in advance
I couldn't answer this any better than this post by Alex Martelli. Basically any way you try to do this will lead to trouble and you are much better off refactoring the code to avoid mutual dependencies between two modules...
If you have two modules A and B which depend on each other, the easiest way is to isolate a part of the code that they both depend on into a third module C, and have both of them import C.
The suggestions to refactor are good ones. If you have to leave the files as they are, then you can edit main.py to make sure that nothing is executed simply by importing the file, then import main in the function that needs it:
class ExampleClass (object):
def __init__(self):
import main
main.addItem('bob')
This avoids the circular imports, but isn't as nice as refactoring in the first place...
I would suggest putting common functions either in classes.py, or probably even better in a third module, perhaps utils.py.
All your executable code should be inside a if __name__ == "__main__" . This will prevent it from being execucted when imported as a module. In main.py
if __name__=="__main__":
myClass = classes.ExampleClass()
However, as dF states, it is probably better to refactor at this stage than to try to resolve cyclic dependencies.