I'm a very inexperienced programmer creating a game (using Python 3.3) as a learning exercise. I currently have a main module and a combat module.
The people in the game are represented by instances of class "Person", and are created in the main module. However, the combat module obviously needs access to those objects. Furthermore, I'm probably going to create more modules later that will also need access to those objects.
How do I allow other modules to access the Persons from main.py?
As things stand, main.py has
import combat
at the top; adding
import main
to combat.py doesn't seem to help.
Should I instantiate my objects in a separate module (common.py?) and import them to every module that needs to access them?
Yes, you should factor this out. What you tried is circular imports between your modules, and that typically causes more problems than it solves. If combat imports main and main imports combat, then you may get an error because some object definitions will be missing from main when you try to import them. This is because main will not have finished executing when combat starts executing for the import. Assuming main is your start up script, it should do nothing more than start the program by calling a method from another module; it may instantiate an object if the desired method is an instance method on a class. Avoid global variables, too. Even if it doesn't seem like they'll be a problem now, that can bite you later on.
That said, you can reference members of a module like so:
import common
x = common.some_method_in_common()
y = common.SomeClass()
or
from common import SomeClass
y = SomeClass()
Personally, I generally avoid referencing a method from another module without qualifying it with the module name, but this is also legal:
from common import some_method_in_common
x = some_method_in_common()
I typically use from ... import ... for classes, and I typically use the first form for methods. (Yes, this sometimes means I have specific class imports from a module in addition to importing the module itself.) But this is only my personal convention.
An alternate syntax of which is strongly discouraged is
from common import *
y = SomeClass()
This will import every member of common into the current scope that does not start with an underscore (_). The reason it's discouraged is because it makes identifying the source of the name harder and it makes breaking things too easy. Consider this pair of imports:
from common import *
from some_other_module import *
y = SomeClass()
Which module does SomeClass come from? There's no way to tell other than to go look at the two modules. Worse, what if both modules define SomeClass or SomeClass is later added to some_other_module?
if you have imported main module in combat module by using import main, then you should use main.*(stuff that are implemented in main module) to access classes and methods in there.
example:
import main
person = main.Person()
also you can use from main import * or import Person to avoid main.* in the previous.
There are some rules for importing modules as described in http://effbot.org/zone/import-confusion.htm :
import X imports the module X, and creates a reference to that
module in the current namespace. Or in other words, after you’ve run
this statement, you can use X.name to refer to things defined in
module X.
from X import * imports the module X, and creates references in
the current namespace to all public objects defined by that module
(that is, everything that doesn’t have a name starting with “_”). Or
in other words, after you’ve run this statement, you can simply use
a plain name to refer to things defined in module X. But X itself is
not defined, so X.name doesn’t work. And if name was already
defined, it is replaced by the new version. And if name in X is
changed to point to some other object, your module won’t notice.
from X import a, b, c imports the module X, and creates references
in the current namespace to the given objects. Or in other words,
you can now use a and b and c in your program.
Finally, X = __import__(‘X’) works like import X, with the
difference that you
1) pass the module name as a string, and
2) explicitly assign it to a variable in your current namespace.
Related
I've run into a bit of a wall importing modules in a Python script. I'll do my best to describe the error, why I run into it, and why I'm tying this particular approach to solve my problem (which I will describe in a second):
Let's suppose I have a module in which I've defined some utility functions/classes, which refer to entities defined in the namespace into which this auxiliary module will be imported (let "a" be such an entity):
module1:
def f():
print a
And then I have the main program, where "a" is defined, into which I want to import those utilities:
import module1
a=3
module1.f()
Executing the program will trigger the following error:
Traceback (most recent call last):
File "Z:\Python\main.py", line 10, in <module>
module1.f()
File "Z:\Python\module1.py", line 3, in f
print a
NameError: global name 'a' is not defined
Similar questions have been asked in the past (two days ago, d'uh) and several solutions have been suggested, however I don't really think these fit my requirements. Here's my particular context:
I'm trying to make a Python program which connects to a MySQL database server and displays/modifies data with a GUI. For cleanliness sake, I've defined the bunch of auxiliary/utility MySQL-related functions in a separate file. However they all have a common variable, which I had originally defined inside the utilities module, and which is the cursor object from MySQLdb module.
I later realised that the cursor object (which is used to communicate with the db server) should be defined in the main module, so that both the main module and anything that is imported into it can access that object.
End result would be something like this:
utilities_module.py:
def utility_1(args):
code which references a variable named "cur"
def utility_n(args):
etcetera
And my main module:
program.py:
import MySQLdb, Tkinter
db=MySQLdb.connect(#blahblah) ; cur=db.cursor() #cur is defined!
from utilities_module import *
And then, as soon as I try to call any of the utilities functions, it triggers the aforementioned "global name not defined" error.
A particular suggestion was to have a "from program import cur" statement in the utilities file, such as this:
utilities_module.py:
from program import cur
#rest of function definitions
program.py:
import Tkinter, MySQLdb
db=MySQLdb.connect(#blahblah) ; cur=db.cursor() #cur is defined!
from utilities_module import *
But that's cyclic import or something like that and, bottom line, it crashes too. So my question is:
How in hell can I make the "cur" object, defined in the main module, visible to those auxiliary functions which are imported into it?
Thanks for your time and my deepest apologies if the solution has been posted elsewhere. I just can't find the answer myself and I've got no more tricks in my book.
Globals in Python are global to a module, not across all modules. (Many people are confused by this, because in, say, C, a global is the same across all implementation files unless you explicitly make it static.)
There are different ways to solve this, depending on your actual use case.
Before even going down this path, ask yourself whether this really needs to be global. Maybe you really want a class, with f as an instance method, rather than just a free function? Then you could do something like this:
import module1
thingy1 = module1.Thingy(a=3)
thingy1.f()
If you really do want a global, but it's just there to be used by module1, set it in that module.
import module1
module1.a=3
module1.f()
On the other hand, if a is shared by a whole lot of modules, put it somewhere else, and have everyone import it:
import shared_stuff
import module1
shared_stuff.a = 3
module1.f()
… and, in module1.py:
import shared_stuff
def f():
print shared_stuff.a
Don't use a from import unless the variable is intended to be a constant. from shared_stuff import a would create a new a variable initialized to whatever shared_stuff.a referred to at the time of the import, and this new a variable would not be affected by assignments to shared_stuff.a.
Or, in the rare case that you really do need it to be truly global everywhere, like a builtin, add it to the builtin module. The exact details differ between Python 2.x and 3.x. In 3.x, it works like this:
import builtins
import module1
builtins.a = 3
module1.f()
As a workaround, you could consider setting environment variables in the outer layer, like this.
main.py:
import os
os.environ['MYVAL'] = str(myintvariable)
mymodule.py:
import os
myval = None
if 'MYVAL' in os.environ:
myval = os.environ['MYVAL']
As an extra precaution, handle the case when MYVAL is not defined inside the module.
This post is just an observation for Python behaviour I encountered. Maybe the advices you read above don't work for you if you made the same thing I did below.
Namely, I have a module which contains global/shared variables (as suggested above):
#sharedstuff.py
globaltimes_randomnode=[]
globalist_randomnode=[]
Then I had the main module which imports the shared stuff with:
import sharedstuff as shared
and some other modules that actually populated these arrays. These are called by the main module. When exiting these other modules I can clearly see that the arrays are populated. But when reading them back in the main module, they were empty. This was rather strange for me (well, I am new to Python). However, when I change the way I import the sharedstuff.py in the main module to:
from globals import *
it worked (the arrays were populated).
Just sayin'
A function uses the globals of the module it's defined in. Instead of setting a = 3, for example, you should be setting module1.a = 3. So, if you want cur available as a global in utilities_module, set utilities_module.cur.
A better solution: don't use globals. Pass the variables you need into the functions that need it, or create a class to bundle all the data together, and pass it when initializing the instance.
The easiest solution to this particular problem would have been to add another function within the module that would have stored the cursor in a variable global to the module. Then all the other functions could use it as well.
module1:
cursor = None
def setCursor(cur):
global cursor
cursor = cur
def method(some, args):
global cursor
do_stuff(cursor, some, args)
main program:
import module1
cursor = get_a_cursor()
module1.setCursor(cursor)
module1.method()
Since globals are module specific, you can add the following function to all imported modules, and then use it to:
Add singular variables (in dictionary format) as globals for those
Transfer your main module globals to it
.
addglobals = lambda x: globals().update(x)
Then all you need to pass on current globals is:
import module
module.addglobals(globals())
Since I haven't seen it in the answers above, I thought I would add my simple workaround, which is just to add a global_dict argument to the function requiring the calling module's globals, and then pass the dict into the function when calling; e.g:
# external_module
def imported_function(global_dict=None):
print(global_dict["a"])
# calling_module
a = 12
from external_module import imported_function
imported_function(global_dict=globals())
>>> 12
The OOP way of doing this would be to make your module a class instead of a set of unbound methods. Then you could use __init__ or a setter method to set the variables from the caller for use in the module methods.
Update
To test the theory, I created a module and put it on pypi. It all worked perfectly.
pip install superglobals
Short answer
This works fine in Python 2 or 3:
import inspect
def superglobals():
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals
save as superglobals.py and employ in another module thusly:
from superglobals import *
superglobals()['var'] = value
Extended Answer
You can add some extra functions to make things more attractive.
def superglobals():
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals
def getglobal(key, default=None):
"""
getglobal(key[, default]) -> value
Return the value for key if key is in the global dictionary, else default.
"""
_globals = dict(inspect.getmembers(
inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
return _globals.get(key, default)
def setglobal(key, value):
_globals = superglobals()
_globals[key] = value
def defaultglobal(key, value):
"""
defaultglobal(key, value)
Set the value of global variable `key` if it is not otherwise st
"""
_globals = superglobals()
if key not in _globals:
_globals[key] = value
Then use thusly:
from superglobals import *
setglobal('test', 123)
defaultglobal('test', 456)
assert(getglobal('test') == 123)
Justification
The "python purity league" answers that litter this question are perfectly correct, but in some environments (such as IDAPython) which is basically single threaded with a large globally instantiated API, it just doesn't matter as much.
It's still bad form and a bad practice to encourage, but sometimes it's just easier. Especially when the code you are writing isn't going to have a very long life.
What is the equivalent of import * in Python using functions (presumably from importlib)?
I know that you can import a module with mod = __import__(...), which will delegate to whatever the currently configured implementation is. You can also do something like
mod_spec = importlib.utl.spec_from_file_location(...)
mod = importlib.util.module_from_spec(mod_spec)
mod_spec.loader.exec_module(mod)
which allows you to do crazy things like injecting things into the module by inserting them before the call to exec_module. (Courtesy of https://stackoverflow.com/a/67692/2988730 and https://stackoverflow.com/a/38650878/2988730)
However, my question remains. How does import * work in function form? What function determines which names to load from a module depending on the presence/contents of __all__?
There's no function for from whatever import *. In fact, there's no function for import whatever, either! When you do
mod = __import__(...)
the __import__ function is only responsible for part of the job. It provides you with a module object, but you have to assign that module object to a variable separately. There's no function that will import a module and assign it to a variable the way import whatever does.
In from whatever import *, there are two parts:
prepare the module object for whatever
assign variables
The "prepare the module object" part is almost identical to in import whatever, and it can be handled by the same function, __import__. There's a minor difference in that import * will load any not-yet-loaded submodules in a package's __all__ list; __import__ will handle this for you if you provide fromlist=['*']:
module = __import__('whatever', fromlist=['*'])
The part about assigning names is where the big differences occur, and again, you have to handle that yourself. It's fairly straightforward, as long as you're at global scope:
if hasattr(module, '__all__'):
all_names = module.__all__
else:
all_names = [name for name in dir(module) if not name.startswith('_')]
globals().update({name: getattr(module, name) for name in all_names})
Function scopes don't support assigning variables determined at runtime.
Can someone explain the logic behind how this works with the Python interpreter? Is this behavior only thread local? Why does the assignment in the first module import persist after the second module import? I just had a long debugging session that came down to this.
external_library.py
def the_best():
print "The best!"
modify_external_library.py
import external_library
def the_best_2():
print "The best 2!"
external_library.the_best = the_best_2
main.py
import modify_external_library
import external_library
external_library.the_best()
Test:
$ python main.py
The best 2!
Nothing thread-local about this. somemodule.anattr = avalue is very global behavior! After this assignment the attribute is changed for good (until maybe changed back later) no matter what.
There's no mysterious mechanics at play! Assignment to any attribute of an object that allows such assignment (as module objects do) just work in the obvious way -- no thread-local anything, nothing strange -- and assignment to attribute persists, as long as the object whose attribute you've assigned persists, of course.
The repeated import external_library doesn't reload the module (reload is a totally separate builtin and import does not call it!) -- it just checks sys.modules, finds an external_library key in that dict, and binds the corresponding value (which was previously modified by that assignment) to name external_library in the appropriate namespace (here, globals of module main).
As Alex indicated you need to reload external_library, simply importing it will do nothing if it's already been imported. You can check that by putting print statements into your external_library and modify_external_library modules.
import modify_external_library
#import external_library
reload(external_library)
external_library.the_best()
output
The best!
Modules are instances of new-style classes. When you modify the attributes of a module (the function in this case), you are modifying the module instance. When you try to import it again (with import external_library), you're just getting the same module object already referenced inside of modify_external_library.py.
Edit: Of course, trying to import the same module again does not really work (as Alex Martelli points out). Once loaded, modules are not re-initialized unless done so explicitly with reload.
Monkey patching works because classes are modifiable in python but the mechanism that allows it to spread like this is that once any module has been imported, and initialised, later imports simply add the existing instance to the local namespace without rerunning the initialisation, this also saves time when a module has a lot of initialisation as well as allowing monkey patches.
I can make this code work, but I am still confused why it won't work the first way I tried.
I am practicing python because my thesis is going to be coded in it (doing some cool things with Arduino and PC interfaces). I'm trying to import a class from another file into my main program so that I can create objects. Both files are in the same directory. It's probably easier if you have a look at the code at this point.
#from ArduinoBot import *
#from ArduinoBot import ArduinoBot
import ArduinoBot
# Create ArduinoBot object
bot1 = ArduinoBot()
# Call toString inside bot1 object
bot1.toString()
input("Press enter to end.")
Here is the very basic ArduinoBot class
class ArduinoBot:
def toString(self):
print ("ArduinoBot toString")
Either of the first two commented out import statements will make this work, but not the last one, which to me seems the most intuitive and general. There's not a lot of code for stuff to go wrong here, it's a bit frustrating to be hitting these kind of finicky language specific quirks when I had heard some many good things about Python. Anyway I must be doing something wrong, but why doesn't the simple 'import ClassName' or 'import FileName' work?
Thank you for your help.
consider a file (example.py):
class foo(object):
pass
class bar(object):
pass
class example(object):
pass
Now in your main program, if you do:
import example
what should be imported from the file example.py? Just the class example? should the class foo come along too? The meaning would be too ambiguous if import module pulled the whole module's namespace directly into your current namespace.
The idea is that namespaces are wonderful. They let you know where the class/function/data came from. They also let you group related things together (or equivalently, they help you keep unrelated things separate!). A module sets up a namespace and you tell python exactly how you want to bring that namespace into the current context (namespace) by the way you use import.
from ... import * says -- bring everything in that module directly into my namespace.
from ... import ... as ... says, bring only the thing that I specify directly into my namespace, but give it a new name here.
Finally, import ... simply says bring that module into the current namespace, but keep it separate. This is the most common form in production code because of (at least) 2 reasons.
It prevents name clashes. You can have a local class named foo which won't conflict with the foo in example.py -- You get access to that via example.foo
It makes it easy to trace down which module a class came from for debugging.
consider:
from foo import *
from bar import *
a = AClass() #did this come from foo? bar? ... Hmmm...
In this case, to get access to the class example from example.py, you could also do:
import example
example_instance = example.example()
but you can also get foo:
foo_instance = example.foo()
The simple answer is that modules are things in Python. A module has its own status as a container for classes, functions, and other objects. When you do import ArduinoBot, you import the module. If you want things in that module -- classes, functions, etc. -- you have to explicitly say that you want them. You can either import them directly with from ArduinoBot import ..., or access them via the module with import ArduinoBot and then ArduinoBot.ArduinoBot.
Instead of working against this, you should leverage the container-ness of modules to allow you to group related stuff into a module. It may seem annoying when you only have one class in a file, but when you start putting multiple classes and functions in one file, you'll see that you don't actually want all that stuff being automatically imported when you do import module, because then everything from all modules would conflict with other things. The modules serve a useful function in separating different functionality.
For your example, the question you should ask yourself is: if the code is so simple and compact, why didn't you put it all in one file?
Import doesn't work quite the you think it does. It does work the way it is documented to work, so there's a very simple remedy for your problem, but nonetheless:
import ArduinoBot
This looks for a module (or package) on the import path, executes the module's code in a new namespace, and then binds the module object itself to the name ArduinoBot in the current namespace. This means a module global variable named ArduinoBot in the ArduinoBot module would now be accessible in the importing namespace as ArduinoBot.ArduinoBot.
from ArduinoBot import ArduinoBot
This loads and executes the module as above, but does not bind the module object to the name ArduinoBot. Instead, it looks for a module global variable ArduinoBot within the module, and binds whatever object that referred to the name ArduinoBot in the current namespace.
from ArduinoBot import *
Similarly to the above, this loads and executes a module without binding the module object to any name in the current namespace. It then looks for all module global variables, and binds them all to the same name in the current namespace.
This last form is very convenient for interactive work in the python shell, but generally considered bad style in actual development, because it's not clear what names it actually binds. Considering it imports everything global in the imported module, including any names that it imported at global scope, it very quickly becomes extremely difficult to know what names are in scope or where they came from if you use this style pervasively.
The module itself is an object. The last approach does in fact work, if you access your class as a member of the module. Either if the following will work, and either may be appropriate, depending on what else you need from the imported items:
from my_module import MyClass
foo = MyClass()
or
import my_module
foo = my_module.MyClass()
As mentioned in the comments, your module and class usually don't have the same name in python. That's more a Java thing, and can sometimes lead to a little confusion here.
mod1.py
import mod2
class Universe:
def __init__(self):
pass
def answer(self):
return 42
u = Universe()
mod2.show_answer(u)
mod2.py
#import mod1 -- not necessary
def show_answer(thing):
print thing.answer()
Coming from a C++ background I had the feeling it was necessary to import the module containing the Universe class definition before the show_answer function would work. I.e. everything had to be declared before it could be used.
Am I right in thinking this isn't necessary? This is duck typing, right? So if an import isn't required to see the methods of a class, I'd at least need it for the class definition itself and the top level functions of a module?
In one script I've written, I even went as far as writing a base class to declare an interface with a set of methods, and then deriving concrete classes to inherit that interface, but I think I get it now - that's just wrong in Python, and whether an object has a particular method is checked at runtime at the point where the call is made?
I realise Python is so much more dynamic than C++, it's taken me a while to see how little code you actually need to write!
I think I know the answer to this question, but I just wanted to get clarification and make sure I was on the right track.
UPDATE: Thanks for all the answers, I think I should clarify my question now:
Does mod2.show_answer() need an import (of any description) to know that thing has a method called answer(), or is that determined dynamically at runtime?
In this case you're right: show_answer() is given an object, of which it calls the method "answer". As long as the object given to show_answer() has such a method, it doesn't matter where the object comes from.
If, however, you wanted to create an instance of Universe inside mod2, you'd have to import mod1, because Universe is not in the mod2 namespace, even after mod2 has been imported by mod1.
import is all about names -- mostly "bare names" that are bound at top level (AKA global level, AKA module-level names) in a certain module, say mod2. When you've done import mod2, you get the mod2 namespace as an available name (top-level in your own module, if you're doing the import itself as top level, as is most common; but a local import within a function would make mod2 a local variable of that function, etc); and therefore you can use mod2.foobar to access the name foobar that's bound at top level in mod2. If you have no need to access such names, then you have no need to import mod2 in your own module.
Think of import being more like the linker.
With "import mod2" you are simply telling python that it can find the function in the file mod2.py
Actually, in this case, importing mod1 in mod2.py should not work.
Would it not create a circular reference?
In fact, according to this explanation , the circular import will not work the way you want it to work: if you uncomment import mod1, the second module will still not know about the Universe.
I think this is quite reasonable. If both of your files need access to the type of some specific object, like Universe, you have several choices:
if your program is small, just use one file
if it's big, you need to decide if your files both need to know how Universe is implemented, perhaps passing an object of not-yet-known type to show_answer is fine
if that doesn't work for you, by all means put Universe in a separate module and load it first.
import in Python loads the module into the given namespace. As such, is it as if the def show_answer actually existed in the mod1.py module. Because of this, mod2.py does not need to know of the Universe class and thus you do not need to import mod1 from mod2.py.
I don't know much about C++, so can't directly compare it, but..
import basically loads the other Python script (mod2.py) into the current script (the top level of mod1.py). It's not so much a link, it's closer to an eval
For example, in Python'ish psuedo-code:
eval("mod2.py")
is the same as..
from mod2 import *
..it executes mod2.py, and makes the functions/classes defined accessible in the current script.
Both above snippets would allow you to call show_answer() (well, eval doesn't quite work like that, thus I called it pseudo code!)
import mod2
..is basically the same, but instead of bringing in all the functions into the "top level", it brings them into the mod2 module, so you call show_answer by doing..
mod2.show_answer
Am I right in thinking [the import in mod2.py] isn't necessary?
Absolutely. In fact if you try and import mod1 from mod2 you get a circular dependancy error (since mod2 then tries to import mod1 and so on..)