If I wanted to get the current module, e.g. to reload it, I would do:
import sys
sys.modules[__name__]
Is there a better way to do this (e.g. not involving __name__)? Better in this context means more idiomatic, more portable, more robust, or more...any of the other things we usually desire in our software.
I use python 2, but answers for python 3 will no doubt be useful to others.
There is no more idiomatic method to get the current module object from sys.modules than what you used.
__name__ is set by Python on import, essentially doing:
module_object = import_py_file(import_name)
module_object.__name__ = import_name
sys.modules[import_name] = module_object
so the __name__ reference is exactly what you want to use here.
Related
I'd like to dynamically create a module from a dictionary, and I'm wondering if adding an element to sys.modules is really the best way to do this. EG
context = { a: 1, b: 2 }
import types
test_context_module = types.ModuleType('TestContext', 'Module created to provide a context for tests')
test_context_module.__dict__.update(context)
import sys
sys.modules['TestContext'] = test_context_module
My immediate goal in this regard is to be able to provide a context for timing test execution:
import timeit
timeit.Timer('a + b', 'from TestContext import *')
It seems that there are other ways to do this, since the Timer constructor takes objects as well as strings. I'm still interested in learning how to do this though, since a) it has other potential applications; and b) I'm not sure exactly how to use objects with the Timer constructor; doing so may prove to be less appropriate than this approach in some circumstances.
EDITS/REVELATIONS/PHOOEYS/EUREKA:
I've realized that the example code relating to running timing tests won't actually work, because import * only works at the module level, and the context in which that statement is executed is that of a function in the testit module. In other words, the globals dictionary used when executing that code is that of __main__, since that's where I was when I wrote the code in the interactive shell. So that rationale for figuring this out is a bit botched, but it's still a valid question.
I've discovered that the code run in the first set of examples has the undesirable effect that the namespace in which the newly created module's code executes is that of the module in which it was declared, not its own module. This is like way weird, and could lead to all sorts of unexpected rattlesnakeic sketchiness. So I'm pretty sure that this is not how this sort of thing is meant to be done, if it is in fact something that the Guido doth shine upon.
The similar-but-subtly-different case of dynamically loading a module from a file that is not in python's include path is quite easily accomplished using imp.load_source('NewModuleName', 'path/to/module/module_to_load.py'). This does load the module into sys.modules. However this doesn't really answer my question, because really, what if you're running python on an embedded platform with no filesystem?
I'm battling a considerable case of information overload at the moment, so I could be mistaken, but there doesn't seem to be anything in the imp module that's capable of this.
But the question, essentially, at this point is how to set the global (ie module) context for an object. Maybe I should ask that more specifically? And at a larger scope, how to get Python to do this while shoehorning objects into a given module?
Hmm, well one thing I can tell you is that the timeit function actually executes its code using the module's global variables. So in your example, you could write
import timeit
timeit.a = 1
timeit.b = 2
timeit.Timer('a + b').timeit()
and it would work. But that doesn't address your more general problem of defining a module dynamically.
Regarding the module definition problem, it's definitely possible and I think you've stumbled on to pretty much the best way to do it. For reference, the gist of what goes on when Python imports a module is basically the following:
module = imp.new_module(name)
execfile(file, module.__dict__)
That's kind of the same thing you do, except that you load the contents of the module from an existing dictionary instead of a file. (I don't know of any difference between types.ModuleType and imp.new_module other than the docstring, so you can probably use them interchangeably) What you're doing is somewhat akin to writing your own importer, and when you do that, you can certainly expect to mess with sys.modules.
As an aside, even if your import * thing was legal within a function, you might still have problems because oddly enough, the statement you pass to the Timer doesn't seem to recognize its own local variables. I invoked a bit of Python voodoo by the name of extract_context() (it's a function I wrote) to set a and b at the local scope and ran
print timeit.Timer('print locals(); a + b', 'sys.modules["__main__"].extract_context()').timeit()
Sure enough, the printout of locals() included a and b:
{'a': 1, 'b': 2, '_timer': <built-in function time>, '_it': repeat(None, 999999), '_t0': 1277378305.3572791, '_i': None}
but it still complained NameError: global name 'a' is not defined. Weird.
As a beginner, what I understood is that Python Standard Library (PSL) provides a lot of modules which provide a lot of functionalities, but still if I want to use those then I have to import the module, for example, sys, os etc. are PSL modules but still those need to be imported.
Now, I wonder if that is the case then how without importing anything I am able to use functions like print, list, len etc.? Is it that their "support is built-in into the interpreter"?
Yes. They're built-in functions (or in the case of list, a built-in class). You can explicitly import the __builtin__ module (Py2) or the builtins module (Py3) if you want qualified access to the names, but by default, those modules are searched whenever an attempt to access a global name doesn't find the name in the module globals. They're not normally needed though, per the docs:
This module is not normally accessed explicitly by most applications, but can be useful in modules that provide objects with the same name as a built-in value, but in which the built-in of that name is also needed.
The print function comes from the builtins module.
You can find its documentation here.
Here is an example session.
I first check what module print comes from,which is stored in its __module__ attribute.
Then, I import the builtins module, and checks if its print function is the same as the prefix-less print.
>>> print.__module__
'builtins'
>>> import builtins
>>> builtins.print("hello")
hello
>>> print is builtins.print
True
You should give the page on built-in functions a read
Quote:
The Python interpreter has a number of functions and types built into
it that are always available.
I am trying to access the module docstring from within the same module it is defined. Practical example:
#!/usr/bin/env python3
"Module docstring that explains what the script does."
import argparse as ap
parser = ap.ArgumentParser(description=__doc__)
...
I have always used __doc__, but I later stumbled upon some code using sys.module[__name__].__doc__. The two objects appear to be the same, but I am wondering if they are always the same. In other words, is the following:
>>> sys.module[__name__].__doc__ is __doc__
True
always true? Can I safely use __doc__ in my example instead of sys.modules[__name__].__doc__?
In practice they will be the same unless you go out of your way to make them different.
If you assign to __name__ then you can make the expression refer to a different module:
"""My doc string"""
import sys
print(__doc__)
__name__='sys'
print(sys.modules[__name__].__doc__)
will print two different doc strings.
Alternatively, you could leave __name__ alone but delete your module from sys.modules and then import another module with the same __name__ (or in Python 2.x use reload() to reload the module after editing the docstring). If you held onto a reference to a function in the old module you could still call it and __doc__ will be the old value while sys.modules[__name__].__doc__ is the new one.
So they could be different, but only if you work at it.
Introduction
Pydev is a great eclipse plugin that let us write python code easily.
It can even give autocompletion suggestion when I do this:
from package.module import Random_Class
x = Random_Class()
x. # the autocompletion will be popped up,
# showing every method & attribute inside Random_Class
That is great !!!
The Problem (And My Question)
However, when I don't use explicit import, and use __import__ for example, I can't have the same autocompletion effect.
import_location = ".".join(('package', 'module'))
__import__(import_location, globals(), locals(), ['*'])
My_Class = getattr(sys.modules[import_location], 'Random_Class')
x = My_Class()
x. # I expect autocompletion, but nothing happened
Question: is there any way (in pydev or any IDE) to make the second one also
show autocompletion?
Why do I need to do this?
Well, I make a simple MVC framework, and I want to provide something like load_model, load_controller, and load_view which is still work with autocompletion (or at least possible to work)
So, instead of leave users do this (although I don't forbid them to do so):
from applications.the_application.models.the_model import The_Model
x = The_Model()
x. # autocompletion works
I want to let users do this:
x = load_model('the_application', 'the_model')()
x. # autocompletion still works
The "applications" part is actually configurable by another script, and I don't want users to change all of their importing model/controller part everytime they change the configuration. Plus, I think load_model, load_controller, and load_view make MVC pattern shown more obvious.
Unexpected Answer
I know some tricks such as doing this (as what people do with
web2py):
import_location = ".".join(('package', 'module'))
__import__(import_location, globals(), locals(), ['*'])
My_Class = getattr(sys.modules[import_location], 'Random_Class')
x = My_Class()
if 0:
from package.module import Random_Class
x = Random_Class()
x. # Now autocompletion is going to work
and I don't expect to do this, since it will only add unnecessary
extra work.
I don't expect any don't try to be clever comments. I have enough of them
I don't expect dynamic import is evil comments. I'm not a purist.
I don't expect any just use django, or pylons, or whatever comments. Such as comments even unrelated to my question.
I have done this before. This may be slightly different from your intended method, so let me know if it doesn't apply.
I dynamically import different modules that all subclass a master class, using similar code to your example. Because the subclassing module already imports the master, I don't need to import it in the main module.
To get highlighting, the solution was to import the master class into the main module first, even though it wasn't used directly. In my case it was a good fallback if the particular subclass didn't exist, but that's an implementation detail.
This only works if your classes all inherit from one parent.
Not really an answer to my own question. However, I can change the approach. So, instead of provide "load_model()", I can use relative import. Something like this:
from ..models.my_model import Model_Class as Great_Model
m = Great_Model()
Suppose you have 3 modules, a.py, b.py, and c.py:
a.py:
v1 = 1
v2 = 2
etc.
b.py:
from a import *
c.py:
from a import *
v1 = 0
Will c.py change v1 in a.py and b.py? If not, is there a way to do it?
All that a statement like:
v1 = 0
can do is bind the name v1 to the object 0. It can't affect a different module.
If I'm using unfamiliar terms there, and I guess I probably am, I strongly recommend you read Fredrik Lundh's excellent article Python Objects: Reset your brain.
The from ... import * form is basically intended for handy interactive use at the interpreter prompt: you'd be well advised to never use it in other situations, as it will give you nothing but problems.
In fact, the in-house style guide at my employer goes further, recommending to always import a module, never contents from within a module (a module from within a package is OK and in fact recommended). As a result, in our codebase, references to imported things are always qualified names (themod.thething) and never barenames (which always refer to builtin, globals of this same module, or locals); this makes the code much clearer and more readable and avoids all kinds of subtle anomalies.
Of course, if a module's name is too long, an as clause in the import, to give it a shorter and handier alias for the purposes of the importing module, is fine. But, with your one-letter module names, that won't be needed;-).
So, if you follow the guideline and always import the module (and not things from inside it), c.v1 will always be referring to the same thing as a.v1 and b.v1, both for getting AND setting: here's one potential subtle anomaly avoided right off the bat!-)
Remember the very last bit of the Zen of Python (do import this at the interpreter prompt to see it all):
Namespaces are one honking great idea -- let's do more of those!
Importing the whole module (not bits and pieces from within it) preserves its integrity as a namespace, as does always referring to things inside the imported module by qualified (dotted) names. It's one honking great idea: do more of that!-)
Yes, you just need to access it correctly (and don't use import *, it's evil)
c.py:
import a
print a.v1 # prints 1
a.v1 = 0
print a.v1 # prints 0