In Python, once I have imported a module X in an interpreter session using import X, and the module changes on the outside, I can reload the module with reload(X). The changes then become available in my interpreter session.
I am wondering if this also possible when I import a component Y from module X using from X import Y.
The statement reload Y does not work, since Y is not a module itself, but only a component (in this case a class) inside of a module.
Is it possible at all to reload individual components of a module without leaving the interpreter session (or importing the entire module)?
EDIT:
For clarification, the question is about importing a class or function Y from a module X and reloading on a change, not a module Y from a package X.
Answer
From my tests, the marked answer, which suggests a simple reload(X), does not work.
From what I can tell the correct answer is:
from importlib import reload # python 2.7 does not require this
import X
reload( X )
from X import Y
Test
My test was the following (Python 2.6.5 + bpython 0.9.5.2)
X.py:
def Y():
print "Test 1"
bpython:
>>> from X import Y
>>> print Y()
Test 1
>>> # Edit X.py to say "Test 2"
>>> print Y()
Test 1
>>> reload( X ) # doesn't work because X not imported yet
Traceback (most recent call last):
File "<input>", line 1, in <module>
NameError: name 'X' is not defined
>>> import X
>>> print Y()
Test 1
>>> print X.Y()
Test 1
>>> reload( X ) # No effect on previous "from" statements
>>> print Y()
Test 1
>>> print X.Y() # first one that indicates refresh
Test 2
>>> from X import Y
>>> print Y()
Test 2
>>> # Finally get what we were after
If Y is a module (and X a package) reload(Y) will be fine -- otherwise, you'll see why good Python style guides (such as my employer's) say to never import anything except a module (this is one out of many great reasons -- yet people still keep importing functions and classes directly, no matter how much I explain that it's not a good idea;-).
from modulename import func
import importlib, sys
importlib.reload(sys.modules['modulename'])
from modulename import func
First off, you shouldn't be using reload at all, if you can avoid it. But let's assume you have your reasons (i.e. debugging inside IDLE).
Reloading the library won't get the names back into the module's namespace. To do this, just reassign the variables:
f = open('zoo.py', 'w')
f.write("snakes = ['viper','anaconda']\n")
f.close()
from zoo import snakes
print snakes
f = open('zoo.py', 'w')
f.write("snakes = ['black-adder','boa constrictor']\n")
f.close()
import zoo
reload(zoo)
snakes = zoo.snakes # the variable 'snakes' is now reloaded
print snakes
You could do this a few other ways. You could automate the process by searching through the local namespace, and reassigning anything that was from the module in question, but I think we are being evil enough.
If you want to do this:
from mymodule import myobject
Do this instead:
import mymodule
myobject=mymodule.myobject
You can now use myobject in the same way as you were planning (without the tiresome unreadable mymodule references everywhere).
If you're working interactively and want to reload myobject from mymodule you now can using:
reload(mymodule)
myobject=mymodule.myobject
If you're working in a jupyter environment, and you already have from module import function can use the magic function, autoreload by
%load_ext autoreload
%autoreload
from module import function
The introduction of the autoreload in IPython is given here.
assuming you used from X import Y, you have two options:
reload(sys.modules['X'])
reload(sys.modules[__name__]) # or explicitly name your module
or
Y=reload(sys.modules['X']).Y
few considerations:
A. if the import scope is not module-wide (e,g: import in a function) - you must use the second version.
B. if Y is imported into X from another module (Z) - you must reload Z, than reload X and than reload your module, even reloading all your modules (e,g: using [ reload(mod) for mod in sys.modules.values() if type(mod) == type(sys) ]) might reload X before reloading Z - and than not refresh the value of Y.
reload() module X,
reload() module importing Y from X.
Note that reloading won't change already created objects bound in other namespaces (even if you follow style guide from Alex).
Just to follow up on AlexMartelli's and Catskul's answers, there are some really simple but nasty cases that appear to confound reload, at least in Python 2.
Suppose I have the following source tree:
- foo
- __init__.py
- bar.py
with the following content:
init.py:
from bar import Bar, Quux
bar.py:
print "Loading bar"
class Bar(object):
#property
def x(self):
return 42
class Quux(Bar):
object_count = 0
def __init__(self):
self.count = self.object_count
self.__class__.object_count += 1
#property
def x(self):
return super(Quux,self).x + 1
def __repr__(self):
return 'Quux[%d, x=%d]' % (self.count, self.x)
This works just fine without using reload:
>>> from foo import Quux
Loading bar
>>> Quux()
Quux[0, x=43]
>>> Quux()
Quux[1, x=43]
>>> Quux()
Quux[2, x=43]
But try to reload and it either has no effect or corrupts things:
>>> import foo
Loading bar
>>> from foo import Quux
>>> Quux()
Quux[0, x=43]
>>> Quux()
Quux[1, x=43]
>>> reload(foo)
<module 'foo' from 'foo\__init__.pyc'>
>>> Quux()
Quux[2, x=43]
>>> from foo import Quux
>>> Quux()
Quux[3, x=43]
>>> reload(foo.bar)
Loading bar
<module 'foo.bar' from 'foo\bar.pyc'>
>>> Quux()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "foo\bar.py", line 17, in __repr__
return 'Quux[%d, x=%d]' % (self.count, self.x)
File "foo\bar.py", line 15, in x
return super(Quux,self).x + 1
TypeError: super(type, obj): obj must be an instance or subtype of type
>>> Quux().count
5
>>> Quux().count
6
>>> Quux = foo.bar.Quux
>>> Quux()
Quux[0, x=43]
>>> foo.Quux()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "foo\bar.py", line 17, in __repr__
return 'Quux[%d, x=%d]' % (self.count, self.x)
File "foo\bar.py", line 15, in x
return super(Quux,self).x + 1
TypeError: super(type, obj): obj must be an instance or subtype of type
>>> foo.Quux().count
8
The only way I could ensure the bar submodule was reloaded was to reload(foo.bar); the only way I access the reloaded Quux class is to reach in and grab it from the reloaded sub module; but the foo module itself kept holding onto the original Quux class object, presumably because it uses from bar import Bar, Quux (rather than import bar followed by Quux = bar.Quux); furthermore the Quux class got out of sync with itself, which is just bizarre.
Related
The basic idea of this is to have a variable that can only be used in the same script it is defined/declared scope.
So even if you import the script to another file you still can't access that variable.
Basically a local scope but for the script file.
Sorry if i'm bad at explaining.
Because normaly you can just do this:
Script a (the script with the variable)
scriptvar = 21
def somefunction():
return "banana"
Script b (the script that should not be able to access the variable)
import script_a
print(script_a.somefunction())
print(script_a.scriptvar)
Script b should return
>> banana
>> AttributeError: module 'script_a' has no attribute 'scriptvar'
You can't do this but there is a convention that if you add a underscore to the name you indicate that it is a internal variable.
I think, that there is no elegant way to get this.
There is a __all__ directive to specify what will be imported via from module_name import * but it will not work with
import module_name
module_name.scriptvar
There is a workaround: you can initialise all the necessary objects inside function and export only useful ones. Simple example
# lib.py
import sys
__all__ = ['x', 'y']
def __init(module):
def x():
return 2
z = 2
def y():
return z
module.x = x
module.y = y
del module.__init
__init(sys.modules[__name__])
# script.py
import lib
print(lib.x())
try:
lib.__init()
except AttributeError:
print('As expected')
from lib import *
print(x())
print(y())
try:
print(z)
except NameError:
print('As expected')
Running python3 script.py gives
2
As expected
2
2
As expected
I'm trying to do a dynamic import (using "__import__()") of a submodule in web2py and it doesn't seem to be finding the submodule.
Here's an example path:
web2py/app/modules/module.py <-- This works
web2py/app/modules/module_folder/submodule.py <-- This won't get spotted.
Right now as a workaround I'm using 'exec()' but I'd rather not do that.
Answers to questions:
"Do you have __init__.py in module_folder?"
Yep.
"What exactly is the line of code you write to import web2py/app/modules/module_folder/submodule.py?"
mapping_module = __import__('communication_templates.mappings.%s' % module_name)
"what is the error?"
<type 'exceptions.AttributeError'> 'module' object has no attribute 'mapping_dict'
Explanation: I'm trying to get a variable named 'mapping_dict' from the module once I've loaded it.
The problem here is that __import__ does not do the most intuitive thing (neither does import btw).
Here's what happens:
>>> import sys
>>> x = __import__('package1.package2.module')
>>> x
<module 'package1' from 'C:\\Temp\\package1\\__init__.py'>
Even though package1.package2.module was imported, the variable returned is actually package1.
So, to access something that is in package1.package2.module, one has to dig down again:
>>> x.package2.module.Class
<class 'package1.package2.module.Class'>
Is it then the same as importing just package1?
Not really:
>>> x = __import__('package1')
>>> x
<module 'package1' from 'C:\\Temp\\package1\\__init__.py'>
>>> x.package2.module.Class
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'package1' has no attribute 'package2'
Why?
Well, __import__ is the same as `import, that is, these are the same:
package1 = __import__('package1.package2.module')
# is the same as:
import package1.package2.module
And:
package1 = __import__('package1')
# is the same as:
import package1
In each of those cases, you get just one local variable package1.
In the case of importing package1.package2.module, package2 is also imported and stored in package2 attribute of package package1 etc.
Solution
To access package1.package2.module.Class, if package1.package2.module is in a string:
>>> s = 'package1.package2.module'
>>> module = __import__(s)
>>> for submodule_name in s.split('.')[1:]:
... module = getattr(module, submodule_name)
...
>>> module.Class
<class 'package1.package2.module.Class'>
I have a function called prepared_db in submodule db.db_1:
from spam import db
submodule_name = "db_1"
func_name = "prepare_db"
func = ...
how can I get the function by the submodule name and function name in the context above?
UPDATE:
To respond #histrio 's answer, I can verify his code works for os module. But it does not work in this case. To create the example:
$ mkdir -p spam/db
$ cat > spam/db/db_1.py
def prepare_db():
print('prepare_db func')
$ touch spam/db/__init__.py
$ PYTHONPATH=$PYTHONPATH:spam
now, you can do the import normally:
>>> from spam.db.db_1 import prepare_db
>>> prepare_db()
prepare_db func
but if you do this dynamically, I get this error:
>>> getattr(getattr(db, submodule_name), func_name)
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-9-1b6aa1216551> in <module>()
----> 1 getattr(getattr(db, submodule_name), func_name)
AttributeError: module 'spam.db.db_1' has no attribute 'prepared_db'
It's simple. You can consider module as object.
import os
submodule_name = "path"
func_name = "exists"
submodule = getattr(os, submodule_name)
function = getattr(submodule, func_name)
function('/home') # True
or [just for fun, don't do that]
fn = reduce(getattr, ('sub1', 'sub2', 'sub3', 'fn'), module)
UPDATE
import importlib
submodule = importlib.import_module('.'+submodule_name, module.__name__)
function = getattr(submodule, func_name)
I think I figured it out, you need to add the function to the __all__ variable in the __init__.py file, so that would be something along the lines of:
from .db_1 import prepare_db
__all__ = ['prepare_db']
After that, it should work just fine.
Im pretty new to python, and have been having a rough time learning it. I have a main file
import Tests
from Tests import CashAMDSale
CashAMDSale.AMDSale()
and CashAMDSale
import pyodbc
import DataFunctions
import automa
from DataFunctions import *
from automa.api import *
def AMDSale():
AMDInstance = DataFunctions.GetValidAMD()
And here is GetValidAMD
import pyodbc
def ValidAMD(GetValidAMD):
(short method talking to a database)
My error comes up on the line that has AMDInstance = DataFunctions.GetValidAMD()
I get the error AttributeError: 'module' object has no attribute 'GetValidAMD'
I have looked and looked for an answer, and nothing has worked. Any ideas? Thanks!
DataFunctions is a folder, which means it is a package and must contain an __init__.py file for python to recognise it as such.
when you import * from a package, you do not automatically import all it's modules. This is documented in the docs
so for your code to work, you either need to explicitly import the modules you need:
import DataFunctions.GetValidAMD
or you need to add the following to the __init__.py of DataFunctions:
__all__ = ["GetValidAMD"]
then you can import * from the package and everything listen in __all__ will be imported
When you create the file foo.py, you create a python module. When you do import foo, Python evaluates that file and places any variables, functions and classes it defines into a module object, which it assigns to the name foo.
# foo.py
x = 1
def foo():
print 'foo'
>>> import foo
>>> type(foo)
<type 'module'>
>>> foo.x
1
>>> foo.foo()
foo
When you create the directory bar with an __init__.py file, you create a python package. When you do import bar, Python evaluates the __init__.py file and places any variables, functions and classes it defines into a module object, which it assigns to the name bar.
# bar/__init__.py
y = 2
def bar():
print 'bar'
>>> import bar
>>> type(bar)
<type 'module'>
>>> bar.y
2
>>> bar.bar()
bar
When you create python modules inside a python package (that is, files ending with .py inside directory containing __init__.py), you must import these modules via this package:
>>> # place foo.py in bar/
>>> import foo
Traceback (most recent call last):
...
ImportError: No module named foo
>>> import bar.foo
>>> bar.foo.x
1
>>> bar.foo.foo()
foo
Now, assuming your project structure is:
main.py
DataFunctions/
__init__.py
CashAMDSale.py
def AMDSale(): ...
GetValidAMD.py
def ValidAMD(GetValidAMD): ...
your main script can import DataFunctions.CashAMDSale and use DataFunctions.CashAMDSale.AMDSale(), and import DataFunctions.GetValidAMD and use DataFunctions.GetValidAMD.ValidAMD().
Check out this.
It's the same problem. You are importing DataFunctions which is a module. I expct there to be a class called DataFunctions in that module which needs to be imported with
from DataFunctions import DataFunctions
...
AMDInstance = DataFunctions.GetValidAMD()
Suppose I have the following code.
foo.py
------
import bar
def abc(n):
return bar.xyz(n-1) if n>0 else "abc"
bar.py
------
import foo
def xyz(n):
return foo.abc(n-1) if n>0 else "xyz"
As explained in this post Circular (or cyclic) imports in Python, it will work. (The short explanation is, suppose we call import foo from python repl, when import bar is first encountered, python puts bar into sys.modules and executes bar.py, and when import foo is encountered, since foo is already in sys.modules, the import statement directly returns even though foo module is incomplete.)
Now if I change the code into the following:
foo.py
------
from bar import xyz
def abc(n):
return xyz(n-1) if n>0 else "abc"
bar.py
------
from foo import abc
def xyz(n):
return abc(n-1) if n>0 else "xyz"
The import will fail:
$ python foo.py
Traceback (most recent call last):
File "foo.py", line 1, in <module>
from bar import xyz
File "/Users/yxiong/bar.py", line 1, in <module>
from foo import abc
File "/Users/yxiong/foo.py", line 1, in <module>
from bar import xyz
ImportError: cannot import name xyz
What is worth noting here is python seems to fail on the second try of from bar import xyz.
My question is, what is exactly going on at those steps. Specifically, what does python do when it sees a from foo import abc statement?
Walk through the frames, first Python attempts to load/compile the foo module (to be known as __main__, only compiles it once, but will be executed twice):
Traceback (most recent call last):
File "foo.py", line 1, in <module>
from bar import xyz # first attempt to import xyx, but it depends on abc
Python attempts to execute the import statement.
So since Python only loads/compiles modules once (unless using reload), it looks in sys.modules and since it isn't there, it attempts to load/compile the bar module to import xyz.
File "/Users/yxiong/bar.py", line 1, in <module>
from foo import abc # so we attempt to import abc, but it depends on xyz
But bar attempts to load/compile foo and import abc, which we see is already in sys.modules. And we're back to our original import:
File "/Users/yxiong/foo.py", line 1, in <module>
from bar import xyz
and we get the ImportError:
ImportError: cannot import name xyz
Let's empirically demonstrate, from a terminal in Unix:
cat > foo.py
print('executing ' + __name__)
from bar import xyz
def abc(n):
return xyz(n-1) if n>0 else "abc"
ctrl-d
cat > bar.py
print('executing ' + __name__)
from foo import abc
def xyz(n):
return abc(n-1) if n>0 else "xyz"
ctrl-d
python foo.py
Next experiment:
cat > main.py
print('executing ' + __name__)
import foo
ctrl-d
python main.py