In python is it possible to write a module which results in the following behavior:
from my_module import normal_stuff
# nothing happens
but
from my_module import normal_stuff, arcane_magic
# some code has been executed behind the scenes (i.g. redefining sys.excepthook)
I imagine that this could be done by parsing the importing script with the help of the inspect-module but maybe there is a "cleaner" way.
I would just put the following code under my imports:
from my_module import normal_stuff, arcane_magic
if 'arcane_magic' in globals():
# do some magic stuff
pass
Related
I have a Jupyter Notebook - we'll call it main, and I'm running it piecemeal in Jupyter Lab 3.0.14 using a Python 3 environment.
Within this notebook, I'm trying to import custom functions from a module, we'll call this one module.py, which is in the same directory as main.
It looks something like this:
//main
import sys
import os
from module import foo, bar
//module.py
def foo():
return
def bar():
return
When I run the import statements in main, it throws an error
ImportError: cannot import name 'bar' from 'module' (/module.py)
If I run
from module import foo
or
from module import *
It's fine (although calling the functions will throw an error that the function is not defined). In fact, it will import foo even if there's no function named foo in module.py. It won't import any of my own functions by name. I feel like I must be missing something fundamental, here...
Any help would be greatly appreciated!!
Okay, this is my first time using Jupyter Lab, and this problem came from a fundamental misunderstanding of how Jupyter (and maybe other IDEs?) manages packages.
In short, as I understand it, re-running the code which imported functions from module.py did not actually re-import the contents. The only things it imported were what I asked for the first time, and any subsequent calls to import functions only referenced that first static image I pulled.
Therefore, if module.py had one function, foo, and I ran:
from module import foo
...then made some changes, such as editing the contents of foo and adding bar, running:
from module import foo, bar
...didn't actually do anything but reference that first copy of module.py I pulled. Because bar didn't exist the first time I ran the import function, it "couldn't see it", no matter how many times I re-ran the import function. I had to restart the kernel to fix it, basically.
I noticed this because when I changed the name of the script, it would import the functions that weren't previously working, but then edits to the functions, even once imported, weren't recognized.
This link provided the clue I needed to figure this out.
Hopefully this helps somebody!
Restart the kernel in the Jupiter notebook. I solved a similar import name issue by restarting the kernel.
In my working directory, I have python3 files like this
/Path/to/cwd/main.py
/Path/to/cwd/Folder/one.py
/Path/to/cwd/Folder/two.py
So I had a main.py file like this
import Folder.one as one
#Do something
In one.py I had code like this
import two
#Some functions defined locally utilizing functions written in two.py
if __name__ == '__main__':
#Code for testing Functions
When I run one.py, it runs fine. But when I run main.py, it throws an error
ModuleNotFoundError: No module named 'two'
Ideally, I would not be expecting such an error at all.
It worked when I changed the import statement from import two to import Folder.two, which works. But I would like to do this in some other way without affecting such import statements much. How to achieve this?
In order for the python interpreter to know what directories contain code to be loaded, you need to include a __init__.py file.
Have a look at this answer to learn more about how to import packages.
In the case of your second import, to access that method you would need to use this syntax.
from .two import *
I know that when we do 'import module_name', then it gets loaded only once, irrespective of the number of times the code passes through the import statement.
But if we move the import statement into a function, then for each function call does the module get re-loaded? If not, then why is it a good practice to import a module at the top of the file, instead of in function?
Does this behavior change for a multi threaded or multi process app?
It does not get loaded every time.
Proof:
file.py:
print('hello')
file2.py:
def a():
import file
a()
a()
Output:
hello
Then why put it on the top?:
Because writing the imports inside a function will cause calls to that function take longer.
I know that when we do 'import module_name', then it gets loaded only once, irrespective of the number of times the code passes through the import statement.
Right!
But if we move the import statement into a function, then for each function call does the module get re-loaded?
No. But if you want, you can explicitly do it something like this:
import importlib
importlib.reload(target_module)
If not, then why is it a good practice to import a module at the top of the file, instead of in function?
When Python imports a module, it first checks the module registry (sys.modules) to see if the module is already imported. If that’s the case, Python uses the existing module object as is.
Even though it does not get reloaded, still it has to check if this module is already imported or not. So, there is some extra work done each time the function is called which is unnecessary.
It doesn't get reloaded after every function call and threading does not change this behavior. Here's how I tested it:
test.py:
print("Loaded")
testing.py:
import _thread
def call():
import test
for i in range(10):
call()
_thread.start_new_thread(call, ())
_thread.start_new_thread(call, ())
OUTPUT:
LOADED
To answer your second question, if you import the module at the top of the file, the module will be imported for all functions within the python file. This saves you from having to import the same module in multiple functions if they use the same module.
I would like to import a function foo() from module abc.py
However, abc.py contains other functions which rely on modules which are not available for Python (i.e. I cannot import them into python interpreter, because I use ImageJ to run abc.py as Jython)
One solution I found is to put the problematic imports inside the name == "main" check, such as:
# abc.py
def foo():
print("Hello, World!")
def run_main_function():
foo()
...other stuff using IJ...
if __name__ == "__main__":
from ij import IJ
run_main_function()
So when I try to import foo from into another script def.py, e.g.:
# def.py
from abc import foo
def other_func():
foo()
if __name__ == "__main__":
other_func()
This works. But when I put imports in normal fashion, at the top of the script, I get an error: No module named 'ij'. I would like to know if there is a solution to this problem? Specifically, that I put the imports at the top of the script and then within def.py I say to import just the function, without dependencies of abc.py?
I would like to know if there is a solution to this problem? Specifically, that I put the imports at the top of the script and then within def.py I say to import just the function, without dependencies of abc.py?
As far I know, it's the way that python works. You should put that import in the function that uses it if won't be aviable always.
def run_main_function():
from ij import IJ
foo()
Also, don't use abc as a module name, it's a standard library module: Abstract Base Class 2.7, Abstract Base Class 3.6
Edit: don't use trailing .py when importing as Kind Stranger stated.
I want to make use of an existing python module (called "module.py"). I'm only interested in one function from that module ("my_function()"). The module also contains a lot of other functions, which I'm not using. These other functions cause the module to have a lot of imports that are not used in my_function.
"""module.py"""
import useful_import
import useless_import1
import useless_import2
def my_function():
return useful_import.do()
def other_function1():
return useless_import1.do()
def other_function2():
return useless_import2.do()
The code I've written (main.py) imports only my_function, but it still requires me to include/install the other useless modules. I've checked and none of the useless modules run any code on import, so I should be able to safely remove them.
"""main.py"""
from module import my_function
print my_function()
How do I best deal with this?
Should I included the useless imports in my project anyway?
Should I make a copy of module.py and edit it so that it only contains my_function and the right imports?
Should I copy my_function and its imports into main.py?
(some other option I didn't think/know of)?
It kind of depends on the context, e.g. how will this code be used later, who will maintain it, what kind of code is it realy etc etc.
But my suggestion under most circumstances would be:
refactor my_function and its needed imports into a new_module.py
use this module in main.py
Either remove module.py from your code base, or have it import from new_module