This question already has answers here:
How to share imports between modules?
(2 answers)
Closed 3 years ago.
Considering the following python code setup:
main.py
MTS/
optimization.py
__init__.py
Where __init__.py consists of one line import optimization
and optimization.py uses different modules such as numpy (called as np). Let's main.py be:
from MTS import optimization
import numpy as np
a=np.load('data.npy')
b=optimization.minimize(a)
this code returns the following error:
global name 'np' is not defined
If I import numpy inside optimization.py, it works but I want to avoid importing the same module twice. How to share an imported module across other modules?
In python, imports are mainly about namespaces. Importing a module puts it into the current namespace, and if it's not already in memory it loads it into memory.
If you import a module that another module has already imported, then that module is already in memory so you just get a reference to the existing module.
So, in other words, just do import numpy as np inside every file you need to use it in. This will not consume any extra memory, as numpy will only be properly loaded once, and every subsequent import statement will just add it to the namespace of whoever calls it.
An additional benefit of this is that it's much more clear that your code is going to use numpy - it would be confusing if you just used np.load() halfway through your program and there was no indication that you'd defined np anywhere (C has this problem, for example; pretty much every programming language since has used the idea of namespaces to avoid it, including even C++). It is poor practice in python to not do this.
Related
This question already has answers here:
Why import when you need to use the full name?
(8 answers)
Closed 1 year ago.
I'm new to PyCharm, and after downloading a Python package (Manim) my code won't recognize the methods used in the package, unless I very precisely tell it where to look. In other words, when trying to just import Manim, I get this:
where PyCharm even seems to indicate that it doesn't need the two gray lines, as it has already imported Manim. The problem is the errors (underlined in red), they point to classes and/or methods from the Manim package, but are not recognized until I precise:
How can I optimize my imports so that one line suffices for all that's concerning Manim? (This works fine with just from manimlib import * using Spyder3 editor.)
As #Mike Scotty pointed out in his comments, import * and from a import * are generally bad ideas, since python won't know what to import, especially if there are multiple classes with the same name.
An IDE not complaining does NOT mean, your code runs smoothly.
You have several options here:
Having a rather large import list (which is by no means wrong) as you have in your second picture.
Using import manimlib and having rather long function/class calls: intro_words = manimlib.mobject.svg.text_mobject.Text
It's possible to bunch similar imports together like this:
from manimlib.mobject import geometry.Polygon as Polygon, svg.text_mobject.Text as Text
To my humble knowledge, the most pythonic way to go, is having very specific imports, even if that means you'll end up with a large import list. To add to this, most IDEs like PyCharm, Atom or Visual Studio have ways collapsing large import lists into one line:
for example PyCharm does this: import ... which displays all imports by clicking on it.
Refer to pythons documentation on imports and pythons documentation on modules to get a better understanding on how imports work.
Suppose I want to import a python library for use inside a function. Is it better to import the library within the function or import it globally?
Do this
def test_func:
import pandas as pd
# code implementation
or have the line below at the top of the python file to import globally?
import pandas as pd
What are the pros and cons of each approach? Which is the best practice in python?
I am using python v3.6
EDIT: Some clarifications to make.
Suppose I have 2 functions.
def func1:
import pandas as pd
# code implementation
def func2:
import pandas as pd
# code implementation
The python script runs both functions. Will the library be imported twice or is the python compiler smart enough to import it only once? This has performance implications.
It's a difference in name-visibility and execution time-point. The module-level import is imported when the file you are loading is imported or run itself. The function local one obviously only if the function is run. The imported names are either visible to all things in the file, or just within the function the import is executed in.
As there is a cost for hitting the import statement (albeit a small one, but still), the local one will always execute, not just once. It will not fully re-import the module though, python caches modules once they are imported the first time (see reload and sys.modules).
The best practice clearly is to use module level imports, and that's what you see in 99.999% of code. A huge reason is maintainability - if you want to understand what dependencies a module has, it's convenient to just look at the top, instead of having to comb through all code.
So when to use function local imports?
There are three scenarios:
you can't use the import earlier. This happens when e.g. a backend for a db or other system/functionality is chosen at runtime through configuration or system inspection.
you otherwise have circular imports. This is a rare case and also a code-smell, so if that is necessary, consider refactoring.
reducing startup-time by deferring module imports. This is very rarely useful though.
So for your case, the answer is a quick and simple "don't do it".
The module will be loaded when you import it, so if you need to import a rarely used module but cost a lot of time to initialize, you should import it when you need it.
Actually, if we just care about performance but not readability, it maybe always better to import module when we really need it.
But we need to keep our program maintainable. Importing all modules on the top is the most explicit way to tell others and author himself which modules are used.
To sum up, if you really have a very costly but rarely used module, you should import it locally. Otherwise you should import them on the top.
This question already has answers here:
Import folder of modules in Python
(3 answers)
Closed 7 years ago.
I know this question has been asked before but I can't make heads or tails of what the answer means.
I am making the transition from MATLAB to Python. In MATLAB I can write my own functions and use them in my code. I know I can do the same thing in Python. But I am having a hard time figuring out how to do it.
What I would like to do it create a file with multiple function definitions and then import it into Python like any other module.
First, is this the proper way of thinking it about it? Or do I just need to create multiple definition files for each function?
Second, if it is the proper way of thinking about it how do I access the file? I know you have to set the PYTHONPATH. I have looked at it and where it is looking makes no sense to me.
As an Example: I created a folder called User. In it I have a python function called ted.py. I put said file where the rest of the library files are located (as in numpy or scipy). I want to import the file called User. How can I do this?
After working with Python for awhile I get it. As long as the file is in the same directory and you use the import properly you can use one , some or all of the function definitions in the file.
You have an un-matlab-like (matlab-unlike? dis-matlab-like?) option of putting multiple function definitions into the same .py file. Once the file -- say, fundefs.py -- is on your path, possibly through having issued import sys; sys.path.append('path/to/fundefs');, you can import it
through import fundefs, after which you can access the functions therein by fundefs.fun1, fundefs.fun2 etc.
through from fundefs import *, which will throw all the functions into your current namespace. This is generally discouraged (and frowned upon) for larger modules as it will pollute your namespace, but for a few functions of your own this might just be what you're after. See also this very informative answer (and also comments therein).
as a middle ground through import very_long_and_descriptive_module_name as shorthand to access your functions as shorthand.fun1, shorthand.fun2 etc. (in the obvious case if your definitions are in the file very_long_and_descriptive_module_name.py)
You don't import User. What you want is to import ted. Typically, you would put ted.py in the same folder as your main python file, not in a separate folder.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Python import coding style
When I write some code that requires an import, and the import is only introduced by the bit of code I'm currently writing, should I:
Stick the import at the top of the file where it is made clear that to make this module work it needs these imports, yet the import is detached from the usage and should the need be removed later the module may still import stuff it doesn't ever actually use, or
Keep the import with the code that uses it immediately thereafter so it is obvious what the import is used to do and whence it can be safely removed, but risk importing the same libs multiple times and make it hard to work out what libs are required to make the module work.
Best practice?
Put the import at the top? Or put it where it gets used?
Import_Statement_Overhead from the Python wiki states:
"import statements can be executed just about anywhere. It's often
useful to place them inside functions to restrict their visibility
and/or reduce initial startup time. Although Python's interpreter is
optimized to not import the same module multiple times, repeatedly
executing an import statement can seriously affect performance in some
circumstances."
I follow general stylistic conventions and put all of my import statements at the top of the program. PEP 8 states re imports:
"Imports are always put at the top of the file, just after any module
comments and docstrings, and before module globals and constants."
This question already has answers here:
What does __all__ mean in Python?
(10 answers)
Closed 6 years ago.
I've seen it a lot in python/Lib source code but I don't know what it is for.
I thought it was used to limit accessible members of of a module. So only the elements at __all__ will show up when dir(module).
I did a little example and saw it was not working as I expected.
So... What's the python __all__ module level variable for?
It has two purposes:
Anybody who reads the source will know what the exposed public API is. It doesn't prevent them from poking around in private declarations, but does provide a good warning not to.
When using from mod import *, only names listed in __all__ will be imported. This is not as important, in my opinion, because importing everything is a really bad idea.
http://docs.python.org/tutorial/modules.html#importing-from-a-package
Now what happens when the user writes
from sound.effects import *? Ideally,
one would hope that this somehow goes
out to the filesystem, finds which
submodules are present in the package,
and imports them all. This could take
a long time and importing sub-modules
might have unwanted side-effects that
should only happen when the sub-module
is explicitly imported.
The only solution is for the package
author to provide an explicit index of
the package. The import statement uses
the following convention: if a
package’s __init__.py code defines a
list named __all__, it is taken to be
the list of module names that should
be imported when from package import *
is encountered. It is up to the
package author to keep this list
up-to-date when a new version of the
package is released. Package authors
may also decide not to support it, if
they don’t see a use for importing *
from their package.
It controls what you get pulled into your namepsace when you
from blah import *
See Importing * from a Package