I've noticed that importing a module will import its functions and methods, and the functions and methods of those as well. Is there a set rule for how many levels down python will import when you import an upper-level module?
edit
sorry, I think I've been misunderstood by the answers so far responding about multiple imports of some dependencies. I'm thinking of nested folders e.g. in django, if you import django, you can access django.contrib.auth, but you can't access django.contrib.auth.views unless you import that specifically. I was just wondering if it's always two levels down in such a case
second edit
to clarify again.. in the django example, the layout is /django/contrib/auth/views.py, where each of the subfolders has a "init.py" making it a module, none of which define any "all" attributes. Is my example bad, since maybe you can't use the dot syntax to navigate to a file within a module designated folder?
No, python will import what it needs to import. However, each module is only imported once. For example, if one module does import sys and another module does import sys, it will not physically do it twice.
Not really. A module imports stuff from other modules because it needs to use them in that module, otherwise it'll break.
There is no pre-defined import depth level. Import statements are executed, just like any other python statement.
But, you may wonder, how are cycles avoided? Modules are added to sys.modules (i.e., cached) when they get imported for the first time, and that is the first location examined when an import statement is executed. So each module is loaded just once, although it may appear in many import statements.
Related
Suppose I want to import a python library for use inside a function. Is it better to import the library within the function or import it globally?
Do this
def test_func:
import pandas as pd
# code implementation
or have the line below at the top of the python file to import globally?
import pandas as pd
What are the pros and cons of each approach? Which is the best practice in python?
I am using python v3.6
EDIT: Some clarifications to make.
Suppose I have 2 functions.
def func1:
import pandas as pd
# code implementation
def func2:
import pandas as pd
# code implementation
The python script runs both functions. Will the library be imported twice or is the python compiler smart enough to import it only once? This has performance implications.
It's a difference in name-visibility and execution time-point. The module-level import is imported when the file you are loading is imported or run itself. The function local one obviously only if the function is run. The imported names are either visible to all things in the file, or just within the function the import is executed in.
As there is a cost for hitting the import statement (albeit a small one, but still), the local one will always execute, not just once. It will not fully re-import the module though, python caches modules once they are imported the first time (see reload and sys.modules).
The best practice clearly is to use module level imports, and that's what you see in 99.999% of code. A huge reason is maintainability - if you want to understand what dependencies a module has, it's convenient to just look at the top, instead of having to comb through all code.
So when to use function local imports?
There are three scenarios:
you can't use the import earlier. This happens when e.g. a backend for a db or other system/functionality is chosen at runtime through configuration or system inspection.
you otherwise have circular imports. This is a rare case and also a code-smell, so if that is necessary, consider refactoring.
reducing startup-time by deferring module imports. This is very rarely useful though.
So for your case, the answer is a quick and simple "don't do it".
The module will be loaded when you import it, so if you need to import a rarely used module but cost a lot of time to initialize, you should import it when you need it.
Actually, if we just care about performance but not readability, it maybe always better to import module when we really need it.
But we need to keep our program maintainable. Importing all modules on the top is the most explicit way to tell others and author himself which modules are used.
To sum up, if you really have a very costly but rarely used module, you should import it locally. Otherwise you should import them on the top.
I created a module named util that provides classes and functions I often use in Python.
Some of them need imported features. What are the pros and the cons of importing needed things inside class/function definition? Is it better than import at the beginning of a module file? Is it a good idea?
It's the most common style to put every import at the top of the file. PEP 8 recommends it, which is a good reason to do it to start with. But that's not a whim, it has advantages (although not critical enough to make everything else a crime). It allows finding all imports at a glance, as opposed to looking through the whole file. It also ensures everything is imported before any other code (which may depend on some imports) is executed. NameErrors are usually easy to resolve, but they can be annoying.
There's no (significant) namespace pollution to be avoided by keeping the module in a smaller scope, since all you add is the actual module (no, import * doesn't count and probably shouldn't be used anyway). Inside functions, you'd import again on every call (not really harmful since everything is imported once, but uncalled for).
PEP8, the Python style guide, states that:
Imports are always put at the top of
the file, just after any module
comments and docstrings, and before module globals and constants.
Of course this is no hard and fast rule, and imports can go anywhere you want them to. But putting them at the top is the best way to go about it. You can of course import within functions or a class.
But note you cannot do this:
def foo():
from os import *
Because:
SyntaxWarning: import * only allowed at module level
Like flying sheep's answer, I agree that the others are right, but I put imports in other places like in __init__() routines and function calls when I am DEVELOPING code. After my class or function has been tested and proven to work with the import inside of it, I normally give it its own module with the import following PEP8 guidelines. I do this because sometimes I forget to delete imports after refactoring code or removing old code with bad ideas. By keeping the imports inside the class or function under development, I am specifying its dependencies should I want to copy it elsewhere or promote it to its own module...
Only move imports into a local scope, such as inside a function definition, if it’s necessary to solve a problem such as avoiding a circular import or are trying to reduce the initialization time of a module. This technique is especially helpful if many of the imports are unnecessary depending on how the program executes. You may also want to move imports into a function if the modules are only ever used in that function. Note that loading a module the first time may be expensive because of the one time initialization of the module, but loading a module multiple times is virtually free, costing only a couple of dictionary lookups. Even if the module name has gone out of scope, the module is probably available in sys.modules.
https://docs.python.org/3/faq/programming.html#what-are-the-best-practices-for-using-import-in-a-module
I believe that it's best practice (according to some PEP's) that you keep import statements at the beginning of a module. You can add import statements to an __init__.py file, which will import those module to all modules inside the package.
So...it's certainly something you can do the way you're doing it, but it's discouraged and actually unnecessary.
While the other answers are mostly right, there is a reason why python allows this.
It is not smart to import redundant stuff which isn’t needed. So, if you want to e.g. parse XML into an element tree, but don’t want to use the slow builtin XML parser if lxml is available, you would need to check this the moment you need to invoke the parser.
And instead of memorizing the availability of lxml at the beginning, I would prefer to try importing and using lxml, except it’s not there, in which case I’d fallback to the builtin xml module.
If I were to create a module that was called for example imp_mod.py and inside it contained all (subjectively used) relevant modules that I frequently used.
Would importing this module into my main program allow me access to the imports contained inside imp_mod.py?
If so, what disadvantages would this bring?
I guess a major advantage would be a reduction of time spent importing even though its only a couple of seconds saved...
Yes, it would allow you to access them. If you place these imports in imp_mod.py:
from os import listdir
from collections import defaultdict
from copy import deepcopy
Then, you could do this in another file, say, myfile.py:
import imp_mod
imp_mod.listdir
imp_mod.defaultdict
imp_mod.deepcopy
You're wrong about reduction of importing time, as what happens is the opposite. Python will need to import imp_mod and then import the other modules afterwards, while the first import would not be needed if you were importing these modules in myfile.py itself. If you do the same imports in another file, they will already be in cache, so virtually no time is spent in the next import.
The real disadvantage here is less readability. Whoever looks at imp_mod.listdir, for example, will ask himself what the heck is this method and why it has the same name as that os module's method. When he had to open imp_mod.py just to find out that it's the same method, well, he probably wouldn't be happy. I wouldn't.
As lucasnadalutti mentioned, you can access them by importing your module.
In terms of advantages, it can make your main program care less about where the imports are coming from if the imp_mod handles all imports, however, as your program gets more complex and starts to include more namespaces, this approach can get more messy. You can start to handle a bit of this by using __init__.py within directories to handle imports to do a similar thing, but as things get more complex, personally, I feel it add a little more complexity. I'd rather just know where a module came from to look it up.
I am new to Python. I am developing a small project. I need to follow coding standards from starting on wards. How to use import statements in a proper way. Now I am working on Python 2.7. If I move to 3.x are there any conflicts with absolute imports? And what is the difference between absolute and relative imports?
The distinction between absolute and relative that's being drawn here is very similar to the way we talk about absolute and relative file paths or even URLs.
An absolute {import, path, URL} tells you exactly how to get the thing you are after, usually by specifying every part:
import os, sys
from datetime import datetime
from my_package.module import some_function
Relative {imports, paths, URLs} are exactly what they say they are: they're relative to their current location. That is, if the directory structure changes or the file moves, these may break (because they no longer mean the same thing).
from .module_in_same_dir import some_function
from ..module_in_parent_dir import other_function
Hence, absolute imports are preferred for code that will be shared.
I was asked in comments to provide an example of how from __future__ import absolute_import ties into this, and how it is meant to be used. In trying to formulate this example, I realized I couldn't quite explain its behavior either, so I asked a new question. This answer gives a code sample showing a correctly working implementation of from __future__ import absolute_import, where it actually resolves an ambiguity.
The accepted answer goes into more detail about why this works the way it does, including a discussion of the confusing wording of the Python 2.5 changelog. Essentially, the scope of this directive (and by extension the distinction between absolute and relative imports in Python) is very, very narrow. If you find yourself needing these distinctions to make your code work, you're probably better off renaming your local module if at all possible.
Imports should usually be on separate lines:
Yes: import os
import sys
No: import sys, os
It's okay to say this though:
from subprocess import Popen, PIPE
Imports are always put at the top of the file, just after any module comments and docstrings, and before module globals and constants.
Imports should be grouped in the following order:
Standard library imports.
Related third party imports.
Local application/library specific imports.
You should put a blank line between each group of imports.
As per Pep8 :-
Absolute imports are recommended, as they are usually more readable and tend to be better behaved (or at least give better error messages) if the import system is incorrectly configured (such as when a directory inside a package ends up on sys.path):
import mypkg.sibling
from mypkg import sibling
from mypkg.sibling import example
However, explicit relative imports are an acceptable alternative to absolute imports, especially when dealing with complex package layouts where using absolute imports would be unnecessarily verbose:
from . import sibling
from .sibling import example
Standard library code should avoid complex package layouts and always use absolute imports.
Implicit relative imports should never be used and have been removed in Python 3.
When importing a class from a class-containing module, it's usually okay to spell this:
from myclass import MyClass
from foo.bar.yourclass import YourClass
If this spelling causes local name clashes, then spell them explicitly:
import myclass
import foo.bar.yourclass
and use "myclass.MyClass" and "foo.bar.yourclass.YourClass".
Wildcard imports (from <module> import *) should be avoided, as they make it unclear which names are present in the namespace, confusing both readers and many automated tools. There is one defensible use case for a wildcard import, which is to republish an internal interface as part of a public API (for example, overwriting a pure Python implementation of an interface with the definitions from an optional accelerator module and exactly which definitions will be overwritten isn't known in advance).
https://www.python.org/dev/peps/pep-0008/#imports
This is pretty much Python, but asking from a Django user.
Suppose this is how Django apps are layout:
Webclient
apps
myapp#1
library
library.py
myapp#2
views.py
myapp#3
If I am working with views.py, and I want to import library.py, which one seems better?
from webclient.apps.myapp.library import LibraryClass
from webclient.apps.myapp.library.library import LibraryClass
I am using PyCharm, and either way doesn't complain about "unresolved references".
Is it better to import very speifically. Is second import method more likely to avoid name collison, if possible at all (say /library/ has several .py files)?
Thanks.
You should always import names from where they're defined. That way if webclient.apps.myapp.library should stop importing LibraryClass one day, you won't break the other imports.
As a follow-up to Ignacio's answer, you should look at the documentation of the libraries you are using, to see where it suggests you import things. It may be that although LibraryClass is defined in webclient.apps.myapp.library.library, it is documented as being in webclient.apps.myapp.library, so at some point, it the definition might be moved there, or webclient.apps.myapp.library.oldversion, but still accessible from webclient.apps.myapp.library.