Are there any rules or guidelines concerning when to use relative imports in Python? I see them in use all the time, such as in the Flask web framework. When searching for this topic, I only see articles on how to use relative imports, but not why.
So is there some special benefit to using:
from . import x
rather than:
from package import x
Moreover, I noticed that a related SO post mentions that relative imports are discouraged. Yet people still continue to use them.
Check out PEP 328's section on relative imports
The rationale seems to be as written:
Several use cases were presented, the most important of which is being able to rearrange the structure of large packages without having to edit sub-packages. In addition, a module inside a package can't easily import itself without relative imports.
Related
so this is a collection of questions that are more to clarify things and help better understand rather than an issue I am having.
I apologise now if I got things wrong or if these questions have been answered before. I wasn't able to find them.
First clarification I want to ask is:
Let us assume:
import scipy
First, I have noticed that you cannot in general access a module in a package by doing import package and then trying to access package.module.
For example scipy.io
You often have to do import package.module or even import astropy.io.fits, or you can do from package import module.
My question is why is this the case, and why is it so random -dependent on the package? I can't seem to identify any stable pattern.
Is it due to the fact that some of these libraries (packages) are very big and in order to not have memory problems it only imports the core attributes/modules?
The second question:
It relates to actually checking the size of these packages. Is there any way to see how big they are when imported? Any way of knowing what will work and what won't other than trying it? I guess I could check with sys.modules and try to obtain it from there?
The third and final question:
In the scenario that I am not running my code on a Raspberry Pi and I don't necessarily have to worry about the memory issue (if that is the reason why they don't allow direct access), is there any way of actually importing package, such that it also loads all the sub packages?
I am just being lazy and wondering if it is possible. I am aware that it isn't good practice, but curiosity killed the cat.
Just to update and make it accessible to people to see related questions I have looked at:
This answer gives good advice on good general practice:
What are good rules of thumb for Python imports?
Why can't I use the scipy.io? just like the documentation explains why the subpackage isn't necessarily imported
Then there is obviously the documentation:
https://docs.python.org/3/reference/import.html#packages
Section 5.2.1 is the reason why import scipy doesn't also import scipy.io, but I was wondering why would developers not make it an automated process.
This question is actually similar to part of my question but doesn`t seem to have a clear answer Python complex subpackage importing
Status of Questions:
Question 1: Good reason in answers
Question 2: Pending
Question 3: Pending
Answer Q1
When you import a package, especially large ones like SciPy, it uses the init.py module intialisation module which prevents all subpackages/modules from being imported automatically to save space. I won't go into this further as this is already mentioned in this question, documented here, and talked about in other answers.
Additionally, if you have questions about scripts vs. modules, this post is incredibly descriptive.
Answer Q2
To find the size of a package I would point you towards this post about finding package directories, and then this post about reporting the size of a particular directory. You could create some combined code to do both for you.
Answer Q3
Update: Unsure on how to do this as the normal from package import * works as explained in the documentation (similar to Q1):
if a package’s __init__.py code defines a list named __all__, it is taken to be the list of module names that should be imported when from package import * is encountered
A package is represented by the file __init__.py. Therefore, the packge scipy is represented by scipy/__init__.py. Inside this file you see a lot of imports like this:
from scipy.version import version as __version__
This is the reason why scipy.__version__ works, even though __version__ actually lives in scipy.version. Not all packages do this. There is no rule when such kind of behavior can be expected. It is totally up to the package author(s).
The key difference between these import calls is the namespace the module is imported into. Given the following example:
import mypackage
import mypackage.myclass
from mypackage import myclass
The first example imports everything exposed by __init__.py into the package's namespace. I.E. its elements can be accessed as mypackage.myclass(). The second example imports only mypackage.myclass and still imports it into that package's namespace, so it is still accessed as mypackage.myclass(). The third example imports mypackage.myclass into the current namespace, so it is accessed explicitly as myclass(), as if you had defined it yourself in the same script. This may hide things that you have named elsewhere.
One other important use case looks like this:
import mypackage as mp
This lets you set the namespace that you want that package to be imported into, perhaps making it a shorthand or something more convenient.
In the case of your question about why scipy doesn't import everything when you call import scipy, what it comes back to is that that import call only imports whatever the developers tell it to in the __init__.py. For scipy specifically, if you do:
import scipy
dir(scipy)
You will see that it imports a bunch of classes and functions that are used throughout the package. I suspect that they intentionally don't import the submodules so as not to litter your runtime space with things that you aren't using. Perhaps there is a way to import everything automatically, but you probably shouldn't.
Whenever you use autoimport provided by PyCharm it generates an absolute path import. i.e.
from my_package.my_subpackage import my_thing
instead of
from .my_subpackage import my_thing
Is there a setting to use relative imports instead of absolute ones when importing a python package?
It appears currently there is no way to change the import style for auto-import feature to use relative imports. The only style changes you can make to import statements are how the absolute imports are structured:
(The Jetbrains/PyCharm settings search functionality is excellent by the way).
The same thing happens when refactoring, it's definitely an option I'd like to see added.
It will be possible starting with 2019.3 https://youtrack.jetbrains.com/issue/PY-6054
Note, that it will not do the relative import automatically if there are no other relative imports in that file yet. There is also an action to convert the existing absolute import to relative:
I would advise against relative in general, you can refer to this question:
https://softwareengineering.stackexchange.com/a/159505
Also, you can check official pep8 specs https://www.python.org/dev/peps/pep-0008/
Absolute imports are recommended, as they are usually more readable and tend to be better behaved (or at least give better error messages) if the import system is incorrectly configured (such as when a directory inside a package ends up on sys.path):
From my personal experience it turns out that they sometimes poorly integrate with Pycharm IDE when there's more complex package layout breaking test running through Pycharm. I suppose there might be some issues in other tools too.
So I have thought about it this way for a Python 2.7 project. It would be composed of two independent parts requiring a common class (module) file in a third package:
SomeRootFolder/Package1Folder/manyPythonModuleFiles.py
SomeRootFolder/Package2Folder/manyPythonModuleFiles.py
SomeRootFolder/SharedPackageFolder/OneCommonClassNeedsToBeShared.py
What I want to do is to import the common class in the shared package from both packages. The two first packages do not require to interact together but needs that one class. The python programs might be runned with the console opened from within the two package folders themselves, such as:
cd Package1Folder
python SomeMainFile.py
If it is easier, the Python call could be like python Package1Folder/SomeMainFile.py but I need to plan this.
Could you provide how I could do the relative imports from package 1 or 2 for a file in the third shared package? Do I need an __init__.py file in the SomeRootFolder folder? I am always confused by relative imports and those import standards and syntax that are different between Python 2 and 3. Also could you validate to me that this is an acceptable way to proceed? Any other ideas?
Thanks all!
If you want to use relative imports,you need __init__.py in SharedPackageFolder folder,and you can use this to import OneCommonClassNeedsToBeShared.py:
from ..SharedPackageFolder import OneCommonClassNeedsToBeShared
See more details about Rationale for Relative Imports .
With the shift to absolute imports, the question arose whether
relative imports should be allowed at all. Several use cases were
presented, the most important of which is being able to rearrange the
structure of large packages without having to edit sub-packages. In
addition, a module inside a package can't easily import itself without
relative imports.
Also you can use absolute imports,relative imports are no longer strongly discouraged,using absolute_import is strongly suggested in some case.
You need to make sure SomeRootFolder is in your PYTHONPATH,or make this folder as sources root,it's more easier for you to import package or scripts in your large project,but sometimes you should be careful with absolute imports.
from SharedPackageFolder import OneCommonClassNeedsToBeShared.py
Absolute imports. From PEP 8:
Relative imports for intra-package imports are highly discouraged. Always use the absolute package path for all imports. Even now that
PEP 328 [7] is fully implemented in Python 2.5, its style of explicit
relative imports is actively discouraged; absolute imports are more
portable and usually more readable.
By the way,relative imports in Python 3 might return SystemError,have a look at Question:Relative imports in Python 3.
#vaultah offers some solutions,they might be helpful.
Hope this helps.
I write a package P. At the root of P there is a module m0.
Somewhere inside P there are modules m1,m2,... that need to import from m0
Of course I can write in each of these modules:
from P.m0 import ...
However if I change the name of P I have to revisit all the places and rewrite such statements.
I could also use relative imports but if I move a module at a different level in the package hierarchy I have to go fix the number of dots.
There are some other reasons too, but bottom line, I really want to say import from the module m0 that is located at the root of my package, what is the best way to express this?
That's not possible.
However, if you perform major refactoring where you move around modules between subpackages having to update some relative imports is not a huge problem.
Same applies for renaming the top-level package name if you do not use relative imports - that could even be done really fast with search-and-replace over all your files.
If you are willing to modify your question a tiny bit, you can get away with this.
IF the only entrypoints to your package are controlled; e.g. you only test your code by doing something like invoking testsuite package/.../module.py which will
THEN you can make sure that the first thing you do is import firstthing, and in package/firstthing.py you have:
import sys
import os.path
packageDir = os.path.split(__name__)[0]
sys.path[:] = sys.path+[packageDir] # or maybe you want it first...
The main caveat being that you will not be able to run python files without going through your entrypoints. I always want to do this for every project I write in python (to make relative imports work nicely), but I personally find this so inconvenient that I just give up.
There is also a second alternative. It is not that unreasonable to specify that your package requires another package in the python path. This package could be a utility package which performs a major hack. For example if the name of the package was "x", you could do import x which would use the inspect module to perform reflection on the interpreter stack, letting you figure out which module you were importing it from. Then you could do a sort of "backwards os.walk" by going up parent directories until you found the root of your package (by checking for some special indicator file, or manifest, or something). Then the code would programatically perform the above modification of the python path via sys.path. It's the same as the above, but you have the liberty to do things like run any python file without having to go through an awful entrypoint.
If you have extreme control over the shell environment, you can also just augment the $PYTHONPATH to include your package directory, but this is extremely fragile in many ways, and rather inelegant.
I'm developing a Python application; it has all its code in one package and runs inside this of course. The application's Python package is of no interest from the interpreter to the user, it's simply a GUI application.
The question is, which style is preferred when importing modules inside the application package
from application import settings, utils
or
from . import settings, utils
That is I can either specify the name as it is (here 'application') or I can say "current package" by using "."
This is a Free software package so the possibility exists that someone wants to make a fork of my application and change its name. In that case, alternative 1 is a slight nuisance. Still, I use style 1 all the time (although early code uses style 2 in some places), since style 1 looks much better.
Are there any arguments for my style (1) that I have missed? Or is it stupid not to go with style 2?
The Python Style Guide recommends explicitly against relative imports (the . style):
Relative imports for intra-package imports are highly discouraged.
Always use the absolute package path for all imports.
Even now that PEP 328 [7] is fully implemented in Python 2.5,
its style of explicit relative imports is actively discouraged;
absolute imports are more portable and usually more readable.
I tend to agree. Relative imports mean the same module is imported in different ways in different files, and requires that I remember what I'm looking at when reading and writing. Not really worth it, and a rename can be done with sed.
Besides the issue of renaming, the only problem with absolute imports is that import foo might mean the top-level module foo or a submodule foo beneath the current module. If this is a problem, you can use from __future__ import absolute_import; this is standard in Python 3.