I've done some research, and I came across the following article: http://effbot.org/zone/import-confusion.htm. While this seems to be a great guide, it was written in 1999, a while back. I'm am using Python 3.4.3, so I am thinking that some things have changed, which worries me, because I don't want to learn what is not applicable. Therefore, in Python 3, what are all of the ways to import packages and modules, in detail? Which ways are the most common and should be used above others?
The only ways that matter for ordinary usage are the first three ways listed on that page:
import module
from module import this, that, tother
from module import *
These haven't changed in Python 3. (Some of the details about where Python looks for the module.py file to load module have been tweaked, but the behavior of the import itself still works as described on the page you linked.)
One thing has been added, before Python 3 but since that article. That is explicit relative imports. These let you do things like from ..module import blah. This kind of import can only be used from inside a package; it lets modules in a package refer to other modules in the same package in a way that is relative to the package (i.e., without having to specify how to import the top-level package). You can read the details in PEP 328. Even this, though, is basically just a new variation on the from module import blah style syntax mentioned on the page you linked to.
__import__ also still works in Python 3. This is an internal function that you only would need to use if doing something rather unusual. The same applies to various functions in the importlib module (and the deprecated imp module). The exact level of wizardliness of these importing functions varies from one to another, but for ordinary usage of "I just want to import this module and use it", you essentially never need to use them. They're only needed if you want to do something like dynamically import a module whose name isn't known until runtime.
The Zen of Python gives you some hints:
>>> import this
The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
So given the simple, obvious method is: import module_name and it preserves namespaces I would suggest that while there are several import methods as you can see from the python3 manual entry and you can extend them by overriding the __import__() method or by rolling your own I would say stick with it until you have a good reason not to.
The fact that__import__() is surrounded by double underscores is also a hint to leave it alone.
If you are looking to understand the design decisions behind the import mechanisms then start with the manual then follow up into the PEPs 302 & 420 are good starting points.
I think import as tuple would be much better for readability and Maximum Line Length(pep8)
The import statement has two problems:
Long import statements can be difficult to write, requiring various
contortions to fit Pythonic style guidelines.
Imports can be ambiguous in the face of packages; within a package,
it's not clear whether import foo refers to a module within the package or some module outside the package.
golang language have the same thing for that
so would more prefer import kinda this
from package import (x, y)
instead of this
from authentication.views import SignupView, LoginView, VerificationView, SignupDetailView
https://legacy.python.org/dev/peps/pep-0328/
We can import modules in Python using the following ways
import module
from module import function
from module import *
Although using from module import *
is not a good practice, because of readability: Other programmer cannot understand what all are actually used in the current module. Memory overload: All are loaded in to memory. Best practices for using import in a module.
Say you have python modules (mymod1.py, mymod2.py files containing different functions) inside mypkg package (folder having init.py file, it can be an empty file).
#mymod1.py
def add_fun(a,b):
return a+b
def sub_fun(a,b):
return a-b
def mul_fun(a,b):
return a*b
def div_fun(a,b):
return a/b
#mymod2.py
def fun1(...):
........
........
def fun2(...):
........
........
Following are different ways to import:
from mypkg.mymod1 import * #import all the function from mymod1
add_fun(10, 20)
mul_fun(10, 2)
from mypkg.mymod1 import add_fun,div_fun #import only needed functions from mymod1
add_fun(10, 20)
div_mul(10, 2)
from mypkg import mymod1 #import mymod module
mymod1.add_fun(10, 20)
mymod1.mul_fun(10, 2)
import mypkg #import package and use different models inside it
mypkg.mymod1.add_fun(10, 20)
mypkg.mymod1.mul_fun(10, 2)
Related
I've generally been told that the following is bad practice.
from module import *
The main reasoning (or so I've been told), is that you could possibly import something you didn't want, and it could shadow a similarly named function or class from another module.
However, what about PyQt
from PyQt4.QtCore import *
Every example I've ever seen is written this way, mainly because everything exported from Qt starts with "Q", so it's not going to shadow anything.
What's the concensus? Is it always bad to use * imports?
EDIT:
Just to be clear, this question is specifically in regards to using PyQt4. It has nothing to do with the way I am designing some other project.
Basically, I've found that coding to PEP8 has improved my code readability, except with regards to importing PyQt4, and so I've disregarded the frowns from purists until now. But now my dev group is deciding on one convention and I'm wondering if this is a scenario "where practicality beats purity", or if I should just suck it up and deal with monstrous PyQt4 imports
from PyQt4.QtGui import QComboBox, QLineEdit, QLayout, Q;lakdfaf.......
This can sort of turn into a religious war. It's a matter of whether you want to be explicit or whether you want to avoid being too verbose. In general, following the Zen of Python, it's better to be explicit, but sometimes people just don't find it practical to list every import from particular module.
My general rule is that if I didn't write the module, I don't import it all. My biggest fear is actually over writing local variables that might have been defined in the imported module. So to keep from having to type in long module names, I use the import as feature. Using your module as an example I would do the following:
import PyQt4.QtCore as qt
That being said, I have many support modules that I write that I will import everything. Like the pyqt module, I name them with a descriptive name that helps show which module it came from.
Edit per comment
When I use import*, my support modules do not contain classes or anything that can create a new instance. They tend to be groups of functions that modify existing instances only. To help clarify my opinion: If I am the owner of the source code and I will be the primary maintainer, I will use the import* otherwise I would use the import as.
Another reason that I use the import as feature is to allow me to mock modules for debugging purposes. In a project that I am working on now, I use pyVisa to talk to a number of GPIB devices. When I'm not connected to the devices GPIB network, I can use a dummy_visa module to write to the stdout(to verify I am sending the correct format) and return a random number (to test my application). See below
if visa_debug:
import dummy_visa as visa
else:
import visa
gpib = visa.Instrument("GPIB0::10")
gpib.write("MEAS:VOLT?")
Making an explicit exception for modules that already include a namespace in their naming convention (such as the Q* of PyQT) is perfectly reasonably. However, I recommend being clear that the default is still "don't use it" and simply list this exception in your coding guidelines.
import * is also acceptable when it is used as a namespace manipulation trick within an application (the two forms of that I am familiar with are optional C acceleration modules that are imported at the end of the pure Python version, and "flattening" a package namespace in __init__). The key point is that the importing module and the module being imported are under the control of the same set of developers, and hence avoiding namespace clashes is completely within their control.
The final exception is for convenience at the interactive prompt.
In other situations, it is best to either import specific names or reference them indirectly through the module name (or, if there are some commonly reference items, do both:
import module # Can access anything
from module import a, b, c # But we reference these a lot, so retrieve them directly
Tutorial, chapter 6:
Note that in general the practice of importing * from a module or package is frowned upon, since it often causes poorly readable code. However, it is okay to use it to save typing in interactive sessions.
Tutorial, chapter 10:
Be sure to use the import os style instead of from os import *. This will keep os.open() from shadowing the built-in open() function which operates much differently.
So it seems that it is definitely a bad idea sometimes; not-the-best idea most of the time; and an acceptable shortcut in cases where you'd want to type less, eg in interactive sessions.
I find import * gets abused, and can become a maintenance headache, so I avoid it for this and the other reasons you state. That said I feel it's okay for short interactive sessions, e.g. from pylab import *.
In production code, for packages like PyQt4.QtCore where you plan to use many of the symbols, I'd use one of the following syntaxes which make it explicit which namespace the symbols come from:
from PyQt4 import QtCore
# explicit where the symbol came from
QtCore.QTime()
import PyQt4.QtCore as QT
# less desirable since you need to look in the header to find out what QT is
# but I still prefer it to import *
QT.QTime()
In general, if you're going to use from X import Y, it's a good idea to be explicit about what you're importing. That's not just because it's safer, but also because it makes your code more readable (and upgrades to the third-party modules you're using won't have as much potential to incidentally break your code).
In some code examples demonstrating big packages, like Qt or matplotlib, the examples will use from module import * because they're often only importing from one module, and it saves typing and lets their example code get to the point. There's no reason you can't do it in your code, but at least use it in moderation, especially if it's in big source files or other people will be looking at your code.
The PyQt design is water under the bridge. I'm not sure it's the best thing, but it's probably influenced by the way Qt was designed. AFAIK, Qt didn't use C++ namespaces historically (I'm not sure if it uses them now), and therefore had to use prefixes. But all of those design decisions probably happened more than 10 years ago, and that shouldn't affect your design decisions now.
I know that if I was designing a library now, I would definitely choose to use package namespaces (which you would import explicitly in Python) over prefixes. Let's say my prefix is Pfx - if the library user doesn't care writing PfxFunc() every time he needs a function, he certainly wouldn't care about the extra character in Pfx.Func(), especially when he can use import with specific symbol names to shorten that even more.
Consider this case
from foo import *
from bar import *
x=baz()
Now suppose foo has a function called baz() that we are using in our code and everything is working fine. Months or years pass, the author of bar adds a function called baz(). Someone updates the egg ang bingo - we have a potentially hard to detect bug in the program.
Not to mention that just looking at those three lines of code, I can't tell where baz comes from without going and looking at foo and bar
The only time I'd use import * would be in the interpreter to save some typing
I want to know why we use from module import module?
For example from BeautifulSoup import BeautifulSoup instead of only import BeautifulSoup.
Are both of those different?
And if yes, then how?
Yes, they are different.
There are several ways to import a module
import module
The module is imported, but all its functions, classes and variables remain within the name space of 'module . To reference them, they must be prepended with the module name:
module.somefunction() module.someclass() module.somevariable = 1
import module as md
The module is imported, but given a new name space 'md'. To reference the functions, classes and variables, they must be prepended with the new namespace:
md.somefunction() module.someclass() module.somevariable = 1
This keeps the namespaces separated, but provides a shorthand notation which makes the code more readable.
from module import *
All functions, classes and variabes from the module are imported into the current namespace as if they were defined in the current module. They can be called with their own name:
somefunction() someclass() somevariable = 1
The disadvantage is that there might be overlapping names!
from module import something
from module import something, somethingelse
This imports only 'something' (and 'somethingelse') a function, a class, a variable into the current namespace.
This is not only more efficient, but it also reduces the risk of overlapping names.
A module name and a class inside the module may have the same name. Don't let that confuse you:
import BeautifulSoup reference: BeautifulSoup.BeautifulSoup()
from BeautifulSoup import BeautifulSoup reference: BeautifulSoup()
import BeautifulSoup as bs reference: bs.BeautifulSoup()
I think it comes down to what the individual library (Python package) owner wants to do and however they think it will be easiest for people to use their product. As part of the Python community, they may take into account the current conventions. Those may be done formally as a PEP like PEP 8, informally through common practice, or as third-party formalized linters.
It can be confusing that there are some distinctions, like package vs module, that have the potential to help with understanding, but do not really mean anything to a Python application itself. For example, with your example, it is not from module import module; it is more like from package import class for BeautifulSoup. But it could be argued that a package is an external module. I have heard some call any file a module and any directory a package.
More background, if you are new to python:
In python it is all about namespacing. Namespacing has the potential to allow for very clean and flexible code organization compared to earlier languages. However it also allows that any package, module, class, function, method, variable, or whatever name can be hard for someone writing a client application to know what is what. There is even a convenience class in the standard library for faking it: types.Namespace. Then there are metaclasses and dynamic programming as well. Anyway, yes there are common conventions followed to reduce confusion, but they have to be learned separately from the language itself (like learning customs of a country, not only its language, to be able to understand common phrases).
What PEP 20 has to say about namespaces:
import this
Back to PEP 8, generally, it is accepted that classes are word-uppercase (class SomeClass:) without underscores, and functions, variables and methods are all lowercase with words separated by underscores (some_variable). Not always the case, but those are probably the most widely accepted styles. Somewhere I see things going against what is "pythonic" and/or commonplace is when a bindings library is a thin wrapper around a library from another language (e.g. C++) or when the Python code is tightly coupled with code in another language, so the styles smear together for easier transitions. Nothing says anyone has to follow specific style.
Some people prefer terseness, so they may shorten and combine words for variable names (e.g. foo bar as fb). Others may like to be explicit (e.g. foo bar as foo_bar). And still others may prefer to be generic and use comments to describe their variables (why? it may be convenient for large complex equations):
# x is for foo bar.
# y is for spam and z is for ham.
assert x + y != z
Some people really like formatters like Black that follow very strict rules, but others like flexibility or have reason to go against convention. Even in the standard libraries that come with Python, there are inconsistency from legacy code that the maintainers have left alone or allowed before the community settled on common practices that the code goes against.
I created a module named util that provides classes and functions I often use in Python.
Some of them need imported features. What are the pros and the cons of importing needed things inside class/function definition? Is it better than import at the beginning of a module file? Is it a good idea?
It's the most common style to put every import at the top of the file. PEP 8 recommends it, which is a good reason to do it to start with. But that's not a whim, it has advantages (although not critical enough to make everything else a crime). It allows finding all imports at a glance, as opposed to looking through the whole file. It also ensures everything is imported before any other code (which may depend on some imports) is executed. NameErrors are usually easy to resolve, but they can be annoying.
There's no (significant) namespace pollution to be avoided by keeping the module in a smaller scope, since all you add is the actual module (no, import * doesn't count and probably shouldn't be used anyway). Inside functions, you'd import again on every call (not really harmful since everything is imported once, but uncalled for).
PEP8, the Python style guide, states that:
Imports are always put at the top of
the file, just after any module
comments and docstrings, and before module globals and constants.
Of course this is no hard and fast rule, and imports can go anywhere you want them to. But putting them at the top is the best way to go about it. You can of course import within functions or a class.
But note you cannot do this:
def foo():
from os import *
Because:
SyntaxWarning: import * only allowed at module level
Like flying sheep's answer, I agree that the others are right, but I put imports in other places like in __init__() routines and function calls when I am DEVELOPING code. After my class or function has been tested and proven to work with the import inside of it, I normally give it its own module with the import following PEP8 guidelines. I do this because sometimes I forget to delete imports after refactoring code or removing old code with bad ideas. By keeping the imports inside the class or function under development, I am specifying its dependencies should I want to copy it elsewhere or promote it to its own module...
Only move imports into a local scope, such as inside a function definition, if it’s necessary to solve a problem such as avoiding a circular import or are trying to reduce the initialization time of a module. This technique is especially helpful if many of the imports are unnecessary depending on how the program executes. You may also want to move imports into a function if the modules are only ever used in that function. Note that loading a module the first time may be expensive because of the one time initialization of the module, but loading a module multiple times is virtually free, costing only a couple of dictionary lookups. Even if the module name has gone out of scope, the module is probably available in sys.modules.
https://docs.python.org/3/faq/programming.html#what-are-the-best-practices-for-using-import-in-a-module
I believe that it's best practice (according to some PEP's) that you keep import statements at the beginning of a module. You can add import statements to an __init__.py file, which will import those module to all modules inside the package.
So...it's certainly something you can do the way you're doing it, but it's discouraged and actually unnecessary.
While the other answers are mostly right, there is a reason why python allows this.
It is not smart to import redundant stuff which isn’t needed. So, if you want to e.g. parse XML into an element tree, but don’t want to use the slow builtin XML parser if lxml is available, you would need to check this the moment you need to invoke the parser.
And instead of memorizing the availability of lxml at the beginning, I would prefer to try importing and using lxml, except it’s not there, in which case I’d fallback to the builtin xml module.
From what I've read and the reasons given, I know that the from foo import * method is bad practice, due to complications with variable assignment as well as memory problems. However, would the tkinter module be an exception to this rule?
I figure that when using a GUI, one may be looking to use perhaps most of the functions/methods from the respective module, where in comparison to something like itertools, one may only need access to a handful of modules (one wouldn't need to use permutations and combinations).
Independent of whether tkinter is an exception, are there any modules that are?
There are numerous problems with from foo import *; it can import lots of stuff that you didn't expect to, and especially something that overwrite the names of built-in functions or something that you imported from other modules. Worse yet, your program might work now, but when in a later version the author of foo module adds function bar in the module, your code could suddenly break if it expected bar to be something else.
The preferred way with long modules is often to import the module with a short alias:
import tkinter as tk
and then use for example tk.Button. This is the idiom used by the documentation.
Python has a style guide, which is explicit about the one acceptable use for wildcard imports:
Wildcard imports (from <module> import *) should be avoided, as they make it unclear which names are present in the namespace, confusing both readers and many automated tools. There is one defensible use case for a wildcard import, which is to republish an internal interface as part of a public API (for example, overwriting a pure Python implementation of an interface with the definitions from an optional accelerator module and exactly which definitions will be overwritten isn't known in advance).
I've generally been told that the following is bad practice.
from module import *
The main reasoning (or so I've been told), is that you could possibly import something you didn't want, and it could shadow a similarly named function or class from another module.
However, what about PyQt
from PyQt4.QtCore import *
Every example I've ever seen is written this way, mainly because everything exported from Qt starts with "Q", so it's not going to shadow anything.
What's the concensus? Is it always bad to use * imports?
EDIT:
Just to be clear, this question is specifically in regards to using PyQt4. It has nothing to do with the way I am designing some other project.
Basically, I've found that coding to PEP8 has improved my code readability, except with regards to importing PyQt4, and so I've disregarded the frowns from purists until now. But now my dev group is deciding on one convention and I'm wondering if this is a scenario "where practicality beats purity", or if I should just suck it up and deal with monstrous PyQt4 imports
from PyQt4.QtGui import QComboBox, QLineEdit, QLayout, Q;lakdfaf.......
This can sort of turn into a religious war. It's a matter of whether you want to be explicit or whether you want to avoid being too verbose. In general, following the Zen of Python, it's better to be explicit, but sometimes people just don't find it practical to list every import from particular module.
My general rule is that if I didn't write the module, I don't import it all. My biggest fear is actually over writing local variables that might have been defined in the imported module. So to keep from having to type in long module names, I use the import as feature. Using your module as an example I would do the following:
import PyQt4.QtCore as qt
That being said, I have many support modules that I write that I will import everything. Like the pyqt module, I name them with a descriptive name that helps show which module it came from.
Edit per comment
When I use import*, my support modules do not contain classes or anything that can create a new instance. They tend to be groups of functions that modify existing instances only. To help clarify my opinion: If I am the owner of the source code and I will be the primary maintainer, I will use the import* otherwise I would use the import as.
Another reason that I use the import as feature is to allow me to mock modules for debugging purposes. In a project that I am working on now, I use pyVisa to talk to a number of GPIB devices. When I'm not connected to the devices GPIB network, I can use a dummy_visa module to write to the stdout(to verify I am sending the correct format) and return a random number (to test my application). See below
if visa_debug:
import dummy_visa as visa
else:
import visa
gpib = visa.Instrument("GPIB0::10")
gpib.write("MEAS:VOLT?")
Making an explicit exception for modules that already include a namespace in their naming convention (such as the Q* of PyQT) is perfectly reasonably. However, I recommend being clear that the default is still "don't use it" and simply list this exception in your coding guidelines.
import * is also acceptable when it is used as a namespace manipulation trick within an application (the two forms of that I am familiar with are optional C acceleration modules that are imported at the end of the pure Python version, and "flattening" a package namespace in __init__). The key point is that the importing module and the module being imported are under the control of the same set of developers, and hence avoiding namespace clashes is completely within their control.
The final exception is for convenience at the interactive prompt.
In other situations, it is best to either import specific names or reference them indirectly through the module name (or, if there are some commonly reference items, do both:
import module # Can access anything
from module import a, b, c # But we reference these a lot, so retrieve them directly
Tutorial, chapter 6:
Note that in general the practice of importing * from a module or package is frowned upon, since it often causes poorly readable code. However, it is okay to use it to save typing in interactive sessions.
Tutorial, chapter 10:
Be sure to use the import os style instead of from os import *. This will keep os.open() from shadowing the built-in open() function which operates much differently.
So it seems that it is definitely a bad idea sometimes; not-the-best idea most of the time; and an acceptable shortcut in cases where you'd want to type less, eg in interactive sessions.
I find import * gets abused, and can become a maintenance headache, so I avoid it for this and the other reasons you state. That said I feel it's okay for short interactive sessions, e.g. from pylab import *.
In production code, for packages like PyQt4.QtCore where you plan to use many of the symbols, I'd use one of the following syntaxes which make it explicit which namespace the symbols come from:
from PyQt4 import QtCore
# explicit where the symbol came from
QtCore.QTime()
import PyQt4.QtCore as QT
# less desirable since you need to look in the header to find out what QT is
# but I still prefer it to import *
QT.QTime()
In general, if you're going to use from X import Y, it's a good idea to be explicit about what you're importing. That's not just because it's safer, but also because it makes your code more readable (and upgrades to the third-party modules you're using won't have as much potential to incidentally break your code).
In some code examples demonstrating big packages, like Qt or matplotlib, the examples will use from module import * because they're often only importing from one module, and it saves typing and lets their example code get to the point. There's no reason you can't do it in your code, but at least use it in moderation, especially if it's in big source files or other people will be looking at your code.
The PyQt design is water under the bridge. I'm not sure it's the best thing, but it's probably influenced by the way Qt was designed. AFAIK, Qt didn't use C++ namespaces historically (I'm not sure if it uses them now), and therefore had to use prefixes. But all of those design decisions probably happened more than 10 years ago, and that shouldn't affect your design decisions now.
I know that if I was designing a library now, I would definitely choose to use package namespaces (which you would import explicitly in Python) over prefixes. Let's say my prefix is Pfx - if the library user doesn't care writing PfxFunc() every time he needs a function, he certainly wouldn't care about the extra character in Pfx.Func(), especially when he can use import with specific symbol names to shorten that even more.
Consider this case
from foo import *
from bar import *
x=baz()
Now suppose foo has a function called baz() that we are using in our code and everything is working fine. Months or years pass, the author of bar adds a function called baz(). Someone updates the egg ang bingo - we have a potentially hard to detect bug in the program.
Not to mention that just looking at those three lines of code, I can't tell where baz comes from without going and looking at foo and bar
The only time I'd use import * would be in the interpreter to save some typing