Related
I've generally been told that the following is bad practice.
from module import *
The main reasoning (or so I've been told), is that you could possibly import something you didn't want, and it could shadow a similarly named function or class from another module.
However, what about PyQt
from PyQt4.QtCore import *
Every example I've ever seen is written this way, mainly because everything exported from Qt starts with "Q", so it's not going to shadow anything.
What's the concensus? Is it always bad to use * imports?
EDIT:
Just to be clear, this question is specifically in regards to using PyQt4. It has nothing to do with the way I am designing some other project.
Basically, I've found that coding to PEP8 has improved my code readability, except with regards to importing PyQt4, and so I've disregarded the frowns from purists until now. But now my dev group is deciding on one convention and I'm wondering if this is a scenario "where practicality beats purity", or if I should just suck it up and deal with monstrous PyQt4 imports
from PyQt4.QtGui import QComboBox, QLineEdit, QLayout, Q;lakdfaf.......
This can sort of turn into a religious war. It's a matter of whether you want to be explicit or whether you want to avoid being too verbose. In general, following the Zen of Python, it's better to be explicit, but sometimes people just don't find it practical to list every import from particular module.
My general rule is that if I didn't write the module, I don't import it all. My biggest fear is actually over writing local variables that might have been defined in the imported module. So to keep from having to type in long module names, I use the import as feature. Using your module as an example I would do the following:
import PyQt4.QtCore as qt
That being said, I have many support modules that I write that I will import everything. Like the pyqt module, I name them with a descriptive name that helps show which module it came from.
Edit per comment
When I use import*, my support modules do not contain classes or anything that can create a new instance. They tend to be groups of functions that modify existing instances only. To help clarify my opinion: If I am the owner of the source code and I will be the primary maintainer, I will use the import* otherwise I would use the import as.
Another reason that I use the import as feature is to allow me to mock modules for debugging purposes. In a project that I am working on now, I use pyVisa to talk to a number of GPIB devices. When I'm not connected to the devices GPIB network, I can use a dummy_visa module to write to the stdout(to verify I am sending the correct format) and return a random number (to test my application). See below
if visa_debug:
import dummy_visa as visa
else:
import visa
gpib = visa.Instrument("GPIB0::10")
gpib.write("MEAS:VOLT?")
Making an explicit exception for modules that already include a namespace in their naming convention (such as the Q* of PyQT) is perfectly reasonably. However, I recommend being clear that the default is still "don't use it" and simply list this exception in your coding guidelines.
import * is also acceptable when it is used as a namespace manipulation trick within an application (the two forms of that I am familiar with are optional C acceleration modules that are imported at the end of the pure Python version, and "flattening" a package namespace in __init__). The key point is that the importing module and the module being imported are under the control of the same set of developers, and hence avoiding namespace clashes is completely within their control.
The final exception is for convenience at the interactive prompt.
In other situations, it is best to either import specific names or reference them indirectly through the module name (or, if there are some commonly reference items, do both:
import module # Can access anything
from module import a, b, c # But we reference these a lot, so retrieve them directly
Tutorial, chapter 6:
Note that in general the practice of importing * from a module or package is frowned upon, since it often causes poorly readable code. However, it is okay to use it to save typing in interactive sessions.
Tutorial, chapter 10:
Be sure to use the import os style instead of from os import *. This will keep os.open() from shadowing the built-in open() function which operates much differently.
So it seems that it is definitely a bad idea sometimes; not-the-best idea most of the time; and an acceptable shortcut in cases where you'd want to type less, eg in interactive sessions.
I find import * gets abused, and can become a maintenance headache, so I avoid it for this and the other reasons you state. That said I feel it's okay for short interactive sessions, e.g. from pylab import *.
In production code, for packages like PyQt4.QtCore where you plan to use many of the symbols, I'd use one of the following syntaxes which make it explicit which namespace the symbols come from:
from PyQt4 import QtCore
# explicit where the symbol came from
QtCore.QTime()
import PyQt4.QtCore as QT
# less desirable since you need to look in the header to find out what QT is
# but I still prefer it to import *
QT.QTime()
In general, if you're going to use from X import Y, it's a good idea to be explicit about what you're importing. That's not just because it's safer, but also because it makes your code more readable (and upgrades to the third-party modules you're using won't have as much potential to incidentally break your code).
In some code examples demonstrating big packages, like Qt or matplotlib, the examples will use from module import * because they're often only importing from one module, and it saves typing and lets their example code get to the point. There's no reason you can't do it in your code, but at least use it in moderation, especially if it's in big source files or other people will be looking at your code.
The PyQt design is water under the bridge. I'm not sure it's the best thing, but it's probably influenced by the way Qt was designed. AFAIK, Qt didn't use C++ namespaces historically (I'm not sure if it uses them now), and therefore had to use prefixes. But all of those design decisions probably happened more than 10 years ago, and that shouldn't affect your design decisions now.
I know that if I was designing a library now, I would definitely choose to use package namespaces (which you would import explicitly in Python) over prefixes. Let's say my prefix is Pfx - if the library user doesn't care writing PfxFunc() every time he needs a function, he certainly wouldn't care about the extra character in Pfx.Func(), especially when he can use import with specific symbol names to shorten that even more.
Consider this case
from foo import *
from bar import *
x=baz()
Now suppose foo has a function called baz() that we are using in our code and everything is working fine. Months or years pass, the author of bar adds a function called baz(). Someone updates the egg ang bingo - we have a potentially hard to detect bug in the program.
Not to mention that just looking at those three lines of code, I can't tell where baz comes from without going and looking at foo and bar
The only time I'd use import * would be in the interpreter to save some typing
I want to know why we use from module import module?
For example from BeautifulSoup import BeautifulSoup instead of only import BeautifulSoup.
Are both of those different?
And if yes, then how?
Yes, they are different.
There are several ways to import a module
import module
The module is imported, but all its functions, classes and variables remain within the name space of 'module . To reference them, they must be prepended with the module name:
module.somefunction() module.someclass() module.somevariable = 1
import module as md
The module is imported, but given a new name space 'md'. To reference the functions, classes and variables, they must be prepended with the new namespace:
md.somefunction() module.someclass() module.somevariable = 1
This keeps the namespaces separated, but provides a shorthand notation which makes the code more readable.
from module import *
All functions, classes and variabes from the module are imported into the current namespace as if they were defined in the current module. They can be called with their own name:
somefunction() someclass() somevariable = 1
The disadvantage is that there might be overlapping names!
from module import something
from module import something, somethingelse
This imports only 'something' (and 'somethingelse') a function, a class, a variable into the current namespace.
This is not only more efficient, but it also reduces the risk of overlapping names.
A module name and a class inside the module may have the same name. Don't let that confuse you:
import BeautifulSoup reference: BeautifulSoup.BeautifulSoup()
from BeautifulSoup import BeautifulSoup reference: BeautifulSoup()
import BeautifulSoup as bs reference: bs.BeautifulSoup()
I think it comes down to what the individual library (Python package) owner wants to do and however they think it will be easiest for people to use their product. As part of the Python community, they may take into account the current conventions. Those may be done formally as a PEP like PEP 8, informally through common practice, or as third-party formalized linters.
It can be confusing that there are some distinctions, like package vs module, that have the potential to help with understanding, but do not really mean anything to a Python application itself. For example, with your example, it is not from module import module; it is more like from package import class for BeautifulSoup. But it could be argued that a package is an external module. I have heard some call any file a module and any directory a package.
More background, if you are new to python:
In python it is all about namespacing. Namespacing has the potential to allow for very clean and flexible code organization compared to earlier languages. However it also allows that any package, module, class, function, method, variable, or whatever name can be hard for someone writing a client application to know what is what. There is even a convenience class in the standard library for faking it: types.Namespace. Then there are metaclasses and dynamic programming as well. Anyway, yes there are common conventions followed to reduce confusion, but they have to be learned separately from the language itself (like learning customs of a country, not only its language, to be able to understand common phrases).
What PEP 20 has to say about namespaces:
import this
Back to PEP 8, generally, it is accepted that classes are word-uppercase (class SomeClass:) without underscores, and functions, variables and methods are all lowercase with words separated by underscores (some_variable). Not always the case, but those are probably the most widely accepted styles. Somewhere I see things going against what is "pythonic" and/or commonplace is when a bindings library is a thin wrapper around a library from another language (e.g. C++) or when the Python code is tightly coupled with code in another language, so the styles smear together for easier transitions. Nothing says anyone has to follow specific style.
Some people prefer terseness, so they may shorten and combine words for variable names (e.g. foo bar as fb). Others may like to be explicit (e.g. foo bar as foo_bar). And still others may prefer to be generic and use comments to describe their variables (why? it may be convenient for large complex equations):
# x is for foo bar.
# y is for spam and z is for ham.
assert x + y != z
Some people really like formatters like Black that follow very strict rules, but others like flexibility or have reason to go against convention. Even in the standard libraries that come with Python, there are inconsistency from legacy code that the maintainers have left alone or allowed before the community settled on common practices that the code goes against.
I'm authoring a set of python coding guidelines for a team of ~30 developers. As a basis for my document, so far I've studied the Google python style guide and the PEP 8 style guide, and incorporated information from both.
One place where the Google style guide is more restrictive than PEP 8 is with imports. The Google guide requests developers only import packages and modules only, and then refer to items within by a more-qualified name. For example:
from pkg import module
...
my_class = module.MyClass()
The justification is that the "source of each identifier is indicated in a consistent way". For our project, we intend to organize with packages two or three levels deep, so to know the full source of the identifier, the reader will likely need to examine the import statement anyway. I'd like to advocate this style of import as a "preferred style":
from pkg.module import MyClass
...
my_class = MyClass()
IMHO, the readability in python constructs such as list comprehensions is improved when the names are more succinct.
What I'm unclear on is what the python interpreter might do behind the scenes. For example, is MyClass now part of the global namespace for both this module, and all importers of this module? (This would be bad, could lead to some weird bugs; if this were true, I'd advocate the Google style).
My python development experience is limited to about 6 months (and there are not many experts on our project to consult), so I wanted to get more information from the community. Here are some items I've researched already:
effbot - discussion on imports
stack overflow - import vs. from import
python documentation - modules
python documentation - import
Thank you for your responses!
In Python, there is no such thing as a variable that is global across more than one module. If you do from pkg.module import MyClass, then MyClass is in the global namespace of the module where you do that, but not of any other module (including modules that import the module that imports MyClass).
As for your more general question, either import mechanism can be acceptable depending on the situation. If the module name is long, you can get some shortening by importing it under a different name:
# Awkward
from package import reallylongmodule
reallylongmodule.MyClass()
# Less awkward
from package import reallylongmodule as rlm
rlm.MyClass()
Importing just the class can be okay if the class name is distinctive enough that you can tell where it comes from and what it is. However, if you have multiple modules that define classes with relatively undescriptive names (e.g., "Processor", "Unit", "Data", "Manager"), then it can be a good idea to access them via the module name to clarify what you're doing.
Style guides are ultimately guides and not laws. My own preference would be to choose a mechanism that maximizes clarity and readability. That involves a tradeoff between avoiding long and cumbersome names, and also avoiding short, vague, or cryptic names. How you make that tradeoff depends on the particular libraries you're using and how you're using them (e.g., how many modules you import, how many things you import from them).
I suggest you to use automatic code checkers, like pylint, pep8, pyflakes, instead of writing code guides.
I personally prefer to use from pkg import module, because of possible name collisions.
from package import module
def my_fun():
module.function()
Inpreter must do 3 hash-table lookups local function namespace, current module's global namespace and imported module's namespace.
In
from package.module import function
def my_fun():
function()
it will do only 2 lookups: the last one performed in import time.
I've generally been told that the following is bad practice.
from module import *
The main reasoning (or so I've been told), is that you could possibly import something you didn't want, and it could shadow a similarly named function or class from another module.
However, what about PyQt
from PyQt4.QtCore import *
Every example I've ever seen is written this way, mainly because everything exported from Qt starts with "Q", so it's not going to shadow anything.
What's the concensus? Is it always bad to use * imports?
EDIT:
Just to be clear, this question is specifically in regards to using PyQt4. It has nothing to do with the way I am designing some other project.
Basically, I've found that coding to PEP8 has improved my code readability, except with regards to importing PyQt4, and so I've disregarded the frowns from purists until now. But now my dev group is deciding on one convention and I'm wondering if this is a scenario "where practicality beats purity", or if I should just suck it up and deal with monstrous PyQt4 imports
from PyQt4.QtGui import QComboBox, QLineEdit, QLayout, Q;lakdfaf.......
This can sort of turn into a religious war. It's a matter of whether you want to be explicit or whether you want to avoid being too verbose. In general, following the Zen of Python, it's better to be explicit, but sometimes people just don't find it practical to list every import from particular module.
My general rule is that if I didn't write the module, I don't import it all. My biggest fear is actually over writing local variables that might have been defined in the imported module. So to keep from having to type in long module names, I use the import as feature. Using your module as an example I would do the following:
import PyQt4.QtCore as qt
That being said, I have many support modules that I write that I will import everything. Like the pyqt module, I name them with a descriptive name that helps show which module it came from.
Edit per comment
When I use import*, my support modules do not contain classes or anything that can create a new instance. They tend to be groups of functions that modify existing instances only. To help clarify my opinion: If I am the owner of the source code and I will be the primary maintainer, I will use the import* otherwise I would use the import as.
Another reason that I use the import as feature is to allow me to mock modules for debugging purposes. In a project that I am working on now, I use pyVisa to talk to a number of GPIB devices. When I'm not connected to the devices GPIB network, I can use a dummy_visa module to write to the stdout(to verify I am sending the correct format) and return a random number (to test my application). See below
if visa_debug:
import dummy_visa as visa
else:
import visa
gpib = visa.Instrument("GPIB0::10")
gpib.write("MEAS:VOLT?")
Making an explicit exception for modules that already include a namespace in their naming convention (such as the Q* of PyQT) is perfectly reasonably. However, I recommend being clear that the default is still "don't use it" and simply list this exception in your coding guidelines.
import * is also acceptable when it is used as a namespace manipulation trick within an application (the two forms of that I am familiar with are optional C acceleration modules that are imported at the end of the pure Python version, and "flattening" a package namespace in __init__). The key point is that the importing module and the module being imported are under the control of the same set of developers, and hence avoiding namespace clashes is completely within their control.
The final exception is for convenience at the interactive prompt.
In other situations, it is best to either import specific names or reference them indirectly through the module name (or, if there are some commonly reference items, do both:
import module # Can access anything
from module import a, b, c # But we reference these a lot, so retrieve them directly
Tutorial, chapter 6:
Note that in general the practice of importing * from a module or package is frowned upon, since it often causes poorly readable code. However, it is okay to use it to save typing in interactive sessions.
Tutorial, chapter 10:
Be sure to use the import os style instead of from os import *. This will keep os.open() from shadowing the built-in open() function which operates much differently.
So it seems that it is definitely a bad idea sometimes; not-the-best idea most of the time; and an acceptable shortcut in cases where you'd want to type less, eg in interactive sessions.
I find import * gets abused, and can become a maintenance headache, so I avoid it for this and the other reasons you state. That said I feel it's okay for short interactive sessions, e.g. from pylab import *.
In production code, for packages like PyQt4.QtCore where you plan to use many of the symbols, I'd use one of the following syntaxes which make it explicit which namespace the symbols come from:
from PyQt4 import QtCore
# explicit where the symbol came from
QtCore.QTime()
import PyQt4.QtCore as QT
# less desirable since you need to look in the header to find out what QT is
# but I still prefer it to import *
QT.QTime()
In general, if you're going to use from X import Y, it's a good idea to be explicit about what you're importing. That's not just because it's safer, but also because it makes your code more readable (and upgrades to the third-party modules you're using won't have as much potential to incidentally break your code).
In some code examples demonstrating big packages, like Qt or matplotlib, the examples will use from module import * because they're often only importing from one module, and it saves typing and lets their example code get to the point. There's no reason you can't do it in your code, but at least use it in moderation, especially if it's in big source files or other people will be looking at your code.
The PyQt design is water under the bridge. I'm not sure it's the best thing, but it's probably influenced by the way Qt was designed. AFAIK, Qt didn't use C++ namespaces historically (I'm not sure if it uses them now), and therefore had to use prefixes. But all of those design decisions probably happened more than 10 years ago, and that shouldn't affect your design decisions now.
I know that if I was designing a library now, I would definitely choose to use package namespaces (which you would import explicitly in Python) over prefixes. Let's say my prefix is Pfx - if the library user doesn't care writing PfxFunc() every time he needs a function, he certainly wouldn't care about the extra character in Pfx.Func(), especially when he can use import with specific symbol names to shorten that even more.
Consider this case
from foo import *
from bar import *
x=baz()
Now suppose foo has a function called baz() that we are using in our code and everything is working fine. Months or years pass, the author of bar adds a function called baz(). Someone updates the egg ang bingo - we have a potentially hard to detect bug in the program.
Not to mention that just looking at those three lines of code, I can't tell where baz comes from without going and looking at foo and bar
The only time I'd use import * would be in the interpreter to save some typing
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Perl's AUTOLOAD in Python (getattr on a module)
I'm coming from a PHP background and attempting to learn Python, and I want to be sure to do things the "Python way" instead of how i've developed before.
My question comes from the fact in PHP5 you can set up your code so if you attempt to call a class that doesn't exist in the namespace, a function will run first that will load the class in and allow you to continue on as if it were already loaded. the advantages to this is classes weren't loaded unless they were called, and you didn't have to worry about loading classes before using them.
In python, there's alot of emphasis on the import statement, is it bad practice to attempt an auto importing trick with python, to alleviate the need for an import statement? I've found this module that offers auto importing, however I dont know if that's the best way of doing it, or if auto importing of modules is something that is recommended, thoughts?
Imports serve at least two other important purposes besides making the modules or contents of the modules available:
They serve as a sort of declaration of intent -- "this module uses services from this other module" or "this module uses services belonging to a certain class" -- e.g. if you are doing a security review for socket-handling code, you can begin by only looking at modules that import socket (or other networking-related modules)
Imports serve as a proxy for the complexity of a module. If you find yourself with dozens of lines of imports, it may be time to reconsider your separation of concerns within the module, or within your application as a whole. This is also a good reason to avoid "from foo import *"-type imports.
In Python, people usually avoid auto imports, just because it is not worth the effort. You may slightly remove startup costs, but otherwise, there is no (or should be no) significant effect. If you have modules that are expensive to import and do a lot of stuff that doesn't need to be done, rather rewrite the module than delay importing it.
That said, there is nothing inherently wrong with auto imports. Because of the proxy nature, there may be some pitfalls (e.g. when looking at a thing that has not been imported yet). Several auto importing libraries are floating around.
If you are learning Python and want to do things the Python way, then just import the modules. It's very unusual to find autoimports in Python code.
You could auto-import the modules, but the most I have ever needed to import was about 10, and that is after I tacked features on top of the original program. You won't be importing a lot, and the names are very easy to remember.