This question already has answers here:
What's Python good practice for importing and offering optional features?
(7 answers)
Closed last month.
I have a package that allows the user to use any one of 4 packages they want to connect to a database. It works great but I'm unhappy with the way I'm importing things.
I could simply import all the packages, but I don't want to do that in case the specific user doesn't ever need to use turbodbc for example:
import pyodbc
import pymssql
import turbodbc
from ibmdbpy.base import IdaDataBase
Currently, I have the following situation. I try to import all of them, but the ones that don't import, no problem, My program simply assumes they will not be called and if they are it errors:
# some envs may not have all these packages installed so we try each:
try:
import pyodbc
except:
pass
try:
import pymssql
except:
pass
try:
import turbodbc
except:
pass
try:
from ibmdbpy.base import IdaDataBase
except:
pass
This doesn't feel pythonic. So I know there are packages such as holoviews or tensorflow that allow you to specify a backend. They are of course orders of magnitude more complicated than mine, but they have to handle the same pattern.
How can I make this code right? it is technically buggy because if they intend to use pyodbc but don't have it installed, my program will not warn them, it will error at runtime. So really this goes beyond esthetics or philosophy; this is technically error-prone code.
How would you handle this situation?
Fyi, here is an example of how the code is called:
connect('Playground', package='pymssql')
try:
import pyodbc
except ImportError:
pyodbc = None
then later:
if pyodbc is None and user_wants_to_use_pyodbc:
print_warning()
raise SomeConfigurationErrorOrSuch()
This approach works well for a small number of options. If you have enough options that you need to abstract out this approach, then you can use the importlib module to import modules under the control of your program.
I would use import_module from importlib:
from importlib import import_module
modules_to_import = ['pyodbc', 'pymssql', 'turbodbc', 'ibmdbpy.base.IdaDataBase']
for m in modules_to_import:
try:
globals()[m.split('.')[-1]] = import_module(m)
except ModuleNotFoundError:
print('Module {} not found'.format(m))
I've used something similar to the answers above, but sometimes you might need to mock an object to fool lint.
try:
from neuralprophet import NeuralProphet
using_neuralprophet = True
except ImportError:
class NeuralMock:
whatever=False
using_neuralprophet = False
NeuralProphet = NeuralMock()
Source: timemachines
You can put imports in places other than the beginning of the file. "Re-importing" something doesn't actually do anything, so it's not computationally expensive to import x frequently:
def switch(x):
if x == 'a':
import json
json.load(file)
elif x == 'b':
import pandas as pd
pd.read_csv(file)
You can also use importlib to dynamically import modules. This is especially useful if you have multiple implementations of the same API that you want to choose between
class Connection:
def __init__(self, driver_module, driver_name):
# or driver_module, driver_name = full_path.rsplit('.', 1)
self.driver = get_attr(importlib.load_module(driver_module), driver_name)()
def method(self):
return self.driver.do()
I want to make some common code of mine working through python2.7 and also python3.6 versions.
In syntax manner it simply implies the following: converting prints to console of types: print "hello" to print("hello") which is acceptable in both versions.
The problem occurs only in one module import for Queue module.
In Python2.7: from Queue import Queue
In Python3.6: from queue import Queue
Trying to do something in the import section like:
try:
from Queue import Queue
except ImportError:
from queue import Queue
Will work but its really not elegant and ugly, any ideas for making it more reasonable?
That is not actualy so bad practice and can be seen in quite a lot of python modules. When it comes to support of both Python2 and Python3, six module can be quite handy.
With six you can import queue like that.
from six.moves import queue
It will automatically proxy your import to the appropriate place depending on Python version.
Maybe the below:
import platform
if platform.python_version().startswith('2.7'):
from Queue import Queue
elif platform.python_version().startswith('3.6'):
from queue import Queue
I was trying to import the following function in Python 2.7
import scipy.signal.savgol_filter as sgolay
I received the following error:
ImportError: No module named savgol_filter
savgol_filter is a function, not a module, so the error makes some sense. My question then is, is it not possible to import, without the use of the word "from" anything besides a module?
In other words, the following works:
from scipy.signal import savgol_filter as sgolay
But in general, does the following "sub_part" need to be a module?
import my_module.sub_part as some_name
I've seen lots of writing suggesting "sub_part" does not need to be a module. Is there something tricky going on with scipy that is making this not work?
Thanks,
Jim
In general, if you do import thing, import thing.subthing, import thing.subthing.subsubthing, etc., the far-right thing needs to be a module. Only the from form allows importing things that aren't modules. If you want a definitive statement of the forms of the import statement and what it allows, the Python language reference explains it in great detail, but it's a pretty dense read.
import is for importing modules. If you are not importing modules then you should see if its an object and create an instance like this:
import scipy as sp
sgolay = scipy.signal.savgol_filter
#Other stuff to do...
I have the following program, and nothing else, python 3.3. When I run it. I get
NameError: name 'threading' is not defined
I googled but none of the answers given explain my situation. any clues? Thanks!
#!/usr/bin/python
import Utilities
import os
import sys
import getopt
import time
from queue import Queue
from threading import Thread
_db_lock=threading.Lock()
I also tried
_db_lock=threading.Lock
You must import threading. Add the following to the beginning of your file:
import threading
The error originates from the line:
_db_lock=threading.Lock()
That's because you've used from threading import Thread, but you've never actually introduced threading in to the local namespace. So far there's only Thread (even though technically the import is there, it's just not in the namespace, you cannot use it).
If for some reason you wish to keep threading from 'polluting' your namespace, import the Lock in the same manner as you've imported Thread, like so:
from threading import Thread, Lock
_db_lock = Lock()
I am a little confused by the multitude of ways in which you can import modules in Python.
import X
import X as Y
from A import B
I have been reading up about scoping and namespaces, but I would like some practical advice on what is the best strategy, under which circumstances and why. Should imports happen at a module level or a method/function level? In the __init__.py or in the module code itself?
My question is not really answered by "Python packages - import by class, not file" although it is obviously related.
In production code in our company, we try to follow the following rules.
We place imports at the beginning of the file, right after the main file's docstring, e.g.:
"""
Registry related functionality.
"""
import wx
# ...
Now, if we import a class that is one of few in the imported module, we import the name directly, so that in the code we only have to use the last part, e.g.:
from RegistryController import RegistryController
from ui.windows.lists import ListCtrl, DynamicListCtrl
There are modules, however, that contain dozens of classes, e.g. list of all possible exceptions. Then we import the module itself and reference to it in the code:
from main.core import Exceptions
# ...
raise Exceptions.FileNotFound()
We use the import X as Y as rarely as possible, because it makes searching for usage of a particular module or class difficult. Sometimes, however, you have to use it if you wish to import two classes that have the same name, but exist in different modules, e.g.:
from Queue import Queue
from main.core.MessageQueue import Queue as MessageQueue
As a general rule, we don't do imports inside methods -- they simply make code slower and less readable. Some may find this a good way to easily resolve cyclic imports problem, but a better solution is code reorganization.
Let me just paste a part of conversation on django-dev mailing list started by Guido van Rossum:
[...]
For example, it's part of the Google Python style guides[1] that all
imports must import a module, not a class or function from that
module. There are way more classes and functions than there are
modules, so recalling where a particular thing comes from is much
easier if it is prefixed with a module name. Often multiple modules
happen to define things with the same name -- so a reader of the code
doesn't have to go back to the top of the file to see from which
module a given name is imported.
Source: http://groups.google.com/group/django-developers/browse_thread/thread/78975372cdfb7d1a
1: http://code.google.com/p/soc/wiki/PythonStyleGuide#Module_and_package_imports
I would normally use import X on module level. If you only need a single object from a module, use from X import Y.
Only use import X as Y in case you're otherwise confronted with a name clash.
I only use imports on function level to import stuff I need when the module is used as the main module, like:
def main():
import sys
if len(sys.argv) > 1:
pass
HTH
Someone above said that
from X import A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P
is equivalent to
import X
import X allows direct modifications to A-P, while from X import ... creates copies of A-P. For from X import A..P you do not get updates to variables if they are modified. If you modify them, you only modify your copy, but X does know about your modifications.
If A-P are functions, you won't know the difference.
Others have covered most of the ground here but I just wanted to add one case where I will use import X as Y (temporarily), when I'm trying out a new version of a class or module.
So if we were migrating to a new implementation of a module, but didn't want to cut the code base over all at one time, we might write a xyz_new module and do this in the source files that we had migrated:
import xyz_new as xyz
Then, once we cut over the entire code base, we'd just replace the xyz module with xyz_new and change all of the imports back to
import xyz
DON'T do this:
from X import *
unless you are absolutely sure that you will use each and every thing in that module. And even then, you should probably reconsider using a different approach.
Other than that, it's just a matter of style.
from X import Y
is good and saves you lots of typing. I tend to use that when I'm using something in it fairly frequently But if you're importing a lot from that module, you could end up with an import statement that looks like this:
from X import A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P
You get the idea. That's when imports like
import X
become useful. Either that or if I'm not really using anything in X very frequently.
I generally try to use the regular import modulename, unless the module name is long, or used often..
For example, I would do..
from BeautifulSoup import BeautifulStoneSoup as BSS
..so I can do soup = BSS(html) instead of BeautifulSoup.BeautifulStoneSoup(html)
Or..
from xmpp import XmppClientBase
..instead of importing the entire of xmpp when I only use the XmppClientBase
Using import x as y is handy if you want to import either very long method names , or to prevent clobbering an existing import/variable/class/method (something you should try to avoid completely, but it's not always possible)
Say I want to run a main() function from another script, but I already have a main() function..
from my_other_module import main as other_module_main
..wouldn't replace my main function with my_other_module's main
Oh, one thing - don't do from x import * - it makes your code very hard to understand, as you cannot easily see where a method came from (from x import *; from y import *; my_func() - where is my_func defined?)
In all cases, you could just do import modulename and then do modulename.subthing1.subthing2.method("test")...
The from x import y as z stuff is purely for convenience - use it whenever it'll make your code easier to read or write!
When you have a well-written library, which is sometimes case in python, you ought just import it and use it as it. Well-written library tends to take life and language of its own, resulting in pleasant-to-read -code, where you rarely reference the library. When a library is well-written, you ought not need renaming or anything else too often.
import gat
node = gat.Node()
child = node.children()
Sometimes it's not possible to write it this way, or then you want to lift down things from library you imported.
from gat import Node, SubNode
node = Node()
child = SubNode(node)
Sometimes you do this for lot of things, if your import string overflows 80 columns, It's good idea to do this:
from gat import (
Node, SubNode, TopNode, SuperNode, CoolNode,
PowerNode, UpNode
)
The best strategy is to keep all of these imports on the top of the file. Preferrably ordered alphabetically, import -statements first, then from import -statements.
Now I tell you why this is the best convention.
Python could perfectly have had an automatic import, which'd look from the main imports for the value when it can't be found from global namespace. But this is not a good idea. I explain shortly why. Aside it being more complicated to implement than simple import, programmers wouldn't be so much thinking about the depedencies and finding out from where you imported things ought be done some other way than just looking into imports.
Need to find out depedencies is one reason why people hate "from ... import *". Some bad examples where you need to do this exist though, for example opengl -wrappings.
So the import definitions are actually valuable as defining the depedencies of the program. It is the way how you should exploit them. From them you can quickly just check where some weird function is imported from.
The import X as Y is useful if you have different implementations of the same module/class.
With some nested try..import..except ImportError..imports you can hide the implementation from your code. See lxml etree import example:
try:
from lxml import etree
print("running with lxml.etree")
except ImportError:
try:
# Python 2.5
import xml.etree.cElementTree as etree
print("running with cElementTree on Python 2.5+")
except ImportError:
try:
# Python 2.5
import xml.etree.ElementTree as etree
print("running with ElementTree on Python 2.5+")
except ImportError:
try:
# normal cElementTree install
import cElementTree as etree
print("running with cElementTree")
except ImportError:
try:
# normal ElementTree install
import elementtree.ElementTree as etree
print("running with ElementTree")
except ImportError:
print("Failed to import ElementTree from any known place")
I'm with Jason in the fact of not using
from X import *
But in my case (i'm not an expert programmer, so my code does not meet the coding style too well) I usually do in my programs a file with all the constants like program version, authors, error messages and all that stuff, so the file are just definitions, then I make the import
from const import *
That saves me a lot of time. But it's the only file that has that import, and it's because all inside that file are just variable declarations.
Doing that kind of import in a file with classes and definitions might be useful, but when you have to read that code you spend lots of time locating functions and classes.