I'd like to look up the latest available version of a debian package programmatically using python. I've looked around, but can't find the right keywords to cut through all of the noise "python" "parse" "package" "index" happens to turn over.
Does anyone know of a way to load and parse such a package index?
Here is a URL to a sample, I can't quite parse it with yaml or json:
http://packages.osrfoundation.org/gazebo/ubuntu/dists/trusty/main/binary-amd64/
http://packages.osrfoundation.org/gazebo/ubuntu/dists/trusty/main/binary-amd64/Packages
I've looked at apt_pkg, but I'm not sure how to work it to what I need from the online index.
Thanks!
You can use the subprocess module to run apt-cache policy <app>:
from subprocess import check_output
out = check_output(["apt-cache", "policy","python"])
print(out)
Output:
python:
Installed: 2.7.5-5ubuntu3
Candidate: 2.7.5-5ubuntu3
Version table:
*** 2.7.5-5ubuntu3 0
500 http://ie.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages
100 /var/lib/dpkg/status
You can pass whatever app you are trying to get the info for using a fucntion:
from subprocess import check_output,CalledProcessError
def apt_cache(app):
try:
return check_output(["apt-cache", "policy",app])
except CalledProcessError as e:
return e.output
print(apt_cache("python"))
Or use *args and run whatever command you like:
from subprocess import check_output,CalledProcessError
def apt_cache(*args):
try:
return check_output(args)
except CalledProcessError as e:
return e.output
print(apt_cache("apt-cache","showpkg ","python"))
If you want to parse the output you can use re:
import re
from subprocess import check_output,CalledProcessError
def apt_cache(*args):
try:
out = check_output(args)
m = re.search("Candidate:.*",out)
return m.group() if m else "No match"
except CalledProcessError as e:
return e.output
print(apt_cache("apt-cache","policy","python"))
Candidate: 2.7.5-5ubuntu3
Or to get the installed and candidate:
def apt_cache(*args):
try:
out = check_output(args)
m = re.findall("Candidate:.*|Installed:.*",out)
return "{}\n{}".format(*m) if m else "No match"
except CalledProcessError as e:
return e.output
print(apt_cache("apt-cache","policy","python"))
Output:
Installed: 2.7.5-5ubuntu3
Candidate: 2.7.5-5ubuntu3
Not fully answering the question but a very elegant way to read the installed Debian package version
from pkg_resources import get_distribution
def get_distribution_version(service_name):
return get_distribution(service_name).version
Related
Without getting confused, there are tons of questions about installing packages, how to import the resulting modules, and listing what packages are available. But there doesn't seem to be the equivalent of a "--what-provides" option for pip, if you don't have a requirements.txt or pipenv. This question is similar to a previous question, but asks for the parent package, and not additional metadata. That said, these other questions did not get a lot of attention or many accepted answers - eg. How do you find python package metadata information given a module. So forging ahead... .
By way of example, there are two packages (to name a few) that will install a module called "serial" - namely "pyserial" and "serial". So assuming that one of the packages was installed, we might find it by using pip list:
python3 -m pip list | grep serial
However, the problem comes in if the name of the package does not match the name of the module, or if you just want to find out what package to install, working on a legacy server or dev machine.
You can check the path of the imported module - which can give you a clue. But continuing the example...
>>> import serial
>>> print(serial.__file__)
/usr/lib/python3.6/site-packages/serial/__init__.py
It is in a "serial" directory, but only pyserial is in fact installed, not serial:
> python3 -m pip list | grep serial
pyserial 3.4
The closest I can come is to generate a requirements.txt via "pipreqs ./" which may fail on a dependent child file (as it does with me), or to reverse check dependencies via pipenv (which brings a whole set of new issues along to get it all setup):
> pipenv graph --reverse
cymysql==0.9.15
ftptool==0.7.1
netifaces==0.10.9
pip==20.2.2
PyQt5-sip==12.8.1
- PyQt5==5.15.0 [requires: PyQt5-sip>=12.8,<13]
setuptools==50.3.0
wheel==0.35.1
Does anyone know of a command that I have missed for a simple solution to finding what pip package provides a particular module?
Use the packages_distributions() function from importlib.metadata (or importlib-metadata). So for example, in your case where serial is the name of the "import package":
import importlib.metadata # or: `import importlib_metadata`
importlib.metadata.packages_distributions()['serial']
This should return a list containing pyserial, which is the name of the "distribution package" (the name that should be used to pip-install).
References
https://importlib-metadata.readthedocs.io/en/stable/using.html#package-distributions
https://github.com/python/importlib_metadata/pull/287/files
For older Python versions and/or older versions of importlib-metadata...
I believe something like the following should work:
#!/usr/bin/env python3
import importlib.util
import pathlib
import importlib_metadata
def get_distribution(file_name):
result = None
for distribution in importlib_metadata.distributions():
try:
relative = (
pathlib.Path(file_name)
.relative_to(distribution.locate_file(''))
)
except ValueError:
pass
else:
if distribution.files and relative in distribution.files:
result = distribution
break
return result
def alpha():
file_name = importlib.util.find_spec('serial').origin
distribution = get_distribution(file_name)
print("alpha", distribution.metadata['Name'])
def bravo():
import serial
file_name = serial.__file__
distribution = get_distribution(file_name)
print("bravo", distribution.metadata['Name'])
if __name__ == '__main__':
alpha()
bravo()
This is just an example of code showing how to get the metadata of the installed project a specific module belongs to.
The important bit is the get_distribution function, it takes a file name as an argument. It could be the file name of a module or package data. If that file name belongs to a project installed in the environment (via pip install for example) then the importlib.metadata.Distribution object is returned.
Edit 2023/01/31: This issue is now solved via the importlib_metadata library. See Provide mapping from "Python packages" to "distribution packages", specifically "Note 2" deals with this exact issue. As such, see comments by #sinoroc, you can locate the package (eg. package "pyserial" providing module "serial") with something like this:
>>> import importlib_metadata
>>> print(importlib_metadata.packages_distributions()['serial'])
['pyserial']
Building on #sinoroc's much-published answer, I came up with the following code (incorporating the mentioned importlib.util.find_spec method, but with a bash-based search against the RECORD file in the path returned). I also tried to implement #sinoroc's version - but was not successful. Both methods are included to demonstrate.
Run as "python3 python_find-module-package.py -m [module-name-here] -d", which will also print debug. Leave off the "-d" switch to get just the package name returned (and errors).
TLDR: Code on github.
#!/usr/bin/python3
import sys
import os.path
import importlib.util
import importlib_metadata
import pathlib
import subprocess
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("-m", "--module", help="Find matching package for the specified Python module",
type=str)
#parser.add_argument("-u", "--username", help="Database username",
# type=str)
#parser.add_argument("-p", "--password", help="Database password",
# type=str)
parser.add_argument("-d", "--debug", help="Debug messages are enabled",
action="store_true")
args = parser.parse_args()
TESTMODULE='serial'
def debugPrint (message="Nothing"):
if args.debug:
print ("[DEBUG] %s" % str(message))
class application ():
def __init__(self, argsPassed):
self.argsPassed = argsPassed
debugPrint("Got these arguments:\n%s" % (argsPassed))
def run (self):
#debugPrint("Running with args:\n%s" % (self.argsPassed))
try:
if self.argsPassed.module is not None:
self.moduleName=self.argsPassed.module #i.e. the module that you're trying to match with a package.
else:
self.moduleName=TESTMODULE
print("[WARN] No module name supplied - defaulting to %s!" % (TESTMODULE))
self.location=importlib.util.find_spec(self.moduleName).origin
debugPrint(self.location)
except:
print("[ERROR] Parsing module name!")
exit(1)
try:
self.getPackage()
except Exception as e:
print ("[ERROR] getPackage failed: %s" % str(e))
try:
distResult=self.getDistribution(self.location)
self.packageStrDist=distResult.metadata['Name']
print(self.packageStrDist)
except Exception as e:
print ("[ERROR] getDistribution failed: %s" % str(e))
debugPrint("Parent package for \"%s\" is: \"%s\"" % (self.moduleName, self.packageStr))
return self.packageStr
def getPackage (self):
locationStr=self.location.split("site-packages/",1)[1]
debugPrint(locationStr)
#serial/__init__.py
locationDir=self.location.split(locationStr,1)[0]
debugPrint(locationDir)
#/usr/lib/python3.6/site-packages
cmd='find \"' + locationDir + '\" -type f -iname \'RECORD\' -printf \'\"%p\"\\n\' | xargs grep \"' + locationStr + '\" -l -Z'
debugPrint(cmd)
#find "/usr/lib/python3.6/site-packages" -type f -iname 'RECORD' -printf '"%p"\n' | xargs grep "serial/__init__.py" -l -Z
#return_code = os.system(cmd)
#return_code = subprocess.run([cmd], stdout=subprocess.PIPE, universal_newlines=True, shell=False)
#findResultAll = return_code.stdout
findResultAll = subprocess.check_output(cmd, shell=True) # Returns stdout as byte array, null terminated.
findResult = str(findResultAll.decode('ascii').strip().strip('\x00'))
debugPrint(findResult)
#/usr/lib/python3.6/site-packages/pyserial-3.4.dist-info/RECORD
findDir = os.path.split(findResult)
self.packageStr=findDir[0].replace(locationDir,"")
debugPrint(self.packageStr)
def getDistribution(self, fileName=TESTMODULE):
result = None
for distribution in importlib_metadata.distributions():
try:
relative = (pathlib.Path(fileName).relative_to(distribution.locate_file('')))
#except ValueError:
#except AttributeError:
except:
pass
else:
if relative in distribution.files:
result = distribution
return result
if __name__ == '__main__':
result=1
try:
prog = application(args)
result = prog.run()
except Exception as E:
print ("[ERROR] Prog Exception: %s" % str(E))
finally:
sys.exit(result)
# exit the program if we haven't already
print ("Shouldn't get here.")
sys.exit(result)
I got this code from Stack overflow but when I try to execute the command I get an error. The code and the error is given below. I would be grateful if someone could help:
import sys
from _winreg import *
# tweak as necessary
version = sys.version[:3]
installpath = sys.prefix
regpath = "SOFTWARE\\Python\\Pythoncore\\%s\\" % (version)
installkey = "InstallPath"
pythonkey = "PythonPath"
pythonpath = "%s;%s\\Lib\\;%s\\DLLs\\" % (
installpath, installpath, installpath
)
def RegisterPy():
try:
reg = OpenKey(HKEY_CURRENT_USER, regpath)
except EnvironmentError as e:
try:
reg = CreateKey(HKEY_CURRENT_USER, regpath)
SetValue(reg, installkey, REG_SZ, installpath)
SetValue(reg, pythonkey, REG_SZ, pythonpath)
CloseKey(reg)
except:
print ("*** Unable to register!")
return
print ("--- Python", version, "is now registered!")
return
if (QueryValue(reg, installkey) == installpath and
QueryValue(reg, pythonkey) == pythonpath):
CloseKey(reg)
print ("=== Python", version, "is already registered!")
return
CloseKey(reg)
print ("*** Unable to register!")
print ("*** You probably have another Python installation!")
if __name__ == "__main__":
RegisterPy()
I get the following error:
from __winreg import *
Traceback (most recent call last):
File "<ipython-input-35-f6f8c1a0ffdd>", line 1, in <module>
from __winreg import *
ModuleNotFoundError: No module named '__winreg'
As the Python 2 documentation for _winreg states,
The _winreg module has been renamed to winreg in Python 3. The 2to3 tool will automatically adapt imports when converting your sources to Python 3.
Check your version of Python. If it is Python 3, as it seems, rename the module references to winreg (without the underscore). Look at the Python 3 documentation.
I am writing python code to install all the library packages required by my program in the linux environment.So the linux may contain python 2.7 or 2.6 or both so I have developed a try and except block codes that will install pip packages in linux. Try block code consists of python 2.7 version pip install and Catch block contains python 2.6 version pip install. My Problem is the peace of code is working fine, when i tried to install pandas in python 2.6 its getting me some errror. I want to catch that exception. Can you please tell me how to improve my try except blocks to catch that exception
required_libraries = ['pytz','requests','pandas']
try:
from subprocess import check_output
pip27_path = subprocess.check_output(['sudo','find','/','-name','pip2.7'])
lib_installs = [subprocess.call((['sudo',pip27_path.replace('\n',''),'install', i])) for i in required_libraries]
except:
p = subprocess.Popen(['sudo','find','/','-name','pip2.6'], stdout=subprocess.PIPE);pip26_path, err = p.communicate()
lib_installs = [subprocess.call((['sudo',pip26_path.replace('\n',''),'install', i])) for i in required_libraries]
You can catch several exceptions using one block. Let's use Exception and ArithmeticError for exceptions.
try:
# Do something
print(q)
# Catch exceptions
except (Exception, ArithmeticError) as e:
template = "An exception of type {0} occurred. Arguments:\n{1!r}"
message = template.format(type(e).__name__, e.args)
print (message)
If you need to catch several exceptions and handle each one on its own then you'd write an except statement for each one.
try:
# Do something
print(q)
# Catch exceptions
except Exception as e:
print (1)
except ArithmeticError as e:
print (2)
# Code to be executed if the try clause succeeded with no errors or no return/continue/break statement
else:
print (3)
You can also check if the exception is of type "MyCustomException" for example using if statements.
if isinstance(e, MyCustomException):
# Do something
print(1)
As for your problem, I suggest splitting the code into two functions.
install(required_libraries)
def install(required_libraries, version='pip2.7'):
# Perform installation
try:
from subprocess import check_output
pip27_path = subprocess.check_output(['sudo','find','/','-name', version])
lib_installs = [subprocess.call((['sudo',pip27_path.replace('\n',''),'install', i])) for i in required_libraries]
except Exception as e:
backup(required_libraries)
def backup(required_libraries, version='pip2.6'):
try:
p = subprocess.Popen(['sudo','find','/','-name',version]], stdout=subprocess.PIPE);pip26_path, err = p.communicate()
lib_installs = [subprocess.call((['sudo',pip26_path.replace('\n',''),'install', i])) for i in required_libraries]
except Exception as e:
template = "An exception of type {0} occurred. Arguments:\n{1!r}"
message = template.format(type(e).__name__, e.args)
print (message)
#Handle exception
Note: I didn't test this, I'm no expert as well so I hope I can help.
Useful links:
Built-in Exceptions
Errors and Exceptions
Compound statements
Could any one help me to check in Linux to meet rpm version
Below command I used to get package version.
rpm -qi libstdc++6 | awk -F': ' '/Version/ {print $2}'
using sub-process, I get below out put
6.2.1+r239768
Now I want to compare 6.2.1+r239768 is greater than 5.1.1
Below is results I am expected
output = 6.2.1+r239768
print "This rpm version is supported"
output = 6.3+r23
print "This rpm version is supported"
output = 7.1.1+r57678
print "This rpm version is supported"
output = 5.1.1+r23677
print "This rpm version is not supported"
output = 4.1+r56888
print "This rpm version is not supported"
You can use pkg_resources module which you should already have as it is part of setuptools:
In []:
from pkg_resources import parse_version
parse_version('6.2.1+r239768') > parse_version('5.1.1')
Out[]:
True
So to get your output:
In []:
versions = ['6.2.1+r239768', '6.3+r23', '7.1.1+r57678', '5.1.1+r23677', '4.1+r56888']
base = parse_version('6.2.1+r239768')
for v in versions:
print("Output = {}".format(v))
print("This rpm version is {}supported".format('not ' if parse_version(v) < base else ''))
Out[]:
Output = 6.2.1+r239768
This rpm version is supported
Output = 6.3+r23
This rpm version is supported
Output = 7.1.1+r57678
This rpm version is supported
Output = 5.1.1+r23677
This rpm version is not supported
Output = 4.1+r56888
This rpm version is not supported
I feels like you should be better of using the python API for rpm.
As I did not have RPM based system on hand right now, could not come up with code.
Following code, taken from Section 16.5, can be adapted accordingly
#!/usr/bin/python
# Reads in package header, compares to installed package.
# Usage:
# python vercompare.py libstdc++6.rpm
#
import os
import sys
import rpm
def readRpmHeader(ts, filename):
""" Read an rpm header. """
fd = os.open(filename, os.O_RDONLY)
try:
h = ts.hdrFromFdno(fd)
finally:
os.close(fd)
return h
ts = rpm.TransactionSet()
h = readRpmHeader( ts, sys.argv[1] )
pkg_ds = h.dsOfHeader()
for inst_h in ts.dbMatch('name', h['name']):
inst_ds = inst_h.dsOfHeader()
if pkg_ds.EVR() >= inst_ds.EVR():
print "Package file is same or newer, OK to upgrade."
else:
print "Package file is older than installed version."
$ rpmdev-vercmp 6.2.1+r239768 6.3+r23
6.2.1+r239768 < 6.3+r23
You can read the exit code:
rpmdev-vercmp <EVR1> <EVR2>
Exit status is 0 if the EVR's are equal, 11 if EVR1 is newer, and 12 if EVR2
is newer. Other exit statuses indicate problems.
The python rpm library gives you a way to compare a EVR (Epoch, Version, Release), but doesn't seem to give you the code to split a string into those parts. It's trivial, though. Here's a little working program.
#! /usr/bin/env python
import re
import sys
import rpm
evr_re = re.compile(r'^(?:([^:]*):)?([^-]+)(?:-(.*))?$')
def str_to_evr(s):
mo = evr_re.match(s)
if not mo:
raise ValueError("not a valid version string")
return mo.groups()
def main():
if len(sys.argv) != 3:
raise SystemExit("usage: vercmp VER1 VER2")
try:
evr_a = str_to_evr(sys.argv[1].strip())
evr_b = str_to_evr(sys.argv[2].strip())
result = rpm.labelCompare(evr_a, evr_b)
except Exception as exc:
raise SystemExit("Problem converting or comparing versions: %s %s" % (
type(exc).__name__, exc))
print (result)
if __name__ == "__main__":
main()
How can I know if a Python module exists, without importing it?
Importing something that might not exist (not what I want) results in:
try:
import eggs
except ImportError:
pass
TL;DR) Use importlib.util.find_spec(module_name) (Python 3.4+).
Python2: imp.find_module
To check if import can find something in Python 2, using imp:
import imp
try:
imp.find_module('eggs')
found = True
except ImportError:
found = False
To find dotted imports, you need to do more:
import imp
try:
spam_info = imp.find_module('spam')
spam = imp.load_module('spam', *spam_info)
imp.find_module('eggs', spam.__path__) # __path__ is already a list
found = True
except ImportError:
found = False
You can also use pkgutil.find_loader (more or less the same as the Python 3 part:
import pkgutil
eggs_loader = pkgutil.find_loader('eggs')
found = eggs_loader is not None
Python 3
Python 3 ≤ 3.3: importlib.find_loader
You should use importlib. I went about doing this like:
import importlib
spam_loader = importlib.find_loader('spam')
found = spam_loader is not None
My expectation being, if you can find a loader for it, then it exists. You can also be a bit more smart about it, like filtering out what loaders you will accept. For example:
import importlib
spam_loader = importlib.find_loader('spam')
# only accept it as valid if there is a source file for the module - no bytecode only.
found = issubclass(type(spam_loader), importlib.machinery.SourceFileLoader)
Python 3 ≥ 3.4: importlib.util.find_spec
In Python 3.4 importlib.find_loader Python documentation was deprecated in favour of importlib.util.find_spec. The recommended method is the importlib.util.find_spec. There are others like importlib.machinery.FileFinder, which is useful if you're after a specific file to load. Figuring out how to use them is beyond the scope of this.
import importlib
spam_spec = importlib.util.find_spec("spam")
found = spam_spec is not None
This also works with relative imports, but you must supply the starting package, so you could also do:
import importlib
spam_spec = importlib.util.find_spec("..spam", package="eggs.bar")
found = spam_spec is not None
spam_spec.name == "eggs.spam"
While I'm sure there exists a reason for doing this - I'm not sure what it would be.
Warning
When trying to find a submodule, it will import the parent module (for ALL of the above methods)!
food/
|- __init__.py
|- eggs.py
## __init__.py
print("module food loaded")
## eggs.py
print("module eggs")
were you then to run
>>> import importlib
>>> spam_spec = importlib.util.find_spec("food.eggs")
module food loaded
ModuleSpec(name='food.eggs', loader=<_frozen_importlib.SourceFileLoader object at 0x10221df28>, origin='/home/user/food/eggs.py')
Comments are welcome on getting around this
Acknowledgements
#rvighne for importlib
#lucas-guido for Python 3.3+ deprecating find_loader
#enpenax for pkgutils.find_loader behaviour in Python 2.7
Python 3 >= 3.6: ModuleNotFoundError
The ModuleNotFoundError has been introduced in Python 3.6 and can be used for this purpose:
try:
import eggs
except ModuleNotFoundError:
# Error handling
pass
The error is raised when a module or one of its parents cannot be found. So
try:
import eggs.sub
except ModuleNotFoundError as err:
# Error handling
print(err)
would print a message that looks like No module named 'eggs' if the eggs module cannot be found; but it would print something like No module named 'eggs.sub' if only the sub module couldn't be found, but the eggs package could be found.
See the documentation of the import system for more information on the ModuleNotFoundError.
After using yarbelk's response, I've made this so I don't have to import ìmp.
try:
__import__('imp').find_module('eggs')
# Make things with a supposed existing module
except ImportError:
pass
It is useful in Django's settings.py file, for example.
Python 2, without relying on ImportError
Until the current answer is updated, here is the way for Python 2
import pkgutil
import importlib
if pkgutil.find_loader(mod) is not None:
return importlib.import_module(mod)
return None
Why another answer?
A lot of answers make use of catching an ImportError. The problem with that is, that we cannot know what throws the ImportError.
If you import your existent module and there happens to be an ImportError in your module (e.g., typo on line 1), the result will be that your module does not exist.
It will take you quite the amount of backtracking to figure out that your module exists and the ImportError is caught and makes things fail silently.
go_as's answer as a one-liner:
python -c "help('modules');" | grep module
Use one of the functions from pkgutil, for example:
from pkgutil import iter_modules
def module_exists(module_name):
return module_name in (name for loader, name, ispkg in iter_modules())
I wrote this helper function:
def is_module_available(module_name):
if sys.version_info < (3, 0):
# python 2
import importlib
torch_loader = importlib.find_loader(module_name)
elif sys.version_info <= (3, 3):
# python 3.0 to 3.3
import pkgutil
torch_loader = pkgutil.find_loader(module_name)
elif sys.version_info >= (3, 4):
# python 3.4 and above
import importlib
torch_loader = importlib.util.find_spec(module_name)
return torch_loader is not None
Here is a way to check if a module is loaded from the command line:
Linux/UNIX script file method: make a file module_help.py:
#!/usr/bin/env python
help('modules')
Then make sure it's executable: chmod u+x module_help.py
And call it with a pipe to grep:
./module_help.py | grep module_name
Invoke the built-in help system. (This function is intended for interactive use.) If no argument is given, the interactive help system starts on the interpreter console. If the argument is a string, then the string is looked up as the name of a module, function, class, method, keyword, or documentation topic, and a help page is printed on the console. If the argument is any other kind of object, a help page on the object is generated.
Interactive method: in the console, load python
>>> help('module_name')
If found, quit reading by typing q.
To exit the Python interpreter interactive session, press Ctrl + D
Windows script file method, also Linux/UNIX compatible, and better overall:
#!/usr/bin/env python
import sys
help(sys.argv[1])
Calling it from the command like:
python module_help.py site
Would output:
Help on module site:
NAME
site - Append module search paths for third-party packages to sys.path.
FILE
/usr/lib/python2.7/site.py
MODULE DOCS
http://docs.python.org/library/site
DESCRIPTION
...
:
And you'd have to press q to exit interactive mode.
Using it for an unknown module, e.g.,
python module_help.py lkajshdflkahsodf
Would output:
no Python documentation found for 'lkajshdflkahsodf'
and exit.
You could just write a little script that would try to import all the modules and tell you which ones are failing and which ones are working:
import pip
if __name__ == '__main__':
for package in pip.get_installed_distributions():
pack_string = str(package).split(" ")[0]
try:
if __import__(pack_string.lower()):
print(pack_string + " loaded successfully")
except Exception as e:
print(pack_string + " failed with error code: {}".format(e))
Output:
zope.interface loaded successfully
zope.deprecation loaded successfully
yarg loaded successfully
xlrd loaded successfully
WMI loaded successfully
Werkzeug loaded successfully
WebOb loaded successfully
virtualenv loaded successfully
...
A word of warning: this will try to import everything, so you'll see things like PyYAML failed with error code: No module named pyyaml, because the actual import name is just yaml. So as long as you know your imports, this should do the trick for you.
There isn't any way to reliably check if "dotted module" is importable without importing its parent package. Saying this, there are many solutions to problem "how to check if a Python module exists".
The below solution addresses the problem that an imported module can raise an ImportError even if it exists. We want to distinguish that situation from such in which the module does not exist.
Python 2:
import importlib
import pkgutil
import sys
def find_module(full_module_name):
"""
Returns module object if module `full_module_name` can be imported.
Returns None if module does not exist.
Exception is raised if (existing) module raises exception during its import.
"""
module = sys.modules.get(full_module_name)
if module is None:
module_path_tail = full_module_name.split('.')
module_path_head = []
loader = True
while module_path_tail and loader:
module_path_head.append(module_path_tail.pop(0))
module_name = ".".join(module_path_head)
loader = bool(pkgutil.find_loader(module_name))
if not loader:
# Double check if module realy does not exist
# (case: full_module_name == 'paste.deploy')
try:
importlib.import_module(module_name)
except ImportError:
pass
else:
loader = True
if loader:
module = importlib.import_module(full_module_name)
return module
Python 3:
import importlib
def find_module(full_module_name):
"""
Returns module object if module `full_module_name` can be imported.
Returns None if module does not exist.
Exception is raised if (existing) module raises exception during its import.
"""
try:
return importlib.import_module(full_module_name)
except ImportError as exc:
if not (full_module_name + '.').startswith(exc.name + '.'):
raise
In django.utils.module_loading.module_has_submodule:
import sys
import os
import imp
def module_has_submodule(package, module_name):
"""
check module in package
django.utils.module_loading.module_has_submodule
"""
name = ".".join([package.__name__, module_name])
try:
# None indicates a cached miss; see mark_miss() in Python/import.c.
return sys.modules[name] is not None
except KeyError:
pass
try:
package_path = package.__path__ # No __path__, then not a package.
except AttributeError:
# Since the remainder of this function assumes that we're dealing with
# a package (module with a __path__), so if it's not, then bail here.
return False
for finder in sys.meta_path:
if finder.find_module(name, package_path):
return True
for entry in package_path:
try:
# Try the cached finder.
finder = sys.path_importer_cache[entry]
if finder is None:
# Implicit import machinery should be used.
try:
file_, _, _ = imp.find_module(module_name, [entry])
if file_:
file_.close()
return True
except ImportError:
continue
# Else see if the finder knows of a loader.
elif finder.find_module(name):
return True
else:
continue
except KeyError:
# No cached finder, so try and make one.
for hook in sys.path_hooks:
try:
finder = hook(entry)
# XXX Could cache in sys.path_importer_cache
if finder.find_module(name):
return True
else:
# Once a finder is found, stop the search.
break
except ImportError:
# Continue the search for a finder.
continue
else:
# No finder found.
# Try the implicit import machinery if searching a directory.
if os.path.isdir(entry):
try:
file_, _, _ = imp.find_module(module_name, [entry])
if file_:
file_.close()
return True
except ImportError:
pass
# XXX Could insert None or NullImporter
else:
# Exhausted the search, so the module cannot be found.
return False
In case you know the location of file and want to check that the respective Python code file has that module or not, you can simply check via the astor package in Python. Here is a quick example:
"""
Check if a module function exists or not without importing a Python package file
"""
import ast
import astor
tree = astor.parse_file('handler.py')
method_to_check = 'handle'
for item in tree.body:
if isinstance(item, ast.FunctionDef):
if item.name == method_to_check:
print('method exists')
break
A simpler if statement from Ask Ubuntu, How do I check whether a module is installed in Python?:
import sys
print('eggs' in sys.modules)
You can also use importlib.util directly
import importlib.util
def module_exists_without_import(module_name):
spec = importlib.util.find_spec(module_name)
return spec is not None