I'm trying to use the importlib library to verify whether the nmap library is installed on the computer executing the script in Python 3.5.2
I'm trying to use importlib.util.find_spec("nmap") but receive the following error.
>>> import importlib
>>> importlib.util.find_spec("nmap")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'importlib' has no attribute 'util'
Can someone tell me where I'm going wrong?
EDIT
I was able to get the function to work using the following code.
#!/usr/bin/pythonw
import importlib
from importlib import util
#check to see if nmap module is installed
find_nmap = util.find_spec("nmap")
if find_nmap is None:
print("Error")
Try this:
from importlib import util
util.find_spec("nmap")
I intend to investigate, but honestly I don't know why one works and the other doesn't. Also, observe the following interactive session:
>>> import importlib
>>> importlib.util
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'importlib' has no attribute 'util'
>>> from importlib import util
>>> util
<module 'importlib.util' from '/usr/lib/python3.5/importlib/util.py'>
>>> importlib.util
<module 'importlib.util' from '/usr/lib/python3.5/importlib/util.py'>
So...yeah. I am sure this makes perfect sense to someone, but not to me. I will update once I figure it out.
Update:
Comparing this to something like:
>>> import datetime
>>> datetime
<module 'datetime' from '/usr/lib/python3.5/datetime.py'>
>>> datetime.datetime
<class 'datetime.datetime'>
I think the difference is that in this case the first datetime is a module and the second is a class, while in the importlib.util case both are modules. So perhaps module.module is not OK unless the code from both modules has been loaded, while module.class is OK, because the class code is loaded when the module is imported.
Update #2
Nope, it seems like in many cases module.module is fine. For example:
>>> import urllib
>>> urllib
<module 'urllib' from '/usr/lib/python3.5/urllib/__init__.py'>
>>> urllib.error
<module 'urllib.error' from '/usr/lib/python3.5/urllib/error.py'>
So perhaps it is something specific to importlib.
Update #3
As #kfb pointed out in the comments, it does seem to be related to importlib specifically. See the following comment from the __init__.py for importlib:
# Until bootstrapping is complete, DO NOT import any modules that attempt
# to import importlib._bootstrap (directly or indirectly). Since this
# partially initialised package would be present in sys.modules, those
# modules would get an uninitialised copy of the source version, instead
# of a fully initialised version (either the frozen one or the one
# initialised below if the frozen one is not available).
importlib/util.py does import importlib._bootstrap so I would assume that this is realted. If my understanding is correct, when you do import importlib the submodules will be initialized, but are not initialized for the importlib module object that you have imported. At this point, if you do dir(importlib) you will not see util. Interestingly, after you have tried to access importlib.util and gotten an AttributeError, util (along with the other submodules) gets loaded/initialized, and now you can access importlib.util!
>>> import importlib
>>> dir(importlib)
['_RELOADING', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__import__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '_bootstrap', '_bootstrap_external', '_imp', '_r_long', '_w_long', 'find_loader', 'import_module', 'invalidate_caches', 'reload', 'sys', 'types', 'warnings']
>>> importlib.util
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'importlib' has no attribute 'util'
>>> importlib.util
<module 'importlib.util' from '/usr/lib/python3.5/importlib/util.py'>
>>> dir(importlib)
['_RELOADING', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__import__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '_bootstrap', '_bootstrap_external', '_imp', '_r_long', '_w_long', 'abc', 'find_loader', 'import_module', 'invalidate_caches', 'machinery', 'reload', 'sys', 'types', 'util', 'warnings']
Related
i want to implement sort of plugin architecture that dynamically loads modules and calls a function from them
for instance plugin code looks like (in file "foo_func.py")
foo_local = []
def foo_add(a: int, b: int) -> int:
c = a + b
foo_local.append(c)
return c
def foo_print():
print(foo_local)
i need to support two plugins with the same code but with different memory state, so i created directory structure like this:
<ROOT_PROJECT>
app.py
bar/
apple/
foo/
foo_func.py
__init__.py
orange/
foo/
foo_func.py
__init__.py
code in "apple" and "orange" folders is the same.
then in app file i try to load modules and invoke functions from them
import importlib
from bar.apple.foo.foo_func import foo_add as apple_foo_add, foo_print as apple_foo_print
from bar.orange.foo.foo_func import foo_add as orange_foo_add, foo_print as orange_foo_print
apple = importlib.import_module('bar.apple.foo')
orange = importlib.import_module('bar.orange.foo')
apple_foo = getattr(apple, 'foo_func')
orange_foo = getattr(orange, 'foo_func')
apple_foo_add_my = getattr(apple_foo, 'foo_add')
apple_foo_print_my = getattr(apple_foo, 'foo_print')
apple_foo_add_my(1, 2)
apple_foo_print_my()
and this works fine, but you see these import lines at the top
from bar.apple.foo.foo_func import foo_add as apple_foo_add, foo_print as apple_foo_print
from bar.orange.foo.foo_func import foo_add as orange_foo_add, foo_print as orange_foo_print
they are not used in code (even pycharm complains about it)
but if i try to comment code and run it - then failure
AttributeError: module 'bar.apple.foo' has no attribute 'foo_func'
why ?
I suppose normal plugins should deal only with "importlib.import_module" and "getattr" and it must be enough ?
what is wrong here ?
Let's switch completely to direct imports for this explanation, because:
import something.whatever as name
is the same as:
name = importlib.import_module("something.whatever")
So let's rewrite your apple code:
apple = importlib.import_module('bar.apple.foo')
apple_foo = getattr(apple, 'foo_func')
will become:
import bar.apple.foo as apple
apple_foo = apple.foo_func
Now, the first line loads bar.apple.foo as a module. In case of packages, this means importing package's __init__.py code. And treating it as a module itself.
And what's the code in the package's init? Usually nothing! That's why the name lookup fails.
However, when you do any import my_package.whatever, the package gets its insides checked and the name becomes visible. You're basically pre-loading the module for interpreter to look at.
Why is pycharm giving you not used suggestion? Because it's not used as a variable anywhere. You're only using a side-effect + pycharm doesn't analyze strings for imports or attributes.
Visual example, with a part of standard library:
>>> import xml
>>> dir(xml)
['__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__']
>>> xml.etree
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'xml' has no attribute 'etree'
>>>
>>> import xml.etree
>>> dir(xml)
['__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'etree']
And another example, what happens if there are multiple modules in the package:
>>> import dateutil
>>> dir(dateutil)
['__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__version__', '_version']
but:
>>> import dateutil.parser
>>> dir(dateutil)
['__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__version__', '_common', '_version', 'parser', 'relativedelta', 'tz']
All sub-modules are now visible and usable with their qualified name.
tl;dr: import my_package == only my_package/__init__.py is looked at. import my_package.whatever == python now knows it's a package and registers its insides, all modules of my_package are visible and usable.
I'm currently trying to import Python files downloaded from external sources into my code. This has worked perfectly fine within the Python shell, but when using the exact same code within an IDLE file, Python is unable to find the file in the given directory, even though it does find the directory itself.
My aim is to import the Python Natural Language Toolkit, which requires the module 'six'. So I've imported first 'six', then NLTK into a shell.
I've then tried to repeat the same code within script mode.
INTERACTIVE MODE
>>> import os
>>> path = "C:/Users/henri/AppData/Local/Programs/Python/Python37-32/lib/site-packages/pip/_vendor/urllib3/packages"
>>> os.chdir(path)
>>> os.getcwd()
Result:
'C:\\Users\\henri\\AppData\\Local\\Programs\\Python\\Python37-32\\lib\\site-packages\\pip\\_vendor\\urllib3\\packages' <br/>
>>> import six
>>> dir(six)
Result:
['BytesIO', 'Iterator', 'MAXSIZE', 'Module_six_moves_urllib', 'Module_six_moves_urllib_error', 'Module_six_moves_urllib_parse', 'Module_six_moves_urllib_request', 'Module_six_moves_urllib_response', 'Module_six_moves_urllib_robotparser', 'MovedAttribute', 'MovedModule', 'PY2', 'PY3', 'PY34', 'StringIO', '_LazyDescr', '_LazyModule', '_MovedItems', '_SixMetaPathImporter', '__author__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__version__', '_add_doc', '_assertCountEqual', '_assertRaisesRegex', '_assertRegex', '_func_closure', '_func_code', '_func_defaults', '_func_globals', '_import_module', '_importer', '_meth_func', '_meth_self', '_moved_attributes', '_urllib_error_moved_attributes', '_urllib_parse_moved_attributes', '_urllib_request_moved_attributes', '_urllib_response_moved_attributes', '_urllib_robotparser_moved_attributes', 'absolute_import', 'add_metaclass', 'add_move', 'advance_iterator', 'assertCountEqual', 'assertRaisesRegex', 'assertRegex', 'b', 'binary_type', 'byte2int', 'callable', 'class_types', 'create_bound_method', 'create_unbound_method', 'exec_', 'functools', 'get_function_closure', 'get_function_code', 'get_function_defaults', 'get_function_globals', 'get_method_function', 'get_method_self', 'get_unbound_function', 'indexbytes', 'int2byte', 'integer_types', 'io', 'iterbytes', 'iteritems', 'iterkeys', 'iterlists', 'itertools', 'itervalues', 'moves', 'next', 'operator', 'print_', 'python_2_unicode_compatible', 'raise_from', 'remove_move', 'reraise', 'string_types', 'sys', 'text_type', 'types', 'u', 'unichr', 'viewitems', 'viewkeys', 'viewvalues', 'with_metaclass', 'wraps']
SCRIPT MODE
import os
# importing six
path="C:/Users/henri/AppData/Local/Programs/Python/Python37-32/lib/site-packages/pip/_vendor/urllib3/packages"
os.chdir(path)
os.getcwd()
print(os.getcwd())
import six
dir(six)
And this is the resulting error message when trying to run this code:
RESTART: C:\Users\henri\AppData\Local\Programs\Python\Python37-32\importing_nltk_file_2.py
C:\Users\henri\AppData\Local\Programs\Python\Python37-32\lib\site-packages\pip_vendor\urllib3\packages
Traceback (most recent call last):
File "C:\Users\henri\AppData\Local\Programs\Python\Python37-32\importing_nltk_file_2.py", line 13, in
import six
ModuleNotFoundError: No module named 'six'
So, does anyone have an idea why Python can't find 'six', although the directory is unmistakeably correct?
when you run a python script all import statements are run first and then other parts of your code are executed. So, import six is executed before os.chdir() and hence, the error
What is so special about ldap.modlist that I have to explicitly import it? Importing ldap is not enough:
>>> import ldap
>>> ldap.modlist
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'modlist'
But if I explicitly import it, then all is well:
>>> import ldap
>>> import ldap.modlist
>>> ldap.modlist
<module 'ldap.modlist' from '/opt/rh/python27/root/usr/lib64/python2.7/site-packages/ldap/modlist.pyc'>
Why? Other modules don't behave like this:
>>> import os
>>> os.path
<module 'posixpath' from '/opt/rh/python27/root/usr/lib64/python2.7/posixpath.pyc'>
>>> import ldap
>>> ldap.dn
<module 'ldap.dn' from '/opt/rh/python27/root/usr/lib64/python2.7/site-packages/ldap/dn.pyc'>
The difference between the modules ldap.dn and ldap.modlist is this:
The module ldap.dn is loaded during the initialization of the ldap package and is therefore already present as an attribute in the ldap package (see The Import System: Submodules for a description about submodules):
# ldap/__init__.py, line 89 of python-ldap version 2.4.27
from ldap.dn import explode_dn,explode_rdn,str2dn,dn2str
The module ldap.modlist on the other hand is not loaded during the initalization of the ldap package.
When I did a dir to find the list of methods in boltons I got the below output
>>> import boltons
>>> dir(boltons)
['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__']
When I explicitly did
>>> from boltons.strutils import camel2under
>>> dir(boltons)
['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', 'strutils']
found that strutils getting added to attribute of boltons
Why is strutils not showing before explicit import?
From the docs on what dir does:
With an argument, attempt to return a list of valid attributes for
that object.
When we import the boltons package we can see that strutils is not an attribute on the boltons object. Therefore we do not expect it to show up in dir(boltons).
>>>import boltons
>>>getattr(boltons, 'strutils')
AttributeError: module 'boltons' has no attribute 'strutils'
The docs on importing submodules say:
For example, if package spam has a submodule foo, after importing spam.foo, spam will have an attribute foo which is bound to the submodule.
Importing a submodule creates an attribute on the package. In your example:
>>>import boltons
>>>getattr(boltons, 'strutils')
AttributeError: module 'boltons' has no attribute 'strutils'
>>>from boltons.strutils import camel2under
>>>getattr(boltons, 'strutils')
<module 'boltons.strutils' from '/usr/local/lib/python3.5/site-packages/boltons/strutils.py'>
Therefore in this case we do expect strutils to show up in dir(boltons)
I am not a professional programmer, and I'm just starting to study python.
I just figured out that just because I could do:
>>> from xml.etree.ElementTree import Element
>>> var = Element("Something")
doesn't mean I can do:
>>> import xml
>>> var = xml.etree.ElementTree.Element("Something")
In fact, doing this:
>>> import xml
>>> dir(xml)
['_MINIMUM_XMLPLUS_VERSION', '__all__', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__']
etree doesn't even appear as one of the methods provided by xml.
What is the relation between xml and xml.etree.ElementTree?
Why can I not see etree as one of xml's methods?
xml.etree is a child module to xml package. It only shows up in xml's namespace after you import xml.etree. A parent may import its child modules on its initialization (e.g. in 2.7, os imports os.path1) but it's not required to do so.
Conversely, when you import a module from a package directly, the package is automatically imported first.
Strangely, I couldn't find any phrasing in the docs stating this. But a test shows that this is exactly what happens:
$ cat test/__init__.py
print "package init"
import traceback
traceback.print_stack()
$ cat test/module.py
print "module init"
import traceback
traceback.print_stack()
$ python
<...>
>>> import test.module
package init
File "<stdin>", line 1, in <module>
File "test\__init__.py", line 3, in <module>
traceback.print_stack()
module init
File "<stdin>", line 1, in <module>
File "test\module.py", line 3, in <module>
traceback.print_stack()
1this is not documented though so don't rely on this