I work on a lib that only supports python 3.8 and up. Therefore, the built-in typing module is always present. However, my lib depends on another lib (raydp) which depends on this backport of typing to older versions of python. When this backport is present on the pythonpath, it shadows the built-in and I get a weird error:
Traceback (most recent call last):
File "/private/var/tmp/_bazel_josh/2e6587ba58699c2ae305f4c72e05bdb6/execroot/sematic/bazel-out/darwin_arm64-fastbuild/bin/sematic/ee/plugins/external_resource/ray/tests/test_spark_cluster_py39.runfiles/sematic/tools/pytest_runner.py", line 4, in <module>
import debugpy
File "/private/var/tmp/_bazel_josh/2e6587ba58699c2ae305f4c72e05bdb6/execroot/sematic/bazel-out/darwin_arm64-fastbuild/bin/sematic/ee/plugins/external_resource/ray/tests/test_spark_cluster_py39.runfiles/pip_dependencies39_debugpy/site-packages/debugpy/__init__.py", line 35, in <module>
from debugpy.public_api import * # noqa
File "/private/var/tmp/_bazel_josh/2e6587ba58699c2ae305f4c72e05bdb6/execroot/sematic/bazel-out/darwin_arm64-fastbuild/bin/sematic/ee/plugins/external_resource/ray/tests/test_spark_cluster_py39.runfiles/pip_dependencies39_debugpy/site-packages/debugpy/public_api.py", line 8, in <module>
import typing
File "/private/var/tmp/_bazel_josh/2e6587ba58699c2ae305f4c72e05bdb6/execroot/sematic/bazel-out/darwin_arm64-fastbuild/bin/sematic/ee/plugins/external_resource/ray/tests/test_spark_cluster_py39.runfiles/pip_dependencies39_typing/site-packages/typing.py", line 1359, in <module>
class Callable(extra=collections_abc.Callable, metaclass=CallableMeta):
File "/private/var/tmp/_bazel_josh/2e6587ba58699c2ae305f4c72e05bdb6/execroot/sematic/bazel-out/darwin_arm64-fastbuild/bin/sematic/ee/plugins/external_resource/ray/tests/test_spark_cluster_py39.runfiles/pip_dependencies39_typing/site-packages/typing.py", line 1007, in __new__
self._abc_registry = extra._abc_registry
AttributeError: type object 'Callable' has no attribute '_abc_registry'
I have two problems:
(1) I want to get past this in my own dev environment (yes we use bazel and rules_python, but there might be some general advice that can help too)
(2) I want to make sure that when we release a wheel with the raydp dependency, we don't cause similar trouble for our users (who will all be on python 3.8 or higher anyway)
Related
I have uninstalled and re-installed the latest version of datatable from the repo
16:42:49/seirdc2.March8.in $sudo pip3 install 'datatable==0.10.1'
Successfully installed datatable-0.10.1
Let's see the version:
import datatable as dt
print(f'datatable version={dt.__version__}')
Um oops !
Traceback (most recent call last):
File "/git/corona/python/pointr/experiments/python/datatable.py", line 18, in <module>
import datatable as dt
File "/git/corona/python/pointr/experiments/python/datatable.py", line 19, in <module>
print(f'datatable version={dt.__version__}')
AttributeError: module 'datatable' has no attribute '__version__'
But why?
Note: I have seen other strangeness with this package: e.g. not finding Frame - though not consistently.
It appears that the problem has nothing to do with datatable. Look at the traceback:
Traceback (most recent call last):
File "/git/corona/python/pointr/experiments/python/datatable.py", line 18, in <module>
import datatable as dt
File "/git/corona/python/pointr/experiments/python/datatable.py", line 19, in <module>
print(f'datatable version={dt.__version__}')
AttributeError: module 'datatable' has no attribute '__version__'
Doesn't it strike you as suspicious that you have line 18 "calling" line 19? I mean, how could it be? Here's how:
When you name your script datatable.py and then do import datatable, then instead of importing the actual module from site-packages, it imports the "module" datatable.py instead. Basically, the file imports itself. And the way python manages imports, is that it creates a "stub" module in the sys.modules first (in order to prevent infinite recursions during imports). In your case, the module tries to import itself, so the stub module is fetched instead -- and then when you try to print its __version__ variable, turns out it doesn't exist.
You can verify this by printing dt.__file__ instead, which should show the location of the file that is being imported.
Needless to say, all this is not specific to datatable in any way; for example if you created a file numpy.py and then tried to import numpy, you'd run into same problems.
I have a script named requests.py that needs to use the third-party requests package. The script either can't import the package, or can't access its functionality.
Why isn't this working, and how do I fix it?
Trying a plain import and then using the functionality results in an AttributeError:
import requests
res = requests.get('http://www.google.ca')
print(res)
Traceback (most recent call last):
File "/Users/me/dev/rough/requests.py", line 1, in <module>
import requests
File "/Users/me/dev/rough/requests.py", line 3, in <module>
requests.get('http://www.google.ca')
AttributeError: module 'requests' has no attribute 'get'
In more recent versions of Python, the error message instead reads AttributeError: partially initialized module 'requests' has no attribute 'get' (most likely due to a circular import).
Using from-import of a specific name results in an ImportError:
from requests import get
res = get('http://www.google.ca')
print(res)
Traceback (most recent call last):
File "requests.py", line 1, in <module>
from requests import get
File "/Users/me/dev/rough/requests.py", line 1, in <module>
from requests import get
ImportError: cannot import name 'get'
In more recent versions of Python, the error message instead reads ImportError: cannot import name 'get' from partially initialized module 'requests' (most likely due to a circular import) (/Users/me/dev/rough/requests.py).
Using from-import for a module inside the package results in a different ImportError:
from requests.auth import AuthBase
Traceback (most recent call last):
File "requests.py", line 1, in <module>
from requests.auth import AuthBase
File "/Users/me/dev/rough/requests.py", line 1, in <module>
from requests.auth import AuthBase
ImportError: No module named 'requests.auth'; 'requests' is not a package
Using a star-import and then using the functionality raises a NameError:
from requests import *
res = get('http://www.google.ca')
print(res)
Traceback (most recent call last):
File "requests.py", line 1, in <module>
from requests import *
File "/Users/me/dev/rough/requests.py", line 3, in <module>
res = get('http://www.google.ca')
NameError: name 'get' is not defined
This happens because your local module named requests.py shadows the installed requests module you are trying to use. The current directory is prepended to sys.path, so the local name takes precedence over the installed name.
An extra debugging tip when this comes up is to look at the Traceback carefully, and realize that the name of your script in question is matching the module you are trying to import:
Notice the name you used in your script:
File "/Users/me/dev/rough/requests.py", line 1, in <module>
The module you are trying to import: requests
Rename your module to something else to avoid the name collision.
Python may generate a requests.pyc file next to your requests.py file (in the __pycache__ directory in Python 3). Remove that as well after your rename, as the interpreter will still reference that file, re-producing the error. However, the pyc file in __pycache__ should not affect your code if the py file has been removed.
In the example, renaming the file to my_requests.py, removing requests.pyc, and running again successfully prints <Response [200]>.
The error occurs because a user-created script has a name-clash with a library filename. Note, however, that the problem can be caused indirectly. It might take a little detective work to figure out which file is causing the problem.
For example: suppose that you have a script mydecimal.py that includes import decimal, intending to use the standard library decimal library for accurate floating-point calculations with decimal numbers. That doesn't cause a problem, because there is no standard library mydecimal. However, it so happens that decimal imports numbers (another standard library module) for internal use, so a script called numbers.py in your project would cause the problem.
In one especially pernicious case, having a file named token.py in a project (or the current working directory, when starting up Python in interactive mode) causes the interactive help to break:
$ touch token.py
$ python
Python 3.8.10 (default, Nov 14 2022, 12:59:47)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> help
Type help() for interactive help, or help(object) for help about object.
>>> help()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.8/_sitebuiltins.py", line 102, in __call__
import pydoc
File "/usr/lib/python3.8/pydoc.py", line 66, in <module>
import inspect
File "/usr/lib/python3.8/inspect.py", line 40, in <module>
import linecache
File "/usr/lib/python3.8/linecache.py", line 11, in <module>
import tokenize
File "/usr/lib/python3.8/tokenize.py", line 35, in <module>
from token import EXACT_TOKEN_TYPES
ImportError: cannot import name 'EXACT_TOKEN_TYPES' from 'token' (/current/working/directory/token.py)
The traceback tells us all we need to know: calling help triggers a deferred import of the standard library pydoc, which indirectly attempts to import the standard library token, but finds our token.py which doesn't contain the appropriate name. In older versions of Python, it was even worse: tokenize would do a star-import from token, and then its top-level code would try to use a name defined there, resulting in NameError - and a stack trace not mentioning the file name token.py.
If you still encounter problems like this after tracking own and renaming or removing the appropriate .py files in your project, also check for .pyc files that Python uses to cache bytecode compilation when importing modules. In 3.x, these will be stored in folders with the special name __pycache__; it is safe to delete such folders and files, and possible to suppress them (but you normally won't want to).
I have a cloud instance of a Linux machine (openSuSE) with multiple users.
I have created a virtual environment and installed all my required libraries (including Klein).
I have two users "a" and "b".
While logged in as "a" and inside virtualenv, when I open python shell at home directory and type
import klein
it imports normally.
Now when I change directory to
/home/b/
and run the same (open python shell, import klein) while being in the same virtualenv, it gives me an error.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/a/.local/lib/python3.6/site-packages/klein/__init__.py", line 3, in <module>
from klein._plating import Plating
File "/home/a/.local/lib/python3.6/site-packages/klein/_plating.py", line 16, in <module>
from .app import _call
File "/home/a/.local/lib/python3.6/site-packages/klein/app.py", line 19, in <module>
from twisted.internet import endpoints, reactor
File "/home/a/.local/lib/python3.6/site-packages/twisted/internet/endpoints.py", line 58, in <module>
from twisted.protocols.tls import TLSMemoryBIOFactory
File "/home/a/.local/lib/python3.6/site-packages/twisted/protocols/tls.py", line 63, in <module>
from twisted.internet._sslverify import _setAcceptableProtocols
File "/home/a/.local/lib/python3.6/site-packages/twisted/internet/_sslverify.py", line 158, in <module>
verifyHostname, VerificationError = _selectVerifyImplementation()
File "/home/a/.local/lib/python3.6/site-packages/twisted/internet/_sslverify.py", line 141, in _selectVerifyImplementation
from service_identity import VerificationError
File "/home/a/.conda/envs/mm/lib/python3.6/site-packages/service_identity/__init__.py", line 7, in <module>
from . import cryptography, pyopenssl
File "/home/a/.conda/envs/mm/lib/python3.6/site-packages/service_identity/cryptography.py", line 16, in <module>
from .exceptions import SubjectAltNameWarning
File "/home/a/.conda/envs/mm/lib/python3.6/site-packages/service_identity/exceptions.py", line 21, in <module>
#attr.s
AttributeError: module 'attr' has no attribute 's'
Command "which python" gives same address at both location which is my virtualenv python address and that should be expected.
But what causes this weird python shell behavior.
Thank you
I solved it and a very shameful reason caused the error.
One of the modules Twisted uses is "attr" module. I had named one of my files attr.py and that is what was causing all the error.
I myself am not deleting this question if moderation has no problem, maybe somebody like me might be stuck at the same situation. It may help them.
Never name your python files same as that of any standard module unless overriding.
Also if your issue persists, then Jean's answer will definitely resolve it.
There can be multiple different Python packages that provide the same Python module. For example, there are at least two packages that provide the attr module:
https://pypi.org/project/attr/
https://pypi.org/project/attrs/
It's possible you've installed the wrong package based on the requirements. You can check what you have installed with pip freeze.
I have a script named requests.py that needs to use the third-party requests package. The script either can't import the package, or can't access its functionality.
Why isn't this working, and how do I fix it?
Trying a plain import and then using the functionality results in an AttributeError:
import requests
res = requests.get('http://www.google.ca')
print(res)
Traceback (most recent call last):
File "/Users/me/dev/rough/requests.py", line 1, in <module>
import requests
File "/Users/me/dev/rough/requests.py", line 3, in <module>
requests.get('http://www.google.ca')
AttributeError: module 'requests' has no attribute 'get'
In more recent versions of Python, the error message instead reads AttributeError: partially initialized module 'requests' has no attribute 'get' (most likely due to a circular import).
Using from-import of a specific name results in an ImportError:
from requests import get
res = get('http://www.google.ca')
print(res)
Traceback (most recent call last):
File "requests.py", line 1, in <module>
from requests import get
File "/Users/me/dev/rough/requests.py", line 1, in <module>
from requests import get
ImportError: cannot import name 'get'
In more recent versions of Python, the error message instead reads ImportError: cannot import name 'get' from partially initialized module 'requests' (most likely due to a circular import) (/Users/me/dev/rough/requests.py).
Using from-import for a module inside the package results in a different ImportError:
from requests.auth import AuthBase
Traceback (most recent call last):
File "requests.py", line 1, in <module>
from requests.auth import AuthBase
File "/Users/me/dev/rough/requests.py", line 1, in <module>
from requests.auth import AuthBase
ImportError: No module named 'requests.auth'; 'requests' is not a package
Using a star-import and then using the functionality raises a NameError:
from requests import *
res = get('http://www.google.ca')
print(res)
Traceback (most recent call last):
File "requests.py", line 1, in <module>
from requests import *
File "/Users/me/dev/rough/requests.py", line 3, in <module>
res = get('http://www.google.ca')
NameError: name 'get' is not defined
This happens because your local module named requests.py shadows the installed requests module you are trying to use. The current directory is prepended to sys.path, so the local name takes precedence over the installed name.
An extra debugging tip when this comes up is to look at the Traceback carefully, and realize that the name of your script in question is matching the module you are trying to import:
Notice the name you used in your script:
File "/Users/me/dev/rough/requests.py", line 1, in <module>
The module you are trying to import: requests
Rename your module to something else to avoid the name collision.
Python may generate a requests.pyc file next to your requests.py file (in the __pycache__ directory in Python 3). Remove that as well after your rename, as the interpreter will still reference that file, re-producing the error. However, the pyc file in __pycache__ should not affect your code if the py file has been removed.
In the example, renaming the file to my_requests.py, removing requests.pyc, and running again successfully prints <Response [200]>.
The error occurs because a user-created script has a name-clash with a library filename. Note, however, that the problem can be caused indirectly. It might take a little detective work to figure out which file is causing the problem.
For example: suppose that you have a script mydecimal.py that includes import decimal, intending to use the standard library decimal library for accurate floating-point calculations with decimal numbers. That doesn't cause a problem, because there is no standard library mydecimal. However, it so happens that decimal imports numbers (another standard library module) for internal use, so a script called numbers.py in your project would cause the problem.
In one especially pernicious case, having a file named token.py in a project (or the current working directory, when starting up Python in interactive mode) causes the interactive help to break:
$ touch token.py
$ python
Python 3.8.10 (default, Nov 14 2022, 12:59:47)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> help
Type help() for interactive help, or help(object) for help about object.
>>> help()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.8/_sitebuiltins.py", line 102, in __call__
import pydoc
File "/usr/lib/python3.8/pydoc.py", line 66, in <module>
import inspect
File "/usr/lib/python3.8/inspect.py", line 40, in <module>
import linecache
File "/usr/lib/python3.8/linecache.py", line 11, in <module>
import tokenize
File "/usr/lib/python3.8/tokenize.py", line 35, in <module>
from token import EXACT_TOKEN_TYPES
ImportError: cannot import name 'EXACT_TOKEN_TYPES' from 'token' (/current/working/directory/token.py)
The traceback tells us all we need to know: calling help triggers a deferred import of the standard library pydoc, which indirectly attempts to import the standard library token, but finds our token.py which doesn't contain the appropriate name. In older versions of Python, it was even worse: tokenize would do a star-import from token, and then its top-level code would try to use a name defined there, resulting in NameError - and a stack trace not mentioning the file name token.py.
If you still encounter problems like this after tracking own and renaming or removing the appropriate .py files in your project, also check for .pyc files that Python uses to cache bytecode compilation when importing modules. In 3.x, these will be stored in folders with the special name __pycache__; it is safe to delete such folders and files, and possible to suppress them (but you normally won't want to).
I have a script named requests.py that needs to use the third-party requests package. The script either can't import the package, or can't access its functionality.
Why isn't this working, and how do I fix it?
Trying a plain import and then using the functionality results in an AttributeError:
import requests
res = requests.get('http://www.google.ca')
print(res)
Traceback (most recent call last):
File "/Users/me/dev/rough/requests.py", line 1, in <module>
import requests
File "/Users/me/dev/rough/requests.py", line 3, in <module>
requests.get('http://www.google.ca')
AttributeError: module 'requests' has no attribute 'get'
In more recent versions of Python, the error message instead reads AttributeError: partially initialized module 'requests' has no attribute 'get' (most likely due to a circular import).
Using from-import of a specific name results in an ImportError:
from requests import get
res = get('http://www.google.ca')
print(res)
Traceback (most recent call last):
File "requests.py", line 1, in <module>
from requests import get
File "/Users/me/dev/rough/requests.py", line 1, in <module>
from requests import get
ImportError: cannot import name 'get'
In more recent versions of Python, the error message instead reads ImportError: cannot import name 'get' from partially initialized module 'requests' (most likely due to a circular import) (/Users/me/dev/rough/requests.py).
Using from-import for a module inside the package results in a different ImportError:
from requests.auth import AuthBase
Traceback (most recent call last):
File "requests.py", line 1, in <module>
from requests.auth import AuthBase
File "/Users/me/dev/rough/requests.py", line 1, in <module>
from requests.auth import AuthBase
ImportError: No module named 'requests.auth'; 'requests' is not a package
Using a star-import and then using the functionality raises a NameError:
from requests import *
res = get('http://www.google.ca')
print(res)
Traceback (most recent call last):
File "requests.py", line 1, in <module>
from requests import *
File "/Users/me/dev/rough/requests.py", line 3, in <module>
res = get('http://www.google.ca')
NameError: name 'get' is not defined
This happens because your local module named requests.py shadows the installed requests module you are trying to use. The current directory is prepended to sys.path, so the local name takes precedence over the installed name.
An extra debugging tip when this comes up is to look at the Traceback carefully, and realize that the name of your script in question is matching the module you are trying to import:
Notice the name you used in your script:
File "/Users/me/dev/rough/requests.py", line 1, in <module>
The module you are trying to import: requests
Rename your module to something else to avoid the name collision.
Python may generate a requests.pyc file next to your requests.py file (in the __pycache__ directory in Python 3). Remove that as well after your rename, as the interpreter will still reference that file, re-producing the error. However, the pyc file in __pycache__ should not affect your code if the py file has been removed.
In the example, renaming the file to my_requests.py, removing requests.pyc, and running again successfully prints <Response [200]>.
The error occurs because a user-created script has a name-clash with a library filename. Note, however, that the problem can be caused indirectly. It might take a little detective work to figure out which file is causing the problem.
For example: suppose that you have a script mydecimal.py that includes import decimal, intending to use the standard library decimal library for accurate floating-point calculations with decimal numbers. That doesn't cause a problem, because there is no standard library mydecimal. However, it so happens that decimal imports numbers (another standard library module) for internal use, so a script called numbers.py in your project would cause the problem.
In one especially pernicious case, having a file named token.py in a project (or the current working directory, when starting up Python in interactive mode) causes the interactive help to break:
$ touch token.py
$ python
Python 3.8.10 (default, Nov 14 2022, 12:59:47)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> help
Type help() for interactive help, or help(object) for help about object.
>>> help()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.8/_sitebuiltins.py", line 102, in __call__
import pydoc
File "/usr/lib/python3.8/pydoc.py", line 66, in <module>
import inspect
File "/usr/lib/python3.8/inspect.py", line 40, in <module>
import linecache
File "/usr/lib/python3.8/linecache.py", line 11, in <module>
import tokenize
File "/usr/lib/python3.8/tokenize.py", line 35, in <module>
from token import EXACT_TOKEN_TYPES
ImportError: cannot import name 'EXACT_TOKEN_TYPES' from 'token' (/current/working/directory/token.py)
The traceback tells us all we need to know: calling help triggers a deferred import of the standard library pydoc, which indirectly attempts to import the standard library token, but finds our token.py which doesn't contain the appropriate name. In older versions of Python, it was even worse: tokenize would do a star-import from token, and then its top-level code would try to use a name defined there, resulting in NameError - and a stack trace not mentioning the file name token.py.
If you still encounter problems like this after tracking own and renaming or removing the appropriate .py files in your project, also check for .pyc files that Python uses to cache bytecode compilation when importing modules. In 3.x, these will be stored in folders with the special name __pycache__; it is safe to delete such folders and files, and possible to suppress them (but you normally won't want to).