I am inspecting the JSON module of python 3.1, and am currently in /Lib/json/scanner.py. At the top of the file is the following line:
from _json import make_scanner as c_make_scanner
There are five .py files in the module's directory: __init__ (two leading and trailing underscores, it's formatting as bold), decoder, encoder, scanner and tool. There is no file called "json".
My question is: when doing the import, where exactly is "make_scanner" coming from?
Yes, I am very new to Python!
It's coming from a C-compiled _json.pyd (or _json.so, etc, etc, depending on the platform) that lives elsewhere on the sys.path. You can always find out where that is in your specific Python installation by importing the module yourself and looking at its __file__, e.g.:
>>> import _json
>>> _json.__file__
'/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload/_json.so'
As you see, in my installation of Python 2.6, _json comes from the lib-dynload subdirectory of lib/python2.6, and the extension used on this platform is .so.
It may be coming from a file, or it may be built-in. On Windows, it appears to be built-in.
Python 3.1.2 (r312:79149, Mar 21 2010, 00:41:52) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import _json
>>> _json.__file__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute '__file__'
and there is no _json.pyd or _json.dll in the offing.
If you want to see the source, having a binary file on your machine or not is irrelevant -- you'll need the SVN browser.
Related
From the python book:
Learning Python. 5th edition, page #727
I read the following:
if Python finds only a byte code file on the search path and no
source, it simply loads the byte code directly; this means you can
ship a program as just byte code files and avoid sending source
But when attempting the same on Python 3.5, it doesn't work:
~/Python/Module_Test$ cat a.py
a = "abc"
l = [1,2,3]
importing module 'a' created the byte-code file as:
~/Python/Module_Test/__pycache__$ ls
a.cpython-35.pyc
Now I removed the 'a.py' file and from the byte-code directory, I'm importing the module 'a':
~/Python/Module_Test/__pycache__$ python
Python 3.5.2 |Anaconda 4.2.0 (64-bit)| (default, Jul 2 2016, 17:53:06)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import a
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'a'
I even tried to add the byte-code directory to the search path, still it fails to load the module:
>>> import sys
>>> sys.path.append('/home/pradeep/Python/Module_Test/__pycache__')
>>> import a
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'a'
What I am doing wrong? Can we import module from the byte code without the source? Is my understanding of the statement of the book wrong?
Your understanding is not wrong; you can but, it isn't the best idea to do this. Afaik the default behavior of the import statement doesn't do this on its own, you'll either need to use a deprecated function from imp, write your own, or customize the import process to do it.
With imp, you'd use load_compiled as so:
from imp import load_compiled
mod = load_compiled('a', '__pycache__/a.cpython-35.pyc')
To get your module imported. The notable thing that I'm aware that Python does, is that it doesn't re-compile a module a.py if it's corresponding *.pyc is around and is still valid.
After you remove 'a.py', you need to put 'a.pyc' in its place. The import mechanism will see it (when it interprets 'import a'), and will import it successfully. It does this without referring to the cache.
To get 'a.pyc', look in the pycache and copy-to-folder/rename 'a.xxxx.pyc' to 'a.pyc' There is a python function to compile to bytecodes too, so you don't have to work with the pycache
So long as your installation of python does not change, this should work. I believe the format of '.pyc' files can differ between installations. I believe this is a way to ship bytecodes instead of source. However, you must preserve the directory structure and the placement of the 'pyc' files in the same directories as the corresponding 'py' files. Of course the target machine must have an appropriate installation of python.
I am trying to import a package present in another folder and it is working perfectly fine in python 3.4. For Example: the files are present in libraries folder
user> python
Python 3.4.1 (default, Nov 12 2014, 13:34:29)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-48)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from libraries.controller_utils import *
>>>
However when I open a new shell and use Python 2.7:
user> python
Python 2.7.4 (default, Jun 1 2015, 10:35:58)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-55)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from libraries.controller_utils import *
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named libraries.controller_utils
>>>
I tried adding the entry to sys.path but it is not helping. I read a similar question here but the solution is not helping me either as I tried both relative and absolute imports. Please advice.
EDIT: The directory structure being ~/tests/libraries/controller_utils.py. I am executing these commands inside the tests directory.
EDIT: I have added the sys.path entry as follows but it still does not recognize it. Please note that the error occurs on 2.7 but works absolutely fine on 3.4
user> cd ~/tests/
user> ls
__pycache__ backups inputs libraries openflow.py test_flow.py
user> ls libraries/
__pycache__ controller_utils.py general_utils.py general_utils.pyc tc_name_list.py test_case_utils.py
user> python
Python 2.7.4 (default, Jun 1 2015, 10:35:58)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-55)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from libraries.controller_utils import *
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named libraries.controller_utils
>>> import sys
>>> sys.path.append('libraries/')
>>> from libraries.controller_utils import *
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named libraries.controller_utils
Your libraries package is missing the __init__.py file. You can create an empty file with that name and then:
from libraries.controller_utils import *
should work.
Alternatively, if you don't want to turn libraries into a package you should add its path to sys.path and import controller_utils:
import sys
sys.path.append('libraries/')
from controller_utils import *
Note that the error is due to the fact that python2 requires the existence of __init__.py to import from packages while python3.3+ provides namespace packages (see PEP420). That's why the import doesn't fail in python3.4.
If you want your code to work in both python2 and python3 in the same way you should always add __init__.py file to packages and use from __future__ import absolute_import in your files.
See PEP 0404.
In Python 3, implicit relative imports within packages are no longer available - only absolute imports and explicit relative imports are supported. In addition, star imports (e.g. from x import *) are only permitted in module level code.
If libraries was in the same directory this would have happened to avoid clashes with packages installed at the system level. It would have been an implicit relative import.
You should be able to navigate to the correct location of the module using .., like:
from ..libraries.controller_utils import *
# or, depending of you directory structure
# put as many dots as directories you need to get out of
from ....common.libraries.controller_utils import *
But your case seems to be related to the star import. In Python 3 you can only use star imports (from x import *) at the top level of the file, i.e. not inside a function or class definition.
Running 3.3 python on CentOS 7.
Tryin' to write simple script but can't get pexpect module to work as I want
if I use interpreter python 3.3, I can write this commands correctly
[root#localhost expect]# python3.3
Python 3.3.3 (default, Apr 7 2015, 02:31:24)
[GCC 4.8.3 20140911 (Red Hat 4.8.3-9)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pexpect
>>> child = pexpect.spawn('telnet 10.1.1.1')
but If I run file pexpect.py with exactly same commands, I get
[root#localhost expect]# python3.3 /usr/etc/pexpect.py
Traceback (most recent call last):
File "/usr/etc/pexpect.py", line 1, in <module>
import pexpect
File "/usr/etc/pexpect.py", line 3, in <module>
child = pexpect.spawn('telnet 10.1.1.1');
AttributeError: 'module' object has no attribute 'spawn'
I found some similar info in the google, advice was to move .py file to another folder.
It didn't work for me.
Another advice was to delete " pycache" folder (I've got same in my pexpect.py location), but it didn't work aswell. Errors are still the same, this folder are still created after running the script (trying, I mean).
Any ideas?
You have called your file pexpect.py. You need to rename it to something else as you are importing from your file not the pexpect module. You also need to delete any .pyc in the same folder. It does not matter where you move your script, the current folder is still going to be in the path before where the actual pexpect module is.
I used help() in the python3 shell on Ubuntu 14.04
I got this output
Please help , don't know whats wrong.
Python 3.4.0 (default, Apr 11 2014, 13:05:11)
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> help()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.4/_sitebuiltins.py", line 98, in __call__
import pydoc
File "/usr/lib/python3.4/pydoc.py", line 65, in <module>
import platform
File "/home/omega/platform.py", line 2, in <module>
print("System : ",platform.uname().system)
AttributeError: 'module' object has no attribute 'uname'
>>>
The problem is that platform is the name of a stdlib module, which help uses. By creating a module of your own with the same name that occurs before the stdlib in your sys.path, you're preventing Python from using the standard one.
The fact that your own platform module tries to use the stdlib module of the same name just compounds the problem. That isn't going to work; your import platform inside that module is just importing itself.
The solution is to not collide names like this. Look at the list of the standard modules, and don't create anything with the same name as any of them if you want to use features from that module, directly or indirectly.
In other words: Rename your platform.py to something else, or put it inside a package.
File "/home/omega/platform.py", line 2, in <module>
print("System : ",platform.uname().system)
This is the problem, go to platform.py and fix it, it will be ok. It says, platform has not any method called uname you probably misstyped.
I am trying to import RRDtool into Python as I want to access an RRD database using Python, but when I am trying to import rrdtool I am getting the following error.
Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.append('/opt/rrdtool-1.4.5/bin')
>>> import rrdtool
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named rrdtool
My RRDtool is located in /opt/rrdtool-1.4.5/bin.
Well, the problem is solved just by executing the following command.
sudo apt-get install python-rrd
You probably need py-rrdtool, which you could get directly from the site or your package manager.
It is unlikely that Python modules are located inside a 'bin' folder. And importing files from some configured path requires that the referred path is a proper Python package. It means it must contain an __init__.py file.
You could use the RRDtool documentation for inspiration (man rrdpyton). Something
along the lines of
import sys
sys.path.append('/path/to/rrdtool/lib/python2.6/site-packages/')
import rrdtool
should do the trick.