From the python book:
Learning Python. 5th edition, page #727
I read the following:
if Python finds only a byte code file on the search path and no
source, it simply loads the byte code directly; this means you can
ship a program as just byte code files and avoid sending source
But when attempting the same on Python 3.5, it doesn't work:
~/Python/Module_Test$ cat a.py
a = "abc"
l = [1,2,3]
importing module 'a' created the byte-code file as:
~/Python/Module_Test/__pycache__$ ls
a.cpython-35.pyc
Now I removed the 'a.py' file and from the byte-code directory, I'm importing the module 'a':
~/Python/Module_Test/__pycache__$ python
Python 3.5.2 |Anaconda 4.2.0 (64-bit)| (default, Jul 2 2016, 17:53:06)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import a
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'a'
I even tried to add the byte-code directory to the search path, still it fails to load the module:
>>> import sys
>>> sys.path.append('/home/pradeep/Python/Module_Test/__pycache__')
>>> import a
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'a'
What I am doing wrong? Can we import module from the byte code without the source? Is my understanding of the statement of the book wrong?
Your understanding is not wrong; you can but, it isn't the best idea to do this. Afaik the default behavior of the import statement doesn't do this on its own, you'll either need to use a deprecated function from imp, write your own, or customize the import process to do it.
With imp, you'd use load_compiled as so:
from imp import load_compiled
mod = load_compiled('a', '__pycache__/a.cpython-35.pyc')
To get your module imported. The notable thing that I'm aware that Python does, is that it doesn't re-compile a module a.py if it's corresponding *.pyc is around and is still valid.
After you remove 'a.py', you need to put 'a.pyc' in its place. The import mechanism will see it (when it interprets 'import a'), and will import it successfully. It does this without referring to the cache.
To get 'a.pyc', look in the pycache and copy-to-folder/rename 'a.xxxx.pyc' to 'a.pyc' There is a python function to compile to bytecodes too, so you don't have to work with the pycache
So long as your installation of python does not change, this should work. I believe the format of '.pyc' files can differ between installations. I believe this is a way to ship bytecodes instead of source. However, you must preserve the directory structure and the placement of the 'pyc' files in the same directories as the corresponding 'py' files. Of course the target machine must have an appropriate installation of python.
Related
$ echo "Your code is bad and you should feel bad" > "<stdin>"
$ python
Python 3.6.0 (default, Dec 28 2016, 19:53:26)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 2 + '2'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
Your code is bad and you should feel bad
TypeError: unsupported operand type(s) for +: 'int' and 'str'
Why does Python confuse the string "<stdin>" with a file matching that filename? I didn't want Python trying to just read whatever files from my disk if it encountered an unhandled exception.
You can also get it with the "<string>" filename:
$ echo "pining for the fjords" > "<string>"
$ python -c 'wat'
Traceback (most recent call last):
File "<string>", line 1, in <module>
pining for the fjords
NameError: name 'wat' is not defined
Is there any way to prevent that behaviour, or is it hardcoded into the REPL?
Python doesn't keep track of what source code corresponds to any compiled bytecode. It might not even read that source code until it needs to print a traceback, for example if a module is loaded from a .pyc file.
When Python needs to print a traceback, that's when it tries to find source code corresponding to all the stack frames involved. The file name and line number you see in the stack trace are all Python has to go on. If it were using the traceback module, the code path would go through a section in linecache that excludes filenames starting and ending with < and >, but the default sys.excepthook doesn't go through that path.
The default sys.excepthook goes through the native call PyErr_Display, which eventually winds up using _Py_DisplaySourceLine to display individual source lines. _Py_DisplaySourceLine unconditionally tries to find the file in the current working directory (for some reason - misguided optimization?), then calls _Py_FindSourceFile to search sys.path for a file matching that name if the working directory didn't have it. Usually, it won't find a <stdin> or <string> file, and it'll just skip printing source code when it can't find a file, but if it finds one, it prints from that file.
I initially thought you could prevent this by running Python with the -I flag, putting it in isolated mode. One of the effects of isolated mode is to remove the script's directory from sys.path. Experiment proved that this didn't change things, which is when I realized _Py_DisplaySourceLine tries the working directory no matter what.
It would be fairly straightforward to fix this by excluding <> filenames in the native code path, like linecache does. The code that unconditionally searches the current directory for the file should also be changed.
When I try to import the module illustris_python I get the error
ImportError: No module named 'util'
The module util is in the directory below the module snapshot.py that needs it, so I am confused as to why Python sees one module, but not the other.
I have included the import call as well as traceback below.
Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 24 2015, 22:44:40) [MSC v.1600 64 bit (AMD64)]
Type "copyright", "credits" or "license" for more information.
IPython 3.0.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
%guiref -> A brief reference about the graphical user interface.
In [1]: import illustris_python as il
Traceback (most recent call last):
File "<ipython-input-1-ff06d24b4811>", line 1, in <module>
import illustris_python as il
File "C:\WinPython-64bit-3.4.3.2\python-3.4.3.amd64\lib\site-packages\illustris_python\__init__.py", line 3, in <module>
from . import *
File "C:\WinPython-64bit-3.4.3.2\python-3.4.3.amd64\lib\site-packages\illustris_python\snapshot.py", line 6, in <module>
from util import partTypeNum
ImportError: No module named 'util'
In [2]:
Screenshot showing location of util:
Looking at the BitBucket repo, I'm pretty sure the problem is that this code is Python 2.x-only. Someone's done some work to clean it up for an eventual port, but there's still more to be done.
This particular error is near the top of snapshot.py:
from util import partTypeNum
In Python 2.6, this is a relative import (it's "deprecated" by PEP 328, but I'm pretty sure you don't actually get the warning by default…), so it first looks in the same package as snapshot.py, where it finds a util.py, before looking through your sys.path.
In Python 3.4, it's an absolute import, so it just looks in your sys.path (well, it calls your top-level module finders, but usually that means looking in your sys.path), and there is no util.py there.
If you're trying to finish porting this sample code to 3.x yourself, just change it to an explicit relative import:
from .util import partTypeNum
Right now I have solve the problem you have. What I have done is open the terminal in the direction of "illustris_python". Hope it could be helpful.
I fixed this issue. I noticed the library need to have access to a file titled.util.py. I just copied the file from the related github, and pasted in the folder where the library was installed.
Running 3.3 python on CentOS 7.
Tryin' to write simple script but can't get pexpect module to work as I want
if I use interpreter python 3.3, I can write this commands correctly
[root#localhost expect]# python3.3
Python 3.3.3 (default, Apr 7 2015, 02:31:24)
[GCC 4.8.3 20140911 (Red Hat 4.8.3-9)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pexpect
>>> child = pexpect.spawn('telnet 10.1.1.1')
but If I run file pexpect.py with exactly same commands, I get
[root#localhost expect]# python3.3 /usr/etc/pexpect.py
Traceback (most recent call last):
File "/usr/etc/pexpect.py", line 1, in <module>
import pexpect
File "/usr/etc/pexpect.py", line 3, in <module>
child = pexpect.spawn('telnet 10.1.1.1');
AttributeError: 'module' object has no attribute 'spawn'
I found some similar info in the google, advice was to move .py file to another folder.
It didn't work for me.
Another advice was to delete " pycache" folder (I've got same in my pexpect.py location), but it didn't work aswell. Errors are still the same, this folder are still created after running the script (trying, I mean).
Any ideas?
You have called your file pexpect.py. You need to rename it to something else as you are importing from your file not the pexpect module. You also need to delete any .pyc in the same folder. It does not matter where you move your script, the current folder is still going to be in the path before where the actual pexpect module is.
I used help() in the python3 shell on Ubuntu 14.04
I got this output
Please help , don't know whats wrong.
Python 3.4.0 (default, Apr 11 2014, 13:05:11)
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> help()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.4/_sitebuiltins.py", line 98, in __call__
import pydoc
File "/usr/lib/python3.4/pydoc.py", line 65, in <module>
import platform
File "/home/omega/platform.py", line 2, in <module>
print("System : ",platform.uname().system)
AttributeError: 'module' object has no attribute 'uname'
>>>
The problem is that platform is the name of a stdlib module, which help uses. By creating a module of your own with the same name that occurs before the stdlib in your sys.path, you're preventing Python from using the standard one.
The fact that your own platform module tries to use the stdlib module of the same name just compounds the problem. That isn't going to work; your import platform inside that module is just importing itself.
The solution is to not collide names like this. Look at the list of the standard modules, and don't create anything with the same name as any of them if you want to use features from that module, directly or indirectly.
In other words: Rename your platform.py to something else, or put it inside a package.
File "/home/omega/platform.py", line 2, in <module>
print("System : ",platform.uname().system)
This is the problem, go to platform.py and fix it, it will be ok. It says, platform has not any method called uname you probably misstyped.
I am inspecting the JSON module of python 3.1, and am currently in /Lib/json/scanner.py. At the top of the file is the following line:
from _json import make_scanner as c_make_scanner
There are five .py files in the module's directory: __init__ (two leading and trailing underscores, it's formatting as bold), decoder, encoder, scanner and tool. There is no file called "json".
My question is: when doing the import, where exactly is "make_scanner" coming from?
Yes, I am very new to Python!
It's coming from a C-compiled _json.pyd (or _json.so, etc, etc, depending on the platform) that lives elsewhere on the sys.path. You can always find out where that is in your specific Python installation by importing the module yourself and looking at its __file__, e.g.:
>>> import _json
>>> _json.__file__
'/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/lib-dynload/_json.so'
As you see, in my installation of Python 2.6, _json comes from the lib-dynload subdirectory of lib/python2.6, and the extension used on this platform is .so.
It may be coming from a file, or it may be built-in. On Windows, it appears to be built-in.
Python 3.1.2 (r312:79149, Mar 21 2010, 00:41:52) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import _json
>>> _json.__file__
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute '__file__'
and there is no _json.pyd or _json.dll in the offing.
If you want to see the source, having a binary file on your machine or not is irrelevant -- you'll need the SVN browser.