I am trying to use a code which was written for python 2 and may run with python 3.6.0, but it does not run with python 3.6.4. It imports the IN module, and uses IN.IP_RECVERR. I tried to google it, but it is a 'bit' hard to find anything about a module called IN (naming fail?). To demonstrate in REPL, that it works in python 2, but not in 3.6.4:
$ python2
Python 2.7.14 (default, Jan 5 2018, 10:41:29)
[GCC 7.2.1 20171224] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import IN
>>> IN.IP_RECVERR
11
>>>
$ python3
Python 3.6.4 (default, Jan 5 2018, 02:35:40)
[GCC 7.2.1 20171224] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import IN
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'IN'
>>>
What is the replacement for this IN module in newer versions of python 3?
This is presumably the private plat-linux/IN.py module, which was never intended to be used. There have been plans to remove these plat-* files for a few zillion years, but it looks like it finally happened in issue 28027 for 3.6. As mentioned in What's New in Python 3.6:
The undocumented IN, CDROM, DLFCN, TYPES, CDIO, and STROPTS modules have been removed. They had been available in the platform specific Lib/plat-*/ directories, but were chronically out of date, inconsistently available across platforms, and unmaintained. The script that created these modules is still available in the source distribution at Tools/scripts/h2py.py.
Most of the useful constants that are at least somewhat portable (as in you can expect them to be available and work the same on your old laptop's linux and your brand-new Galaxy's linux, if not on OS X or Solaris) have long been made available through other places in the stdlib.
I think this specific one you're looking for is an example of not completely useless, but not portable enough to put anywhere safe, because linux documents the existence of IP_RECVERR, but not its value. So, you really need the version from your own system's ip headers.
The way to do this safely, if you actually need the IN module, is to run Tools/scripts/h2py.py with the Python version you're using, on the specific platform you need. That will generate an IN.py from the appropriate headers on your system (or on your cross-compilation target), which you can then use on that system. If you want to distribute your code, you'd probably need to put a step to do that into the setup.py, so it'll be run at install time (and at wheel-building time for people who install pre-built wheels, but you may need to be careful to make sure the targets are specific enough).
If you don't need to be particularly portable, you just need to access the one value in a few scripts that you're only deploying on your laptop or your company's set of identical containers or the like, you may be better off hardcoding the values (with a nice scare comment explaining the details).
Related
I am trying to import a python (2.7.5) module but I'm not sure if I am going at it in the right way. I usually work in Jupyter Notebook (in a seperate Conda env) to keeps things organized per project. Now I am trying to import a module called otbApplication which are Python bindings for a GIS program called Orfeo Toolbox. The thing is, Orfeo Toolbox (together with QGIS) comes with its own Python install (and subsequent paths) and even its own CMD prompt (assuming you use OSGEO4W). If I use this CMD prompt to start Python and import otbApplication, it works fine. But I want to install more packages and just work within my own 'usual' environment (Jupyter Notebook) in this case.
How should you normally reuse modules between Python installations?
I already tried placing a .pth txt file containing the path to the module in one of the sys.path locations of a different Python installation but it wouldn't even find it. I tried to force it by hosting a notebook (with the same Python install) in the same folder as the module and then importing it. I got the following error which makes me question if I am going at this the wrong way:
Python 2.7.5 |Continuum Analytics, Inc.| (default, Jul 1 2013, 12:37:52)
[MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import otbApplication
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "otbApplication.py", line 28, in <module>
_otbApplication = swig_import_helper()
File "otbApplication.py", line 24, in swig_import_helper
_mod = imp.load_module('_otbApplication', fp, pathname, description)
ImportError: DLL load failed: The specified module could not be found.
This means that forcing the same paths to my new installation is not enough, what am I missing? Apologies for the long story (or the probable butchering of some of the terms).
Kind regards,
Jasper
You've got the right idea with the .pth file, but in order to get it to work you need to check some prerequisites. Obviously, the syntax needs to be good (just give the directory locations separated by line breaks). A common issue is that folks don't put the .pth files in the correct directory (usually though not necessarily \Lib\site-packages). I bet that if you check these you'll be okay.
**Also: as noted in the comments be aware that 32-bit python isn't going to like a 64-bit DLL and verse-visa, so ensure that you're running the right version of python when you try to access those libs.
I have weird problem that I cannot put my finger on. There is a program that I use (and contribute from time to time) that has colorized console output. Everything worked great until I reinstalled Windows. Now I cannot get colorized output.
This is the script that is used for colorizing.
I have managed to narrow down the problem to, more or less, simple situation, but I have no idea what is wrong.
This is console prompt that works as expected (string test is printed in red):
Python 2.7.3 (default, Apr 10 2012, 23:24:47) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.insert(0, r'c:\bin\SV\tea\src')
>>> from tea.console.color import cprint, Color
>>> cprint('test\n', Color.red)
test
>>>
But when I run following script with same version of python I get output test but not in red color (there is no color, just default console color):
import sys
sys.path.insert(0, r'c:\bin\SV\tea\src')
from tea.console.color import cprint, Color
cprint('test\n', Color.red)
The same setup worked before I reinstalled my system.
I have checked, environment variables in interactive mode and script are the same.
I have tried this in standard windows command prompt and Console, program that I
usually use.
OS in question is Windows 8 and before reinstall this was also used on Windows 8.
Same code with same setup works at computer at work (Windows 7).
I have Python 2.7 and Python 3.3 installed (as I did before). I have tried to run script
with calling python interpreter directly (c:\Python27\python.exe) or with py -2,
but it does not help.
IPython and mercurial colorizes output as it should.
Any ideas what can I try to make this work?
Edit:
Maybe it was not clear, but script I use to colorize output is given in a link in question. Here it is once again:
https://bitbucket.org/alefnula/tea/src/dc14009a19d66f92463549332a321b29c71d47b8/src/tea/console/color.py?at=default
I have found the problem and solution.
I believe that the problem was the bug in x64 ctypes module. I had Python 2.7 x64 installed and with that version following line (from script that I linked in question):
ctypes.windll.kernel32.SetConsoleTextAttribute(std_out_handle, code)
returns error code 6 with description The handle is invalid. After some investigation, I deduced that problem might be x64 version of python, so I installed 32-bit version and everything works as expected.
Since this solves my problem, and I do not have the time for deeper analysis I will leave it at this, just wanted to give some kind of resolution for question.
I learned about pystones today and so I decided to see what my various environments were like. I ran pystones on my laptop that is running windows on the bare metal and got these results
Python 2.7.2 (default, Jun 12 2011, 15:08:59) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from test import pystone
>>> for i in range(0,10):
... pystone.pystones()
...
(1.636334799754252, 30556.094026423627)
(2.1157907919853756, 23631.82607155689)
(2.5324817108003685, 19743.479207278437)
(2.541626695533182, 19672.4405231788)
(2.536022267835051, 19715.915208695682)
(2.540327088340973, 19682.50475676099)
(2.544761766911506, 19648.20465716261)
(2.540296805235016, 19682.739393664764)
(2.533851636391205, 19732.804905346253)
(2.536483186973612, 19712.3325148696)
Then I ran it on some of our linux VMs and got 2.7-3.4 times better performance. So I fired up my vmware Linux VM on my laptop and reran the same test and got these results:
Python 2.7.2+ (default, Oct 4 2011, 20:03:08)
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> for i in range(0,10):
... pystone.pystones()
...
(1.75, 28571.428571428572)
(1.17, 42735.042735042734)
(1.6600000000000001, 30120.48192771084)
(1.8399999999999999, 27173.913043478264)
(1.8200000000000003, 27472.52747252747)
(1.8099999999999987, 27624.30939226521)
(1.3099999999999987, 38167.938931297744)
(1.7800000000000011, 28089.88764044942)
(1.8200000000000038, 27472.527472527414)
(1.490000000000002, 33557.04697986573)
I can't quite understand how the linux VM running inside the same windows is actually FASTER than python running on the same bare metal under windows.
What is so different about python on windows that it performs slower on the bare OS than it does inside a VM running Linux on the same box?
More details
Windows platform Win7x64
32 bit python running on both platforms
32 bit linux VM running the windows platform in VMWare
Had similar problem on windows 10 - it was because of windows defender.
I had to exclude python directories and process in windows defender settings and restart computer.
Before: I had to wait like ~20 seconds to run any python code - now it's milliseconds.
I can't answer your question, however consider this list of things that could be making a difference:
You're using different versions of Python. "2.7.2+" indicates that your linux Python was built from a version control checkout rather than a release.
They were compiled with different compilers (and conceivably meaningfully different optimization levels).
You haven't mentioned reproducing this much. It's conceivable it was a fluke if you haven't.
Your VM might be timing inaccurately.
You're linking different implementations of Python's dependencies, notably libc as Ignacio Vazquez-Abrams points out.
I don't know what pystone's actual benchmarks are like, but many things work differently--things like unicode handling or disk IO could be system-dependent factors.
Do you run antivirus software on that Windows box? This perhaps could explain it. I personally like to add Python, Cygwin and my sources directory to antivirus exclusion list - I think I get a small, but noticeable speedup. Maybe that explains your results.
Benchmark your startup, but there are just simply some slow modules to initialize on windows. A tiny hack that saves me a second on startup every time:
import os
import mimetypes #mimetypes gets imported later in dep chain
if __name__ == "__main__":
# stub this out, so registry db wont ever be read, not needed
mimetypes._winreg = None
Another source of slowness is, multiple standard library modules compile and cache their regexes at import time. re.compile just looks like its slow on windows
I am teaching myself python from this site. On Chapter 3, when I typed the code in the given example, I got the following error--
Python 3.2 (r32:88445, Mar 25 2011, 19:28:28)
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import turtle
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "turtle.py", line 2, in <module>
wn = turtle.Screen()
AttributeError: 'module' object has no attribute 'Screen'
>>>
Is this something that I need to download and install? I tried looking into docs.python.org, but my nose started to bleed reading all that tech stuff.
Kindly point me in the right direction please? Thank you.
Adam Bernier's answer is probably correct. It looks like you have a file called turtle.py that Python is picking up before the one that came with your Python installation.
To track down these problems:
% python
Python 2.7.1 (r271:86832, Jan 29 2011, 13:30:16)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
[...] # Your ${PYTHONPATH}
>>> import turtle
>>> turtle.__file__
'/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk/turtle.pyc' # Should be under your Python installation.
>>>
If you see something like this:
>>> import turtle
>>> turtle.__file__
'turtle.py'
Then you'll want to move turtle.py (and any corresponding turtle.pyc or turtle.pyo files) in your current working directory out of the way.
As per the comments below, you'll find a wealth of information about a module, including its pathname and contents by calling help() upon it. For example:
>>> import turtle
>>> help(turtle)
Rename turtle.py. It is clashing with the imported module of the same name.
I tested that the code from that site works in Python 2.6 (without installing any external packages).
From http://docs.python.org/tutorial/modules.html#the-module-search-path
When a module named spam is imported, the interpreter searches for a file named spam.py in the current directory, and then in the list of directories specified by the environment variable PYTHONPATH.
So the Python interpreter is finding your turtle.py file, but not seeing a Screen class within that file.
Johnsyweb's answer contains several good tips on how to debug this kind of issue. Perhaps the most direct way of determining where on the filesystem an imported module resides is to use repr(module) or simply type the module name at the REPL prompt, e.g.:
>>> turtle
<module 'turtle' from '/usr/lib/python2.6/lib-tk/turtle.pyc'>
Another problem that people may encounter is due to an installation issue on Linux systems. On my Windows machine, 'turtle' was just there and I was able to import turtle with no problem. When I tried to import turtle in Ubuntu, it didn't find the module, so I tried to install it.
When I did sudo pip install turtle, it installed a package 'turtle' which apparently is very different: "Turtle is an HTTP proxy whose purpose is to throttle connections to specific hostnames ...." This 'turtle' most certainly does not have "Screen" or anything related to a little drawing turtle. So I ended up with the same error as the user in the question of module has no attribute 'Screen'.
For Ubuntu, what I needed to do was:
sudo pip uninstall turtle
sudo apt-get install python-tk
Then when I did import turtle, all of the expected modules were found.
Go to the directory where you save your python files. There is a file named turtle.py. Either remove it, or rename it. This will work.
thanks,
Probably not related, but I spent some time tracking down this same error and found a different cause: I had a file named "copy.py" in the folder with my project.
This "copy.py" was an assignment to make a function that returns a deep copy of a list. The turtle library imports "deepcopy" from "copy"; turns out there's already a "copy.py" as part of python (which I'd never seen/used) & by having a file named "copy.py" in my project folder, it was causing turtle to import the wrong copy.py, which was causing the error to be thrown in turtle (my copy.py assignment's deepcopy function didn't work the same way as the one in python).
This is a more general suggestion, but it's good to double check and make sure you don't have any filenames that are in conflict with actual python imports used in your project. There are too many to list here, but ones used by turtle include: tkinter, types, math, time, inspect, sys, and copy. If you have any of these with a .py in your folder (for example, if you had previously created an inspect.py), turtle will be loading that instead of the built-in library & will not work.
I've installed and configured PyDev version 1.6.5.2011020317 inside Eclipse, running on Mac OS X 10.6.6:
Version: Helios Service Release 1
Build id: 20100917-0705
I used 'Auto Config' to set up my Python interpreter: it correctly found /usr/bin/python (which is Python version 2.6.1) and added various system folders to the PYTHONPATH, including /System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/PyObjC. Now that path is the correct path to the Foundation module in OS X, as evinced by the command-line interpreter:
$ python
Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import Foundation
>>> Foundation.__path__
['/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/PyObjC/Foundation']
So why does PyDev complain about "Undefined variable from import: NSDictionary" on this class:
import Foundation
class MyClass(object):
def __init__(self, projectPath):
'''
Constructor
'''
self.projectDict = Foundation.NSDictionary.dictionaryWithContentsOfFile_(projectPath)
when I can use that class without any problem from the command-line interpreter?
Update: OK, I found out why it complains, which is that the Foundation module is using ScriptingBridge to dynamically generate the classes - presumably pydev isn't actually importing the module to see what classes are inside, it's just looking for .py[c] files. So let my question not be "why does this happen", but "what do I do to fix it"?
Why does this happen?: PyDev has no support for parsing the PyObjC scripting bridge metadata, and therefore has no way of introspecting / extracting symbols for many of the PyObjC classes.
What to do to fix it: In the PyDev source code there are several Python scripts which handle discovery of this metadata. The scripts are executed by Eclipse using the configured interpreter, and they return strings which are used to configure the interpreter, fill in completion lists, show usage tips, etc.
The scripts which seem relevant to your needs are:
interpreterInfo.py - called to obtain the list of directories and other default top-level imports for a given interpreter.
importsTipper.py - generates usage tips for a given symbol.
pycompletion.py - generates a list of completions for a given symbol.
Examples of calls to the above scripts using the system interpreter:
% /usr/bin/python interpreterInfo.py | grep PyObjC
|/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/PyObjCINS_PATH
Generate completions for a module:
>>> import pycompletion
>>> print pycompletion.GetImports('os')
##COMPLETIONS(/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/os.py,(EX_CANTCREAT, 3),(EX_CONFIG, 3),(EX_DATAERR, 3), ....
It seems possible to create a simple macobjc.py library with routines which detect and read the PyObjC.bridgesupport file(s). The PyDev scripts could be modified to call into this library in order to return the list of valid completions for those classes. You'll need to point Eclipse to a local copy of the PyDev source in order to develop and test your patches against these files. Once you're done you can send it upstream; I have to believe the PyDev folks would accept a well-written patch to support PyObjC completions.