We Keep Coding

What is the replacement for python IN package?

I am trying to use a code which was written for python 2 and may run with python 3.6.0, but it does not run with python 3.6.4. It imports the IN module, and uses IN.IP_RECVERR. I tried to google it, but it is a 'bit' hard to find anything about a module called IN (naming fail?). To demonstrate in REPL, that it works in python 2, but not in 3.6.4:
$ python2
Python 2.7.14 (default, Jan 5 2018, 10:41:29)
[GCC 7.2.1 20171224] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import IN
>>> IN.IP_RECVERR
11
>>>
$ python3
Python 3.6.4 (default, Jan 5 2018, 02:35:40)
[GCC 7.2.1 20171224] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import IN
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'IN'
>>>
What is the replacement for this IN module in newer versions of python 3?

This is presumably the private plat-linux/IN.py module, which was never intended to be used. There have been plans to remove these plat-* files for a few zillion years, but it looks like it finally happened in issue 28027 for 3.6. As mentioned in What's New in Python 3.6:
The undocumented IN, CDROM, DLFCN, TYPES, CDIO, and STROPTS modules have been removed. They had been available in the platform specific Lib/plat-*/ directories, but were chronically out of date, inconsistently available across platforms, and unmaintained. The script that created these modules is still available in the source distribution at Tools/scripts/h2py.py.
Most of the useful constants that are at least somewhat portable (as in you can expect them to be available and work the same on your old laptop's linux and your brand-new Galaxy's linux, if not on OS X or Solaris) have long been made available through other places in the stdlib.
I think this specific one you're looking for is an example of not completely useless, but not portable enough to put anywhere safe, because linux documents the existence of IP_RECVERR, but not its value. So, you really need the version from your own system's ip headers.
The way to do this safely, if you actually need the IN module, is to run Tools/scripts/h2py.py with the Python version you're using, on the specific platform you need. That will generate an IN.py from the appropriate headers on your system (or on your cross-compilation target), which you can then use on that system. If you want to distribute your code, you'd probably need to put a step to do that into the setup.py, so it'll be run at install time (and at wheel-building time for people who install pre-built wheels, but you may need to be careful to make sure the targets are specific enough).
If you don't need to be particularly portable, you just need to access the one value in a few scripts that you're only deploying on your laptop or your company's set of identical containers or the like, you may be better off hardcoding the values (with a nice scare comment explaining the details).

pylint 1.4 reports E1101(no-member) on all C extensions

We've been long-time fans of pylint. Its static analysis has become a critical part of all our python projects and has saved tons of time chasing obscure bugs. But after upgrading from 1.3 -> 1.4, almost all compiled c extensions result in E1101(no-member) errors.
Projects that previously run perfectly clean through pylint 1.3 now complain about almost every C extension member with E1101. We've been forced to disable E1101 errors, but this materially detracts from the usefulness of pylint.
For example, this perfectly valid use of the lxml package
r"""valid.py: demonstrate pylint 1.4 error"""
from lxml import etree
print etree.Element('mydoc')
Run this through pylint, and it reports:
$ pylint -rn valid.py
No config file found, using default configuration
************* Module valid
E: 3, 6: Module 'lxml.etree' has no 'Element' member (no-member)
But it is perfectly valid:
$ python valid.py
<Element mydoc at 7fddf67b1ba8>
Here's where it gets really weird. A very small handful of C extensions seem to work just fine through pylint, e.g.:
r"""valid2.py: this one works fine"""
import sqlite3
print sqlite3.version
$ pylint -rn valid2.py
No config file found, using default configuration
My question is, has anyone else witnessed this? And if so, would you be willing to share your workaround/solution?
We've experimented with trying to create plugins to suppress these warnings
(http://docs.pylint.org/plugins.html#enter-plugin), but we're having difficulty making heads or tails of the docs -- and the astroid base class is uber-complex and has defied our attempts to grok it.
For real bonus points (and our eternal gratitude) we'd love to understand what changed in pylint. We'd be happy to fix the code (or at least publish a best practice document for C extension authors) that would satisfy pylint.
Platform details
$ pylint --version
No config file found, using default configuration
pylint 1.4.0,
astroid 1.3.2, common 0.63.2
Python 2.7.5 (default, Jul 1 2013, 18:09:11)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-3)]

Shortly after posting my question, I found the answer. The change was in fact done on purpose as a security measure. Pylint imports modules to effectively identify valid methods and attributes. It was decided that importing c extensions that are not part of the python stdlib is a security risk and could introduce malicious code.
This was done in the release of Astroid 1.3.1 https://mail.python.org/pipermail/code-quality/2014-November/000394.html
Only C extensions from trusted sources (the standard library) are
loaded into the examining Python process to build an AST from the live
module.
There are four solutions if you want to use pylint on projects that import non-stdlib c extensions.
1) Disable safety using the --unsafe-load-any-extension=y command line option. This feature is undocumented and classified as a hidden option (https://mail.python.org/pipermail/code-quality/2014-November/000439.html).
2) Disable safety using the pylint.rc setting unsafe-load-any-extensions=yes. This is recommended over option 1 and includes full documentation in the default pylint.rc file (created with --generate-rcfile).
3) Specifically list packages or modules names that you trust to be loaded by pylint in the pylint.rc file using the extension-pkg-whitelist= option.
4) Create a plugin to manipulate the AST (I have no idea how to effect this -- but it's regularly discussed on on the pylint mailing list).
We opted for Option 3. We added the following line to our project pylint.rc file:
extension-pkg-whitelist=lxml

#user590028, thanks a lot for your answer! I just ran into this same problem with the libraries win32api, win32evtlog, win32file, win32gui, and win32process, and your solution worked.
I used another method I think is worth posting here, which is to call pylint and pass the whitelisted packages as a parameter:
pylint --extension-pkg-whitelist=win32api,win32evtlog,win32file,win32gui,win32process myfile.py

For those of you using VS Code, it's a bit tricky to find where to put the command as I couldn't find my executable.
In VS Code;
click on File > Preferences > Settings.
Scroll down to "Python Configurations" in the left window
scroll down to "Python Linting: Mypy Args" in the right window
click on "Edit in settings.json" link
edit the json to include:
"--extension-pkg-whitelist="
I had to do all this because PyLint isn't executable from my Windows command line...

If you're using VS Code for Mac, this is what you need to do in order to edit the settings.json file:
Click on Code (i.e. the Visual Studio Code tab which is on the left of the 'File' tab) -> Preferences - > Settings
Scroll down to Extensions and click on Python in the list.
Click on any of the Edit in settings.json links. This opens up settings.json for editing.
Add the line "python.linting.pylintArgs": ["----extension-pkg-whitelist=1xml"].

Pylons paster shell does not run in ipython

I install ipython, and then i run
./paster shell dev.ini
command, paster open standard python console. How can I make it run ipython?

Here's how it worked for me on Fedora 17 with IPython 0.12, paste-1.7.5.1 and pylons 1.0:
$ paster shell dev.ini
Pylons Interactive Shell
Python 2.7.3 (default, Jul 24 2012, 10:05:38)
[GCC 4.7.0 20120507 (Red Hat 4.7.0-5)]
All objects from project.lib.base are available
Additional Objects:
mapper - Routes mapper object
wsgiapp - This project's WSGI App instance
app - paste.fixture wrapped around wsgiapp
>>> __name__ = '__main__'
>>> import IPython
>>> IPython.embed()
Python 2.7.3 (default, Jul 24 2012, 10:05:38)
Type "copyright", "credits" or "license" for more information.
IPython 0.12 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]:
Resetting __name__ is necessary because either Pylons/Paste sets __name__ to "pylons-admin" which confuses IPython (it tries to lookup the main module by its name in sys.modules).

I solved this problem downgrading ipython to version 0.10

Did you try the steps from the Pylons Quick Site Development:
13.1.2 Using an IPython embedded shell
IPython provides a more powerfull interactive prompt and a powerful
embedded shell. If you are a Python programmer and have not yet tried
IPython, you definitely should look into it.
First, import from IPython -- Add something like the following at the
top of your controller module, in our case in
firstapp/controllers/firstcontroller.py:
from IPython.Shell import IPShellEmbed args = ['-pdb', '-pi1', 'In
<\#>: ', '-pi2', ' .\D.: ',
'-po', 'Out<\#>: ', '-nosep'] ipshell = IPShellEmbed(args,
banner = 'Entering IPython. Press Ctrl-D to exit.',
exit_msg = 'Leaving Interpreter, back to Pylons.')
Then, place this code in your action/method:
ipshell('We are at action abc')
Return to Pylons and continue on responding to the request by pressing
Ctrl-D.
Note that because of some idiosyncratic feature of
IPython.Shell.IPShellEmbed, I had to put the following before each
call to ipshell():
ipshell.IP.exit_now = False ipshell('We are at action abc')

Is there any chance that your ipython is installed globally, but that you're running pylons from a virtualenv that is --no-site-packages? If that's the case, then pylons won't see your installation of ipython.

Ipython updated to version .11 just recently. Some of the components no longer work correctly, especially with 3rd party libraries. Check the mailing lists for pylons, and you may want to consider a bug report.

Try this:
paster ishell dev.ini

module object has no attribute 'Screen'

I am teaching myself python from this site. On Chapter 3, when I typed the code in the given example, I got the following error--
Python 3.2 (r32:88445, Mar 25 2011, 19:28:28)
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import turtle
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "turtle.py", line 2, in <module>
wn = turtle.Screen()
AttributeError: 'module' object has no attribute 'Screen'
>>>
Is this something that I need to download and install? I tried looking into docs.python.org, but my nose started to bleed reading all that tech stuff.
Kindly point me in the right direction please? Thank you.

Adam Bernier's answer is probably correct. It looks like you have a file called turtle.py that Python is picking up before the one that came with your Python installation.
To track down these problems:
% python
Python 2.7.1 (r271:86832, Jan 29 2011, 13:30:16)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
[...] # Your ${PYTHONPATH}
>>> import turtle
>>> turtle.__file__
'/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk/turtle.pyc' # Should be under your Python installation.
>>>
If you see something like this:
>>> import turtle
>>> turtle.__file__
'turtle.py'
Then you'll want to move turtle.py (and any corresponding turtle.pyc or turtle.pyo files) in your current working directory out of the way.
As per the comments below, you'll find a wealth of information about a module, including its pathname and contents by calling help() upon it. For example:
>>> import turtle
>>> help(turtle)

Rename turtle.py. It is clashing with the imported module of the same name.
I tested that the code from that site works in Python 2.6 (without installing any external packages).
From http://docs.python.org/tutorial/modules.html#the-module-search-path
When a module named spam is imported, the interpreter searches for a file named spam.py in the current directory, and then in the list of directories specified by the environment variable PYTHONPATH.
So the Python interpreter is finding your turtle.py file, but not seeing a Screen class within that file.
Johnsyweb's answer contains several good tips on how to debug this kind of issue. Perhaps the most direct way of determining where on the filesystem an imported module resides is to use repr(module) or simply type the module name at the REPL prompt, e.g.:
>>> turtle
<module 'turtle' from '/usr/lib/python2.6/lib-tk/turtle.pyc'>

Another problem that people may encounter is due to an installation issue on Linux systems. On my Windows machine, 'turtle' was just there and I was able to import turtle with no problem. When I tried to import turtle in Ubuntu, it didn't find the module, so I tried to install it.
When I did sudo pip install turtle, it installed a package 'turtle' which apparently is very different: "Turtle is an HTTP proxy whose purpose is to throttle connections to specific hostnames ...." This 'turtle' most certainly does not have "Screen" or anything related to a little drawing turtle. So I ended up with the same error as the user in the question of module has no attribute 'Screen'.
For Ubuntu, what I needed to do was:
sudo pip uninstall turtle
sudo apt-get install python-tk
Then when I did import turtle, all of the expected modules were found.

Go to the directory where you save your python files. There is a file named turtle.py. Either remove it, or rename it. This will work.
thanks,

Probably not related, but I spent some time tracking down this same error and found a different cause: I had a file named "copy.py" in the folder with my project.
This "copy.py" was an assignment to make a function that returns a deep copy of a list. The turtle library imports "deepcopy" from "copy"; turns out there's already a "copy.py" as part of python (which I'd never seen/used) & by having a file named "copy.py" in my project folder, it was causing turtle to import the wrong copy.py, which was causing the error to be thrown in turtle (my copy.py assignment's deepcopy function didn't work the same way as the one in python).
This is a more general suggestion, but it's good to double check and make sure you don't have any filenames that are in conflict with actual python imports used in your project. There are too many to list here, but ones used by turtle include: tkinter, types, math, time, inspect, sys, and copy. If you have any of these with a .py in your folder (for example, if you had previously created an inspect.py), turtle will be loading that instead of the built-in library & will not work.

Evt.TickCount() not found with Python2.6 on OSX 10.6.3

With Python2.6, the Evt module (from Carbon import Evt) does not have seem to respond to TickCount() on OSX. But Python2.5 is fine:
from Carbon import Evt
s = Evt.TickCount()
On Python2.5 I get a returned integer. On Python2.6 I get:
AttributeError: 'module' object has no attribute 'TickCount'
This is on Snow Leopard. Is there some library that needs to be updated on OSX to allow for TickCount() to work? I'm actually having this problem due to using py2app.
Update for Barry's answer:
The problem is that the application that py2app creates, when launched, gives me:
File "/Users/cybertoast/Projects/scripts/dist/fixcatalystlibs.app/Contents/Resources/__boot__.py", line 40, in mainloop
[0x0-0x913913].org.pythonmac.unspecified.fixcatalystlibs[11722] stoptime = Evt.TickCount() + timeout
[0x0-0x913913].org.pythonmac.unspecified.fixcatalystlibs[11722] AttributeError: 'module' object has no attribute 'TickCount'
I added VERSIONER_PYTHON_PREFER_32_BIT=yes to my .bash_profile, but the app that py2app creates still has the same problem. The python interpreter, however is happy with the 32-bit fix. But still need a solution to py2app.

Since the broken code was only being used for argv emulation, I was able to get my app working by disabling argv emulation. This can be done by setting the 'argv_emulation' key in the py2app OPTIONS hash to False.
Example:
APP = ['MyApp.py']
OPTIONS = {
'argv_emulation': False,
...
}
setup(
app=APP,
options={'py2app': OPTIONS},
setup_requires=['py2app'],
)

The code I have been using requires the use of argv_emulation, so the above solutions have not aided me. In the end, I made use of LSArchitecturePriority in the plist file, setting it to i386. This solved my problem, and allowed me to keep the argv_emulation set to True.
More info can be found here: http://developer.apple.com/library/mac/#documentation/General/Reference/InfoPlistKeyReference/Articles/LaunchServicesKeys.html#/

Python runs in 64-bit mode by default in Snow Leopard. It appears that Carbon.Evt hasn't made the transition to full 64-bit compatibility. You can confirm this by trying to run in 32-bit mode (see man python):
oso:~ barry$ export VERSIONER_PYTHON_PREFER_32_BIT=yes
oso:~ barry$ python
Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import Carbon.Evt as evt
>>> evt.TickCount()
2973070
>>>
So, either run py2app in 32-bit mode, or tell us what you're trying to do and perhaps we can provide a 64-bit compatible alternative.

Probably the easiest solution is to use another, 32-bit-only Python instead of the Apple-supplied one in 10.6 - for example, install Python 2.6 using the python.org installer. If you want to distribute your app as a standalone app that can be used on multiple OS X versions, you'll need to do that anyway.