We Keep Coding

Best way to write Python 2 and 3 compatible code using nothing but the standard library

I am trying to make some of my code Python 2 and 3 compatible.
At the moment I am struggling with functions like range/xrange and methods like dict.items/dict.iteritems. Ideally I would like my code to be able to use the former in Python 3.x and the latter in Python 2.x.
Using if/else seems to me to be the easiest way to implement this:
if py >= 3:
for item in array.items()
...
else:
for item in array.iteritems()
However, doing like that results in lots of repeated and ugly code. Is there a better way to do that using only the standard library? Can I just state somewhere at the beginning of the code to always use range/dict.items if py >= 3 and xrange/dict.iteritems if not?
Is it possible to do something like this?
if py < 3:
use xrange as range
I have looked around and I know that several libraries, like six o futurize) are used to solve this issue. However I am working on a server that run only python 2.7 and I am not allowed to install any extra libraries on it. I have some python3 code I would like to use but I also want to maintain only one version of the code.

The simple, "Don't Make Me Think!" solution I use is to start simple scripts with:
#!/usr/bin/env python
# just make sure that Python 3 code runs fine with 2.7+ too ~98% of the time :)
from __future__ import (division, print_function, absolute_import,
unicode_literals)
from builtins import int
try:
from future_builtins import ascii, filter, hex, map, oct, zip
except:
pass
import sys
if sys.version_info.major > 2:
xrange = range
(Extra tip to stop most pep8 linters for unnecessarily yelling at you for this: move last 3 lines inside and at the top of the try block above)
But the only case I use this is basically "shell scripts that were too large and hairy so I quickly rewrote them to Python and I just want them to run under both Python 2 and 3 with 0 dependencies". Please do NOT use this in real application/library code until you know exactly what are the consequences of all the lines above, and if they are enough for your use case.
Also, the "solution" in this case for .iteritems is "just don't use it", ignore memory use optimizations and just always use .items instead - if this matters, it means you're not writing a "0 dependencies simple script" anymore, so just pick Python 3 and code for it (or Python 2 if you need to pretend we're in 2008).
Also, check these resources to get a proper understanding:
http://python-future.org/compatible_idioms.html
http://lucumr.pocoo.org/2011/1/22/forwards-compatible-python/
https://wiki.python.org/moin/PortingToPy3k/BilingualQuickRef
(NOTE: I'm answering this already answered question mainly because the accepted answers roughly translates to "you are stupid and this is dumb" and I find this very rude for an SO answer: no matter how dumb the question, and how "wrong" to actually answer it, a question deserves a real answer._

import sys
if sys.version_info.major > 2:
xrange = range
But as Wim implies, this is basically rewriting six yourself.
And as you can see, six does a lot more that handling range. Just e.g. look at the _moved_attributes list in the six source code.
And while Python comes with "batteries included", its standard library is not and cannot be all-encompassing. Nor is it devoid of flaws.
Sometimes there are better batteries out there, and it would be a waste not to use them. Just compare urllib2 with requests. The latter is much nicer to work with.

I would recommend writing for py2 or py3 in your projects's modules, but not mix them together and not include any sort of 2/3 checks at all. Your program's logic shouldn't have to care about its version of python, except maybe for avoiding functions on builtin objects that conflict.
Instead, import * from your own compatiblity layer that fixes the differences between your framework and use shadowing to make it transparent to your actual project's module.
For instance, in the compatibility module, you can write Roland Smith's substition for range/xrange, and in your other modules you add "from compatibility import *". Doing this, every module can use "xrange" and the compatibility layer will manage the 2/3 differences.
Unfortunately it won't solve existing objects functions such as dict.iteritems; typically you would monkey-patch the dict methods, but it is not possible on builtin types (see https://stackoverflow.com/a/192857/1741414). I can imagine some workarounds:
Function wrappers (essentially sobolevn's answer)
Don't use .items() functions at all; use simple loops on keys and then access the dictionary with those keys:
for key in my_dict:
value = my_dict[key]
# rest of code goes here

I guess you are mixing up array and dict in this case.
If you are restricted in using 3-d party libraries for any reason, so why not like this:
def iterate_items(to_iterate):
if py >= 3:
return to_iterate.items()
else:
return to_iterate.iteritems()
And then use it:
for item in iterate_items(your_dict):
...

import sys
VERSION = float("{}.{}".format(sys.version_info.major, sys.version_info.minor))
And by using this we can write conditional code for desired versions.
if VERSION >= 3.5:
from subprocess import run as _run
else:
from subprocess import call as _run

How to know that the interpreter is Jython or CPython in the code? [duplicate]

This question already has answers here:
Can I detect if my code is running on cPython or Jython?
(5 answers)
Closed 8 years ago.
Is there a way to detect that the interpreter that executes the code is Jython or CPython?
I have another post: Jython does not catch Exceptions. For this case, if I know the interpreter is Jython, I can have different code that should work.
if JYTHON:
sys.path.insert(0, os.path.dirname(__file__))
from utils import *
else:
from .utils import *

There is an official way to do it! :-) Please have a look at
http://docs.python.org/2/library/platform.html#platform.python_implementation
Returns a string identifying the Python implementation. Possible return values are: ‘CPython’, ‘IronPython’, ‘Jython’, ‘PyPy’.
New in version 2.6.
I did not know that before.
Old answer:
There probably is no standardized interface, but you can use some educated guessing, based on e.g. sys.executable (http://docs.python.org/2/library/sys.html#sys.executable), and sys.version. Furthermore, some interpreters for sure provide features that are specific to them, which you can make use of.

I'm not sure this is safe way, but I got a hint from Find full path of the Python interpreter?.
With print sys.executable, I have different results.
context> jython context/context.py
None
context> python context/context.py
/usr/bin/python
So, checking sys.executable might be one way for the checking.

Might not be the best way, but to detect Jython, couldn't you just try to import something from Java?
try:
import java.lang.System
print "Jython!"
except:
print "Not Jython"

Handing options with a comma with python's argparse

I'm writing a python script that acts as a wrapper for GCC. Most of the options that GCC takes are straightforward to handle with argparse but I'm struggling with the "-Wl,option" option. I want it to store everything after the comma so I tried the following:
parser = argparse.ArgumentParser()
parser.add_argument("-Wl,", help="Option to pass to linker.")
known, unknown = parser.parse_known_args()
print(known)
print(unknown)
However, if I run the script as follows:
python foo.py -Wl,foo
I get the following output:
Namespace(E=False, S=False, Wl,=None, c=False, optimization=None, shared=False, target=None)
['-Wl,foo']
which indicates that it didn't recognize the -Wl option.
I could change the add_argument line to read:
parser.add_argument("-W", help="Option to pass to linker.")
This works, storing "l,foo" in the W option, but GCC uses -W for warning flags and I want to keep those separated from the -Wl options.
I could run through the list of unknown args and handle them that way but I was hoping there was a more elegant solution to the problem. Any tips?

You could do something along the lines of a simple .replace(',',', ') or .replace(',',',=') on all the args before passing them to argparse. I think that one of these should do the trick.

You should be able to fix this with a CustomAction. http://pymotw.com/2/argparse/ has a nice example on how to do this.

Python script argument conditional

Is anyone able to tell me how to write a conditional for an argument on a python script? I want it to print "Argument2 Entered" if it is run with a second command line arguments such as:
python script.py argument1 argument2
And print "No second argument" if it is run without command line arguments, like this:
python script.py argument1
Is this possible?

import sys
if len(sys.argv)==2: # first entry in sys.argv is script itself...
print "No second argument"
elif len(sys.argv)==3:
print "Second argument"

There are many answers to this, depending on what exactly you want to do and how much flexibility you are likely to need.
The simplest solution is to examine the variable sys.argv, which is a list containing all of the command-line arguments. (It also contains the name of the script as the first element.) To do this, simply look at len(sys.argv) and change behaviour based on its value.
However, this is often not flexible enough for what people expect command-line programs to do. For example, if you want a flag (-i, --no-defaults, ...) then it's not obvious how to write one with just sys.argv. Likewise for arguments (--dest-dir="downloads"). There are therefore many modules people have written to simplify this sort of argument parsing.
The built-in solution is argparse, which is powerful and pretty easy-to-use but not particularly concise.
A clever solution is plac, which inspects the signature of the main function to try to deduce what the command-line arguments should be.

There are many ways to do this simple thing in Python. If you are interested to know more than I recommend to read this article. BTW I am giving you one solution below:
import click
'''
Prerequisite: # python -m pip install click
run: python main.py ttt yyy
'''
#click.command(context_settings=dict(ignore_unknown_options=True))
#click.argument("argument1")
#click.argument("argument2")
def main(argument1, argument2):
print(f"argument1={argument1} and argument2={argument2}")
if __name__ == '__main__':
main()

Following block should be self explanatory
$ ./first.py second third 4th 5th
5
$ cat first.py
#!/usr/bin/env python
import sys
print (len(sys.argv))

This is related to many other posts depending upon where you are going with this, so I'll put four here:
What's the best way to grab/parse command line arguments passed to a Python script?
Implementing a "[command] [action] [parameter]" style command-line interfaces?
How can I process command line arguments in Python?
How do I format positional argument help using Python's optparse?
But the direct answer to your question from the Python docs:
sys.argv -
The list of command line arguments passed to a Python script. argv[0] is the script name (it is operating system dependent whether this is a full pathname or not). If the command was executed using the -c command line option to the interpreter, argv[0] is set to the string '-c'. If no script name was passed to the Python interpreter, argv[0] is the empty string.
To loop over the standard input, or the list of files given on the command line, see the fileinput module.

kwargs in python executables

I'm trying to create a program that can be called from the command line and use keyword arguments in python 2.6. So far I've tried:
#!/usr/bin/python
def read(foo = 5):
print foo
return 0
if __name__ == '__main__'
read()
When I try to run this from the command line: ./test.py the program prints 5 as expected. Is there a way to use ./test.py foo=6? I want to preserve the keyword arguments.
It seems like a simple question, but I haven't found a good source for this.

python has built in library to help you achieve passing command line arguments to a script
argparse. THe usage is a little different then what you are describing in your question though...
On a basic level you can access all command line arguments by sys.argv, which will be a list of arguments
Sorry should have mentioned the python 2.6 library is called optparse

Something like this?
if __name__ == '__main__':
kwargs = dict(x.split('=', 1) for x in sys.argv[1:])
read(**kwargs)
That said, argparse and optparse are probably going to give you something more robust and more natural for someone used to the commandline. (Not to mention, supporting arguments of types other than string.)
Oh, and if what you're really after is just interactive use of your function, use the interactive interpreter; either python or ipython. You'd need to put the code into a file ending in .py and import it, then you could just call it.

A less usual, but very interesting alternative is docopt: a library that generates an argument parser from the help message that you write for your program (on github).