Is there a way to change python2.x source code to python 3.x manually. I guess using lib2to3 this can be done but I don't know exactly how to do this ?
Yes, porting is what you are looking here.
Porting is a non-trivial task that requires making various decisions about your code. For instance, whether or not you want to maintaing backward compatibility. There is no single, universal solution to porting. The way you port depends on your specific requirements.
The best resource I have found for porting apps from Python 2 to 3 is the wiki page PortingPythonToPy3k. The page contains several approaches to porting as well as a lot of links to resources that are potentially helpful in porting work.
Thanks. Here is the answer I was looking for:
from lib2to3.refactor import RefactoringTool, get_fixers_from_package
"""assume `files` to a be a list of all filenames you want to convert"""
r = RefactoringTool(get_fixers_from_package('lib2to3.fixes'))
r.refactor(files, write=True)
Related
I am learning Python and how to make classes. I was curious how the classes are made inside Python itself! For example, in datetime.py (I find it by googling) I was checking how they used __add__ or __sub__ which is using "if isinstance(other, timedelta):" that was an interesting learning. Also, I am learning what professionally written programs look like.
My question is how can I find the source codes of internal classes and functions inside Python, for example, I am interested to see how they implement add in print(), that can print(1+2) -> 3 and
print('a'+'b') -> ab
The source code for the reference implementation of Python is available here at their GitHub mirror. However, it's worth noting that large parts of the Python language are implemented in C, including the core engine and many of the standard library libraries. Really understanding how everything is implemented under the hood requires a fair amount of C fluency.
ipython is a great tool for exploring how things work. Just add "??" after a function or other callable, and it show the code when possible, ie. when it's "pure python".
Eg:
import this
this??
Python is an open source language which means the source code is available to any interested party. I would suggest looking at the source files on the machine you are using or looking at the CPython Github repo.
print() is a built in module. It is written in C and the source can be viewed in the file bltinmodule.c.
You may also find it useful to learn about the functions available in Python for getting help, like help() (documentation available here). To learn about the print() function you can call:
help(print)
I recommend reading the Beginner's Guide as a starting point for more resources.
I'm just starting a new Python project, and ideally I'd like to offer Python 2 and 3 support from the start, with minimal developmental overhead. My question is, what is the best way of doing this for brand new projects?
I have come across projects that run 2to3, or even 3to2, as part of their installation script. This seems to be a very common way. However, there seems to be several different ways of doing this. I also came across Distribute.
There is also the option of trying to write polyglot Python 2/Python 3 code. Even though this seems like a horrible idea, I have noticed that I tend to write code lately that is more idiomatic as Python 3 code, even though I still run it as Python 2. I have a feeling this only helps my own transition when the day finally arrives, and doesn't do much for offering or at least helping dual support though.
Most of the projects offering dual support that I have seen added Python 3 support late, so I'm especially curious if there is a better way that is more suited for new projects, where you have the benefit of a clean slate.
Thanks!
In my experience, it depends on the kind of project.
If it is a library or very self contained application, a common choice is develop in Python 2.7 avoiding constructs deprecated in Python 3.x as much as possible and resort to automated tests to identify holes left by py2to3 that you will have to fix manually.
On the other side, for real life applications, be prepared to constantly stumble upon libraries that are not ported to py3k yet (sometimes important ones). Most of the time you will have no choice but port the library to Python 3, so if you can afford that, go for it. Usually I can't, that is why I'm not supporting Python 3 for this kind of project (but I struggle to write code that will be easier to port when opportune).
For unicode handling, I found this PyCon 2012 video very informative. The advice is good for both Python 2.x and 3.x: treat every string coming from outside as bytes and convert to unicode as soon as possible and output strings converting to bytes as late as possible. There is another very informative video about date/time handling.
[update]
This is an old answer. As of today (2019) there is no good rationale to start a project using Python 2.x and there are several compelling reasons to port older projects to Python 3.7+ and abandon support for Python 2.x.
In my experience it is better to not to use a library like six; I instead have a single compat.py for each package with just the needed code, not unlike Scott Griffiths's approach. six has also the burden of trying to support long-gone Python versions: the truth is that life is much easier when you accept that Pythons <=2.6 and <=3.2 are gone. In 2.7 there are backported compatibility features such as .view* methods on dicts that work exactly like their non-prefixed versions on Python 3; and Python 3.3 on the other hand supports u prefix on unicode strings again.
Even for very substantial packages, a compat.py module, which allows other code to work unchanged, can be quite short: here's an example from the pika package that my colleagues and I helped to make 2/3 polyglot. Pika is one of those projects that had really messed internals mixing unicode and 8 bit strings with each other, but now we've used it in production on Python 3 for well over 6 months without problems.
Other important thing is to always use the following __future__s when developing:
from __future__ import absolute_import, division, print_function
I recommend against using unicode_literals, because there are some strings that need to be of the type called str on either platform. If you don't use unicode_literals, you can do the following:
b'123' is the 8-bit string literal
'123' is of the type str on both platforms
u'123' is proper unicode text on both platforms
In any case, please do not do 2to3 on installation/package build time; some packages used to do that in the past - pip installing those packages took some seconds on Python 2, but closer to minute on Python 3.
Pick 2 or 3, whichever is your favorite flavor, and make it work really well in that, with unit tests. Then make sure those tests work after running it through py2to3 or py3to2. Better to maintain one version of code.
My personal experience has been that it's easier to write code that works unchanged in both Python 2 and 3, rather than rely on 2to3/3to2 scripts which often can't quite get the translation right.
Maybe my situation is unusual as I'm doing lots with byte types and 2to3 has a hard task converting these, but the convenience of having one code base outweighs the nastiness of having a few hacks in the code.
As a concrete example, my bitstring module was an early convert to Python 3 and the same code is used for Python 2.6/2.7/3.x. The source is over 4000 lines of code and this is this bit I needed to get it to work for the different major versions:
# For Python 2.x/ 3.x coexistence
# Yes this is very very hacky.
try:
xrange
for i in range(256):
BYTE_REVERSAL_DICT[i] = chr(int("{0:08b}".format(i)[::-1], 2))
except NameError:
for i in range(256):
BYTE_REVERSAL_DICT[i] = bytes([int("{0:08b}".format(i)[::-1], 2)])
from io import IOBase as file
xrange = range
basestring = str
OK, that's not pretty, but it means that I can write 99% of the code in good Python 2 style and all the unit tests still pass for the same code in Python 3. This route isn't for everyone, but it is an option to consider.
If you need support for Python 2.5 or earlier, using Distribute and it's 2to3 integration is typically the best way to go. But if you only need to support Python 2.6 or later, I would make the code run under Python 2 and Python 3 without conversion. I would also use the six library to make this easier.
I'm given a task of converting a bunch of codes written in Python 2.7 into Python 3.
So my question is
What are the fundamental differences between the two and what are the new features expected from conversion? I'm assuming it's not just syntactical issues.
Where should I start and what should I focus on?
It'll be more helpful if you could be as concrete as possible..
Please help me out and thank you in advance
Definitely start here: http://docs.python.org/py3k/whatsnew/3.0.html
For an automated tool, see: http://docs.python.org/library/2to3.html
Building from Greg's answer I find find it easier to grok the changes by looking at different compatibility layers people have built in order to support 2 and 3 in parallel.
CherryPy, or specifically this file.
Six, or specifically this file.
Pyramid, or specifically this file.
To use a compatibility layer or not is a widely discussed topic, however they are a good programmatic reference too scope the major changes and what you need to do in order to support them.
By far the easiest way is to use 2to3 and maintain two branches concurrently for a while. See this article on the python.org wiki.
There's also an entire website with detailed information, which is basically the contents of a book on the subject.
I am trying to learn Python and referencing the documentation for the standard Python library from the Python website, and I was wondering if this was really the only library and documentation I will need or is there more? I do not plan to program advanced 3d graphics or anything advanced at the moment.
Edit:
Thanks very much for the responses, they were very useful. My problem is where to start on a script I have been thinking of. I want to write a script that converts images into a web format but I am not completely sure where to begin. Thanks for any more help you can provide.
For the basics, yes, the standard Python library is probably all you'll need. But as you continue programming in Python, eventually you will need some other library for some task -- for instance, I recently needed to generate a tone at a specific, but differing, frequency for an application, and pyAudiere did the job just right.
A lot of the other libraries out there generate their documentation differently from the core Python style -- it's just visually different, the content is the same. Some only have docstrings, and you'll be best off reading them in a console, perhaps.
Regardless of how the other documentation is generated, get used to looking through the Python APIs to find the functions/classes/methods you need. When the time comes for you to use non-core libraries, you'll know what you want to do, but you'll have to find how to do it.
For the future, it wouldn't hurt to be familiar with C, either. There's a number of Python libraries that are actually just wrappers around C libraries, and the documentation for the Python libraries is just the same as the documentation for the C libraries. PyOpenGL comes to mind, but it's been a while since I've personally used it.
As others have said, it depends on what you're into. The package index at http://pypi.python.org/pypi/ has categories and summaries that are helpful in seeing what other libraries are available for different purposes. (Select "Browse packages" on the left to see the categories.)
One very common library, that should also fit your current needs, is the Python Image Library (PIL).
Note: the latest version is still in beta, and available only at Effbot site.
If you're just beginning, all you'll need to know is the stuff you can get from the Python website. Failing that a quick Google is the fastest way to get (most) Python answers these days.
As you develop your skills and become more advanced, you'll start looking for more exciting things to do, at which point you'll naturally start coming across other libraries (for example, pygame) that you can use for your more advanced projects.
It's very hard to answer this without knowing what you're planning on using Python for. I recommend Dive Into Python as a useful resource for learning Python.
In terms of popular third party frameworks, for web applications there's the Django framework and associated documentation, network stuff there's Twisted ... the list goes on. It really depends on what you're hoping to do!
Assuming that the standard library doesn't provide what we need and we don't have the time, or the knowledge, to implement the code we reuse 3rd party libraries.
This is a common attitude regardless of the programming language.
If there's a chance that someone else ever wanted to do what you want to do, there's a chance that someone created a library for it. A few minutes Googling something like "python image library" will find you what you need, or let you know that someone hasn't created a library for your purposes.
Of course similar questions have been asked in stackoverflow but I don't want to use any third party library like Crypto or something. So I need to generate a ciphertext from a user email and decrypt it back to plaintext. How can I do this in python?
A third-party system is your best bet.
If you really can't/don't want to use a third-party, maybe something simple would suffice.
One of the simpler algorithms is the Tiny Encryption Algorithm (TEA). Here's an example of a Python implementation that you could start with.
Yes, you can.
Read http://www.amk.ca/python/code/crypto.html
You'll find an answer there ;)
You're question is not concrete enough to say more. You may want to read http://en.wikipedia.org/wiki/Cryptography#Modern_cryptography
Cheers,
Tuergeist
Update:
No, you cannot. (with build in functionality due to export restrictions, see http://docs.python.org/library/crypto.html)
But you can, if you're implementing you own algorithm (bad idea).
So, the BEST solution is, to use the extension recommended by python core developers. See post above.
Cheers again.
If what you mean is that you want to roll your own encryption system, you could try using the built-in hmac and hashlib modules. (hashlib is new for 2.5, so if you must use an earlier Python, your hash choices are the older md5 and sha modules.)
If you are opposed to (or are prevented from) installing a third-party library but are OK with using third-party algorithms or even "lightweight" third-party implementations of algorithms (e.g. published Python source code which resides in a single .py file that you can incorporate or import yourself without using setup.py or any other formal installation), then I highly recommend you do so, because these are likely to be better than what you can come up with on your own.
The smallest and user-friendliest of these that I am aware of is known as p3, written by cryptographer Paul Rubin. The original link is no longer active but you can search for it. Googling currently yields a near-exact copy as well as an adaptation for Python 3.
You could also try one of several single-module, pure-Python Rijndael (AES) implementations such as this or this. (Again, links are not guaranteed to be permanent so you may have to do some searching.)