We Keep Coding

Converting codes written in Python 2 to Python 3

I'm given a task of converting a bunch of codes written in Python 2.7 into Python 3.
So my question is
What are the fundamental differences between the two and what are the new features expected from conversion? I'm assuming it's not just syntactical issues.
Where should I start and what should I focus on?
It'll be more helpful if you could be as concrete as possible..
Please help me out and thank you in advance

Definitely start here: http://docs.python.org/py3k/whatsnew/3.0.html
For an automated tool, see: http://docs.python.org/library/2to3.html

Building from Greg's answer I find find it easier to grok the changes by looking at different compatibility layers people have built in order to support 2 and 3 in parallel.
CherryPy, or specifically this file.
Six, or specifically this file.
Pyramid, or specifically this file.
To use a compatibility layer or not is a widely discussed topic, however they are a good programmatic reference too scope the major changes and what you need to do in order to support them.

By far the easiest way is to use 2to3 and maintain two branches concurrently for a while. See this article on the python.org wiki.
There's also an entire website with detailed information, which is basically the contents of a book on the subject.

Python for a hobbyist programmer ( a few questions)

I'm a hobbyist programmer (only in TI-Basic before now), and after much, much, much debating with myself, I've decided to learn Python. I don't have a ton of free time to teach myself a hundred languages and all programming I do will be for personal use or for distributing to people who need them, so I decided that I needed one good, strong language to be good at. My questions:
Is python powerful enough to handle most things that a typical programmer might do in his off-time? I have in mind things like complex stat generators based on user input for tabletop games, making small games, automate install processes, and build interactive websites, but probably a hundred things along those lines
Does python handle networking tasks fairly well?
Can python source be obfuscated, or is it going to be open-source by nature? The reason I ask this is because if I make something cool and distribute it, I don't want some idiot script kiddie to edit his own name in and say he wrote it
And how popular is python, compared to other languages. Ideally, my language would be good and useful with help found online without extreme difficulty, but not so common that every idiot with computer knows python. I like the idea of knowing a slightly obscure language.
Thanks a ton for any help you can provide.

Is python powerful enough to handle
most things?
Yes. Period. Study EveOnline game for more information. Look at pygame framework. Free free to use Google to find more.
Does python handle networking tasks
fairly well?
Yes. Look at the number of Python web frameworks plus the Twisted framework. Feel free to use Google to search for Python networking.
Can python source be obfuscated?
Not usefully. This isn't C.
And how popular is python, compared to
other languages?
Look at the TIOBE index.

I think that Python is very powerful to do a lot of things, but just like Java and C++, it often depends on good third-party libraries. I come from a Java background but use Python for a lot of things, and it's been a fun ride. I've done things like statistics, and automation, not sure about the UI though that often depends on the toolkit more than the language.
Python networking works well. I don't know if I'd use it to build a fast algorithmic trading system or a VOIP application, but for most intents and purposes, especially at higher levels of abstraction, it's fine and easy to use. You would need external libraries for things like SSH or FTP.
Python is quite popular and has very good online support, active community, and major corporations (likeGoogle) that use it. I found the official online tutorial and reference to be excellent.
I have to say that I disagree with the "every idiot with a computer" line. There's a difference between knowing a language and using it right, and that's true about every language, even natural ones :) Python does have a lot of functional elements that are not as trivial to use for people coming from a procedural background, so there's always room for growth.
The one problem with Python compared to languages like C and Java is that it is not statically typed. This makes it much faster to write code, but also makes it *much easier) to make mistakes that can be quite nasty to debug. For instance, the same variable can contain a String reference at some point, and a reference to a list of strings at some other point.

Absolutely.
What type of networking? It has socket, http, xml, smtp/pop, telnet, and much more built in.
Python obfuscation won't be nearly as good as a compiled language. Usually that isn't a problem.
It's the 9th most popular tag on stackoverflow, so there's plenty of help available.

Is python powerful enough to handle most things that a typical programmer might do in his off-time? I have in mind things like complex stat generators based on user input for tabletop games, making small games, automate install processes, and build interactive websites, but probably a hundred things along those lines
Definitely. Python is a good tool for all of those except automating install processes, where it might be the right tool but more likely the right tool will likely be decided by what specifically you are automating.
Does python handle networking tasks fairly well?
Yes. You will want to look into Twisted.
Can python source be obfuscated, or is it going to be open-source by nature? The reason I ask this is because if I make something cool and distribute it, I don't want some idiot script kiddie to edit his own name in and say he wrote it
"Open source" refers to the licensing of your code, not the viewability of its source code. Hiding Python source code isn't especially possible, and the results of decompiling Python bytecode will result in much more readable code than the equivalent tools in languages like C. Don't worry about this! You can't prevent people from stealing your car or your computer if they are willing to break the law, and you can't do the same for your code in any language.
And how popular is python, compared to other languages. Ideally, my language would be good and useful with help found online without extreme difficulty, but not so common that every idiot with computer knows python. I like the idea of knowing a slightly obscure language.
This is an unanswerable question. Google will give you lots of conflicting results with different metrics, most of them useful. You're also being a bit silly ;)
As far as learning materials go, I recommend How to Think Like a Computer Scientist, which is a good text that does not presume any existing programming knowledge. It is available for free online, or you could buy a print copy if you prefer. (Don't bother learning 3.x yet. There is not enough library support to do much useful stuff like you want to do, and when there is picking it up will be a breeze; it's not very different than 2.5/6/7.)

Probably yes. Maybe the stat crunching thing will be kinda slow, and maybe a game depending on what kind of game, but generally the performance is good enough, and you save a lot of time on the actual programming. If you REALLY need performance, you can make a module in C, but usually there is a library written to do what you want..
I haven't used it, but there's a framework called Twisted that seems to be pretty good.
No. Bytecode can be decompiled easily, and it only works on a specific version of Python, so your code isn't as portable.
Python is pretty popular, and the Python Package Index has a big list of third party libraries. It's not as widespread as, say, Java, but a lot of people use it and you can probably get answers for what you want.

Points 1 and 2: HELL YEAH.
Point 4: kind of. Python is good at some network stuff. It's not Java or C++. Just use zlib (zip library) and pickle (serialization) for everything, and look at xmlrpclib if you need IPC.
Point 3: No. However, you can write C modules (for the performance critical, and hard-to-copy) parts of your code, and that would make it non-trivial to reverse-engineer.

Python is up to the task (and better) for 1, 2 and 4.
The best solution for 3 from what you describe would probably be to make your programs really open-source with GPL or BSD like licence. This way people will edit your super-cool sources (but often experienced programmers, not just script kiddies) and build on then but leave your name in for posterity.

Does one often use libraries outside the standard ones?

I am trying to learn Python and referencing the documentation for the standard Python library from the Python website, and I was wondering if this was really the only library and documentation I will need or is there more? I do not plan to program advanced 3d graphics or anything advanced at the moment.
Edit:
Thanks very much for the responses, they were very useful. My problem is where to start on a script I have been thinking of. I want to write a script that converts images into a web format but I am not completely sure where to begin. Thanks for any more help you can provide.

For the basics, yes, the standard Python library is probably all you'll need. But as you continue programming in Python, eventually you will need some other library for some task -- for instance, I recently needed to generate a tone at a specific, but differing, frequency for an application, and pyAudiere did the job just right.
A lot of the other libraries out there generate their documentation differently from the core Python style -- it's just visually different, the content is the same. Some only have docstrings, and you'll be best off reading them in a console, perhaps.
Regardless of how the other documentation is generated, get used to looking through the Python APIs to find the functions/classes/methods you need. When the time comes for you to use non-core libraries, you'll know what you want to do, but you'll have to find how to do it.
For the future, it wouldn't hurt to be familiar with C, either. There's a number of Python libraries that are actually just wrappers around C libraries, and the documentation for the Python libraries is just the same as the documentation for the C libraries. PyOpenGL comes to mind, but it's been a while since I've personally used it.

As others have said, it depends on what you're into. The package index at http://pypi.python.org/pypi/ has categories and summaries that are helpful in seeing what other libraries are available for different purposes. (Select "Browse packages" on the left to see the categories.)

One very common library, that should also fit your current needs, is the Python Image Library (PIL).
Note: the latest version is still in beta, and available only at Effbot site.

If you're just beginning, all you'll need to know is the stuff you can get from the Python website. Failing that a quick Google is the fastest way to get (most) Python answers these days.
As you develop your skills and become more advanced, you'll start looking for more exciting things to do, at which point you'll naturally start coming across other libraries (for example, pygame) that you can use for your more advanced projects.

It's very hard to answer this without knowing what you're planning on using Python for. I recommend Dive Into Python as a useful resource for learning Python.
In terms of popular third party frameworks, for web applications there's the Django framework and associated documentation, network stuff there's Twisted ... the list goes on. It really depends on what you're hoping to do!

Assuming that the standard library doesn't provide what we need and we don't have the time, or the knowledge, to implement the code we reuse 3rd party libraries.
This is a common attitude regardless of the programming language.

If there's a chance that someone else ever wanted to do what you want to do, there's a chance that someone created a library for it. A few minutes Googling something like "python image library" will find you what you need, or let you know that someone hasn't created a library for your purposes.

What's the quickest way for a Ruby programmer to pick up Python?

I've been programming Ruby pretty extensively for the past four years or so, and I'm extremely comfortable with the language. For no particular reason, I've decided to learn some Python this week. Is there a specific book, tutorial, or reference that would be well-suited to someone coming from a nearly-identical language, or should I just "Dive into Python"?
Thanks!

A safe bet is to just dive into python (skim through some tutorials that explain the syntax), and then get coding. The best way to learn any new language is to write code, lots of it. Your experience in Ruby will make it easy to pick up python's dynamic concepts (which might be harder to get used to for say a Java programmer).
Try a python tutorial or book on learning python.

After running through some tutorials on-line (the ones posted so far look pretty good), find a current Ruby project you've done (or are working on) and re-write it in Python. I've used this technique to transition from various languages, and it's helped enormously.

I started learning from the python tutorial. It is well written and easy to follow. Then I started to solve problems in python challenge. It was a really fun way to start :)

I suggest just diving into Python, it's similar to Ruby so you should have no problems:
http://www.diveintopython.net/

Is Python good for big software projects (not web based)? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
Right now I'm developing mostly in C/C++, but I wrote some small utilities in Python to automatize some tasks and I really love it as language (especially the productivity).
Except for the performances (a problem that could be sometimes solved thanks to the ease of interfacing Python with C modules), do you think it is proper for production use in the development of stand-alone complex applications (think for example to a word processor or a graphic tool)?
What IDE would you suggest? The IDLE provided with Python is not enough even for small projects in my opinion.

We've used IronPython to build our flagship spreadsheet application (40kloc production code - and it's Python, which IMO means loc per feature is low) at Resolver Systems, so I'd definitely say it's ready for production use of complex apps.
There are two ways in which this might not be a useful answer to you :-)
We're using IronPython, not the more usual CPython. This gives us the huge advantage of being able to use .NET class libraries. I may be setting myself up for flaming here, but I would say that I've never really seen a CPython application that looked "professional" - so having access to the WinForms widget set was a huge win for us. IronPython also gives us the advantage of being able to easily drop into C# if we need a performance boost. (Though to be honest we have never needed to do that. All of our performance problems to date have been because we chose dumb algorithms rather than because the language was slow.) Using C# from IP is much easier than writing a C Extension for CPython.
We're an Extreme Programming shop, so we write tests before we write code. I would not write production code in a dynamic language without writing the tests first; the lack of a compile step needs to be covered by something, and as other people have pointed out, refactoring without it can be tough. (Greg Hewgill's answer suggests he's had the same problem. On the other hand, I don't think I would write - or especially refactor - production code in any language these days without writing the tests first - but YMMV.)
Re: the IDE - we've been pretty much fine with each person using their favourite text editor; if you prefer something a bit more heavyweight then WingIDE is pretty well-regarded.

You'll find mostly two answers to that – the religous one (Yes! Of course! It's the best language ever!) and the other religious one (you gotta be kidding me! Python? No... it's not mature enough). I will maybe skip the last religion (Python?! Use Ruby!). The truth, as always, is far from obvious.
Pros: it's easy, readable, batteries included, has lots of good libraries for pretty much everything. It's expressive and dynamic typing makes it more concise in many cases.
Cons: as a dynamic language, has way worse IDE support (proper syntax completion requires static typing, whether explicit in Java or inferred in SML), its object system is far from perfect (interfaces, anyone?) and it is easy to end up with messy code that has methods returning either int or boolean or object or some sort under unknown circumstances.
My take – I love Python for scripting, automation, tiny webapps and other simple well defined tasks. In my opinion it is by far the best dynamic language on the planet. That said, I would never use it any dynamically typed language to develop an application of substantial size.
Say – it would be fine to use it for Stack Overflow, which has three developers and I guess no more than 30k lines of code. For bigger things – first your development would be super fast, and then once team and codebase grow things are slowing down more than they would with Java or C#. You need to offset lack of compilation time checks by writing more unittests, refactorings get harder cause you never know what your refacoring broke until you run all tests or even the whole big app, etc.
Now – decide on how big your team is going to be and how big the app is supposed to be once it is done. If you have 5 or less people and the target size is roughly Stack Overflow, go ahead, write in Python. You will finish in no time and be happy with good codebase. But if you want to write second Google or Yahoo, you will be much better with C# or Java.
Side-note on C/C++ you have mentioned: if you are not writing performance critical software (say massive parallel raytracer that will run for three months rendering a film) or a very mission critical system (say Mars lander that will fly three years straight and has only one chance to land right or you lose $400mln) do not use it. For web apps, most desktop apps, most apps in general it is not a good choice. You will die debugging pointers and memory allocation in complex business logic.

In my opinion python is more than ready for developing complex applications. I see pythons strength more on the server side than writing graphical clients. But have a look at http://www.resolversystems.com/. They develop a whole spreadsheet in python using the .net ironpython port.
If you are familiar with eclipse have a look at pydev which provides auto-completion and debugging support for python with all the other eclipse goodies like svn support. The guy developing it has just been bought by aptana, so this will be solid choice for the future.
#Marcin
Cons: as a dynamic language, has way
worse IDE support (proper syntax
completion requires static typing,
whether explicit in Java or inferred
in SML),
You are right, that static analysis may not provide full syntax completion for dynamic languages, but I thing pydev gets the job done very well. Further more I have a different development style when programming python. I have always an ipython session open and with one F5 I do not only get the perfect completion from ipython, but object introspection and manipulation as well.
But if you want to write second Google
or Yahoo, you will be much better with
C# or Java.
Google just rewrote jaiku to work on top of App Engine, all in python. And as far as I know they use a lot of python inside google too.

I really like python, it's usually my language of choice these days for small (non-gui) stuff that I do on my own.
However, for some larger Python projects I've tackled, I'm finding that it's not quite the same as programming in say, C++. I was working on a language parser, and needed to represent an AST in Python. This is certainly within the scope of what Python can do, but I had a bit of trouble with some refactoring. I was changing the representation of my AST and changing methods and classes around a lot, and I found I missed the strong typing that would be available to me in a C++ solution. Python's duck typing was almost too flexible and I found myself adding a lot of assert code to try to check my types as the program ran. And then I couldn't really be sure that everything was properly typed unless I had 100% code coverage testing (which I didn't at the time).
Actually, that's another thing that I miss sometimes. It's possible to write syntactically correct code in Python that simply won't run. The compiler is incapable of telling you about it until it actually executes the code, so in infrequently-used code paths such as error handlers you can easily have unseen bugs lurking around. Even code that's as simple as printing an error message with a % format string can fail at runtime because of mismatched types.
I haven't used Python for any GUI stuff so I can't comment on that aspect.

Python is considered (among Python programmers :) to be a great language for rapid prototyping. There's not a lot of extraneous syntax getting in the way of your thought processes, so most of the work you do tends to go into the code. (There's far less idioms required to be involved in writing good Python code than in writing good C++.)
Given this, most Python (CPython) programmers ascribe to the "premature optimization is the root of all evil" philosophy. By writing high-level (and significantly slower) Python code, one can optimize the bottlenecks out using C/C++ bindings when your application is nearing completion. At this point it becomes more clear what your processor-intensive algorithms are through proper profiling. This way, you write most of the code in a very readable and maintainable manner while allowing for speedups down the road. You'll see several Python library modules written in C for this very reason.
Most graphics libraries in Python (i.e. wxPython) are just Python wrappers around C++ libraries anyway, so you're pretty much writing to a C++ backend.
To address your IDE question, SPE (Stani's Python Editor) is a good IDE that I've used and Eclipse with PyDev gets the job done as well. Both are OSS, so they're free to try!
[Edit] #Marcin: Have you had experience writing > 30k LOC in Python? It's also funny that you should mention Google's scalability concerns, since they're Python's biggest supporters! Also a small organization called NASA also uses Python frequently ;) see "One coder and 17,000 Lines of Code Later".

Nothing to add to the other answers, besides that if you choose python you must use something like pylint which nobody mentioned so far.

One way to judge what python is used for is to look at what products use python at the moment. This wikipedia page has a long list including various web frameworks, content management systems, version control systems, desktop apps and IDEs.
As it says here - "Some of the largest projects that use Python are the Zope application server, YouTube, and the original BitTorrent client. Large organizations that make use of Python include Google, Yahoo!, CERN and NASA. ITA uses Python for some of its components."
So in short, yes, it is "proper for production use in the development of stand-alone complex applications". So are many other languages, with various pros and cons. Which is the best language for your particular use case is too subjective to answer, so I won't try, but often the answer will be "the one your developers know best".

Refactoring is inevitable on larger codebases and the lack of static typing makes this much harder in python than in statically typed languages.

And as far as I know they use a lot of python inside google too.
Well i'd hope so, the maker of python still works at google if i'm not mistaken?
As for the use of Python, i think it's a great language for stand-alone apps. It's heavily used in a lot of Linux programs, and there are a few nice widget sets out there to aid in the development of GUI's.

Python is a delight to use. I use it routinely and also write a lot of code for work in C#. There are two drawbacks to writing UI code in Python. one is that there is not a single ui framework that is accepted by the majority of the community. when you write in c# the .NET runtime and class libraries are all meant to work together. With Python every UI library has at's own semantics which are often at odds with the pythonic mindset in which you are trying to write your program. I am not blaming the library writers. I've tried several libraries (wxwidgets, PythonWin[Wrapper around MFC], Tkinter), When doing so I often felt that I was writing code in a language other than Python (despite the fact that it was python) because the libraries aren't exactly pythonic they are a port from another language be it c, c++, tk.
So for me I will write UI code in .NET (for me C#) because of the IDE & the consistency of the libraries. But when I can I will write business logic in python because it is more clear and more fun.

I know I'm probably stating the obvious, but don't forget that the quality of the development team and their familiarity with the technology will have a major impact on your ability to deliver.
If you have a strong team, then it's probably not an issue if they're familiar. But if you have people who are more 9 to 5'rs who aren't familiar with the technology, they will need more support and you'd need to make a call if the productivity gains are worth whatever the cost of that support is.

I had only one python experience, my trash-cli project.
I know that probably some or all problems depends of my inexperience with python.
I found frustrating these things:
the difficult of finding a good IDE for free
the limited support to automatic refactoring
Moreover:
the need of introduce two level of grouping packages and modules confuses me.
it seems to me that there is not a widely adopted code naming convention
it seems to me that there are some standard library APIs docs that are incomplete
the fact that some standard libraries are not fully object oriented annoys me
Although some python coders tell me that they does not have these problems, or they say these are not problems.

Try Django or Pylons, write a simple app with both of them and then decide which one suits you best. There are others (like Turbogears or Werkzeug) but those are the most used.