We Keep Coding

Python for a hobbyist programmer ( a few questions)

I'm a hobbyist programmer (only in TI-Basic before now), and after much, much, much debating with myself, I've decided to learn Python. I don't have a ton of free time to teach myself a hundred languages and all programming I do will be for personal use or for distributing to people who need them, so I decided that I needed one good, strong language to be good at. My questions:
Is python powerful enough to handle most things that a typical programmer might do in his off-time? I have in mind things like complex stat generators based on user input for tabletop games, making small games, automate install processes, and build interactive websites, but probably a hundred things along those lines
Does python handle networking tasks fairly well?
Can python source be obfuscated, or is it going to be open-source by nature? The reason I ask this is because if I make something cool and distribute it, I don't want some idiot script kiddie to edit his own name in and say he wrote it
And how popular is python, compared to other languages. Ideally, my language would be good and useful with help found online without extreme difficulty, but not so common that every idiot with computer knows python. I like the idea of knowing a slightly obscure language.
Thanks a ton for any help you can provide.

Is python powerful enough to handle
most things?
Yes. Period. Study EveOnline game for more information. Look at pygame framework. Free free to use Google to find more.
Does python handle networking tasks
fairly well?
Yes. Look at the number of Python web frameworks plus the Twisted framework. Feel free to use Google to search for Python networking.
Can python source be obfuscated?
Not usefully. This isn't C.
And how popular is python, compared to
other languages?
Look at the TIOBE index.

I think that Python is very powerful to do a lot of things, but just like Java and C++, it often depends on good third-party libraries. I come from a Java background but use Python for a lot of things, and it's been a fun ride. I've done things like statistics, and automation, not sure about the UI though that often depends on the toolkit more than the language.
Python networking works well. I don't know if I'd use it to build a fast algorithmic trading system or a VOIP application, but for most intents and purposes, especially at higher levels of abstraction, it's fine and easy to use. You would need external libraries for things like SSH or FTP.
Python is quite popular and has very good online support, active community, and major corporations (likeGoogle) that use it. I found the official online tutorial and reference to be excellent.
I have to say that I disagree with the "every idiot with a computer" line. There's a difference between knowing a language and using it right, and that's true about every language, even natural ones :) Python does have a lot of functional elements that are not as trivial to use for people coming from a procedural background, so there's always room for growth.
The one problem with Python compared to languages like C and Java is that it is not statically typed. This makes it much faster to write code, but also makes it *much easier) to make mistakes that can be quite nasty to debug. For instance, the same variable can contain a String reference at some point, and a reference to a list of strings at some other point.

Absolutely.
What type of networking? It has socket, http, xml, smtp/pop, telnet, and much more built in.
Python obfuscation won't be nearly as good as a compiled language. Usually that isn't a problem.
It's the 9th most popular tag on stackoverflow, so there's plenty of help available.

Is python powerful enough to handle most things that a typical programmer might do in his off-time? I have in mind things like complex stat generators based on user input for tabletop games, making small games, automate install processes, and build interactive websites, but probably a hundred things along those lines
Definitely. Python is a good tool for all of those except automating install processes, where it might be the right tool but more likely the right tool will likely be decided by what specifically you are automating.
Does python handle networking tasks fairly well?
Yes. You will want to look into Twisted.
Can python source be obfuscated, or is it going to be open-source by nature? The reason I ask this is because if I make something cool and distribute it, I don't want some idiot script kiddie to edit his own name in and say he wrote it
"Open source" refers to the licensing of your code, not the viewability of its source code. Hiding Python source code isn't especially possible, and the results of decompiling Python bytecode will result in much more readable code than the equivalent tools in languages like C. Don't worry about this! You can't prevent people from stealing your car or your computer if they are willing to break the law, and you can't do the same for your code in any language.
And how popular is python, compared to other languages. Ideally, my language would be good and useful with help found online without extreme difficulty, but not so common that every idiot with computer knows python. I like the idea of knowing a slightly obscure language.
This is an unanswerable question. Google will give you lots of conflicting results with different metrics, most of them useful. You're also being a bit silly ;)
As far as learning materials go, I recommend How to Think Like a Computer Scientist, which is a good text that does not presume any existing programming knowledge. It is available for free online, or you could buy a print copy if you prefer. (Don't bother learning 3.x yet. There is not enough library support to do much useful stuff like you want to do, and when there is picking it up will be a breeze; it's not very different than 2.5/6/7.)

Probably yes. Maybe the stat crunching thing will be kinda slow, and maybe a game depending on what kind of game, but generally the performance is good enough, and you save a lot of time on the actual programming. If you REALLY need performance, you can make a module in C, but usually there is a library written to do what you want..
I haven't used it, but there's a framework called Twisted that seems to be pretty good.
No. Bytecode can be decompiled easily, and it only works on a specific version of Python, so your code isn't as portable.
Python is pretty popular, and the Python Package Index has a big list of third party libraries. It's not as widespread as, say, Java, but a lot of people use it and you can probably get answers for what you want.

Points 1 and 2: HELL YEAH.
Point 4: kind of. Python is good at some network stuff. It's not Java or C++. Just use zlib (zip library) and pickle (serialization) for everything, and look at xmlrpclib if you need IPC.
Point 3: No. However, you can write C modules (for the performance critical, and hard-to-copy) parts of your code, and that would make it non-trivial to reverse-engineer.

Python is up to the task (and better) for 1, 2 and 4.
The best solution for 3 from what you describe would probably be to make your programs really open-source with GPL or BSD like licence. This way people will edit your super-cool sources (but often experienced programmers, not just script kiddies) and build on then but leave your name in for posterity.

What are the downsides of using Python instead of Objective-C? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I know some Python and I'm really impressed by the language's ease of use. From what I've seen of Objective-C it looks a lot less pretty, but it seems to be the lingua franca for Mac OS X development (which means it has better documentation).
I'm thinking about starting Mac development - will using PyObjC+Python make me a second class citizen?

Yes.
For one thing, as you note, all the documentation is written for Objective-C, which is a very different language.
One difference is method name. In Objective-C, when you send a message to (Python would say “call a method of”) an object, the method name (selector) and arguments are mixed:
NSURL *URL = /*…*/;
NSError *error = nil;
QTMovie *movie = [QTMovie movieWithURL:URL
error:&error];
This isn't possible in Python. Python's keyword arguments don't count as part of the method name, so if you did this:
movie = QTMovie.movieWithURL(URL, error = ???)
you would get an exception, because the QTMovie class has no method named movieWithURL; the message in the Objective-C example uses the selector movieWithURL:error:. movieWithURL: and movieWithURL would be two other selectors.
There's no way they can change this, because Python's keyword arguments aren't ordered. Suppose you have a hypothetical three-argument method:
foo = Foo.foo(fred, bar=bar, baz=baz)
Now, this calls foo:bar:baz:, right?
Not so fast. Foo may also have a method named foo:baz:bar:. Because Python's keyword arguments aren't ordered, you may actually be calling that method. Likewise, if you tried to call foo:baz:bar:, you may actually end up calling foo:bar:baz:. In reality, this case is unlikely, but if it ever happens, you would be unable to reliably call either method.
So, in PyObjC, you would need to call the method like this:
movie = QTMovie.movieWithURL_error_(URL, ???)
You may be wondering about the ???. C doesn't allow multiple return values, so, in Objective-C, the error: argument takes a pointer to a pointer variable, and the method will store an object in that variable (this is called return-by-reference). Python doesn't have pointers, so the way the bridge handles arguments like this is that you pass None, and the method will (appear to) return a tuple. So the correct example is:
movie, error = QTMovie.movieWithURL_error_(URL, None)
You can see how even a simple example deviates from what documentation might show you in Objective-C.
There are other issues, such as the GIL. Cocoa apps are only going to get more concurrent, and you're going to want in on this, especially with tempting classes like NSOperation lying around. And the GIL is a serious liability, especially on multi-core machines. I say this as a Python guy myself (when not writing for Cocoa). As David Beazley demonstrates in that video, it's a cold, hard fact; there's no denying it.
So, if I were going to switch away from Objective-C for my apps, I would take up MacRuby. Unlike with PyObjC and RubyCocoa, messages to Cocoa objects don't cross the language bridge; it's a from-the-ground-up Ruby implementation in Cocoa, with language extensions to better support writing Cocoa code in it.
But that's too far ahead of you. You're just getting started. Start with Objective-C. Better to avoid all impedance mismatches between the language you're using and the one the documentation is written for by keeping them the same language.
Plus, you'll find some bugs (such as messages to deceased objects) harder to diagnose without knowledge of how Objective-C works. You will write these bugs as a new Cocoa programmer, regardless of which language you're writing the code in.
So, learn C, then learn Objective-C. A working knowledge of both shouldn't take more than a few weeks, and at the end of it, you'll be better prepared for everything else.
I won't go into how I learned C; suffice to say that I do not recommend the way I did it. I've heard that this book is good, but I've never owned nor read it. I do have this book, and can confirm that it's good, but it's also not Mac-specific; skip the chapter on how to compile the code, and use Xcode instead.
As for Objective-C: The Hillegass book is the most popular, but I didn't use it. (I have skimmed it, and it looks good.) I read Apple's document on the language, then jumped right in to writing small Cocoa apps. I read some of the guides, with mixed results. There is a Currency Converter tutorial, but it didn't help me at all, and doesn't quite reflect a modern Cocoa app. (Modern apps still use outlets and actions, but also Bindings, and a realistic Currency Converter would be almost entirely a couple of Bindings.)

This really says it all:
As the maintainer of PyObjC for nearly
15 years, I will say it bluntly. Use
Objective-C. You will need to know
Objective-C to really understand Cocoa
anyway and PyObjC is just going to add
a layer of bugs & issues that are
alien to 99% of Cocoa programmers.
a comment in an answer to this question. This question is also interesting.

DO NOT ATTEMPT to avoid learning objective-C if you're going to write apps for the Mac. The purpose of PyObjC and the other language bindings is to let you re-use existing libraries in your apps, not to let you avoid learning the native tools.

Second class citizen seems a bit strong. The Objective-C API's are available from Python as well, should you need them, and that's mostly if you want to make Cocoa apps. But then they are restricted to OS X anyway. Personally, I have no interest in building apps that isn't cross-platform, but that's me. That also means I haven't actually done this, so I don't know how tricky it is, but there was an article in the Python Magazine not long ago, and it didn't look that horrible.
The major drawback of Python is execution time, and that mainly comes from it being a dynamic language. This can be solved with Cython and C-extensions, etc, but then you get a mix of Python + ObjectiveC API's + Cython which can be daunting.
So it depends a lot of what kinds of applications you are going to make. Something uniquely OSX-ish that makes no sense anywhere else? ObjectiveC is probably the ticket. Cross-platform servers, well then Python rocks! Something else? Then it depends.

This is something I've been wondering myself, and although I hope someone comes by with more experience, from what I know you will not be seriously constrained by Python itself. Along with Java and GCC, Python is an excellent way to write native cross-platform applications. Once you get the hang of it you should be able to map example code in Objective C to your Python code.
Since you have access to all libraries and events, everything that you can do in Objective C will be there in Python. Of course, the more OS X-only calls and functions you use, the less easy it will be to port to another platform, but that's beside the point. Usually graphics programming and working with device drivers is somewhat of a limiting factor - but in both cases I'm finding evidence of good support and community libraries (search for Python and Quartz, Lightblue, libhid, PyUSB, for some examples).
The decisive factor for me would be: what is the level of tooling and IDE support that is needed. Apple provides some great software for building new software, but then again with something like Pydev you've got a great place to write Python code too! http://pydev.org/
So give it a try, I'm sure you won't regret it, and there will be a supportive community to draw on for help and insipiration.

You're going to need Objective-C: that's what all the tutorials, documentation, sample code, and everything is written in. In addition to a wilder variety of people being able to help you.
So learn ObjC first. If, on your second or third project, or a year down the road, you start a project that needs a Python module (like, say, Twisted, or SQLAlchemy. But a SERIOUS need like foundation of your app need, where the extra boost your app gets makes everything worth it), then you can write a PyObjC app and get a lot of the speed benefits of that language, with your background in Cocoa.

Just as an extra option, consider that wxPython can produce some pretty good applications on Mac as well as on Linux and Windows. For the most part you can get native appearance but maintain portability with little or no attention to platform-specific issues.
In other words, PyObjC + Python is not the only way to do Mac development with Python.

No you dont need to know Objective C you dont need to use PyObjC , and you wont be a second class citizent.
Unless you want to do something extremely specific to the MAC platform , coding in Objective C or using PyObjC is a really bad idea.
The reason is obvious, once you go the objc route you say a big "goodbye" to other platforms. Its that simple.
Apple does not want you to code for other platforms the same way Microsoft does not want you to code for other platforms. And that is why more and more developers are turning to open source languages like, python, java, ruby etc. Because you dont care what Apple and Microsot , you only care about an App that is the most useful and most easy to develop. And making your App available only for MAC will make it less useful and obviously developing in Objective C is way more difficult.
Python has more than enough libraries to accomodate you , hundrends of them , readily available for the mac platform. I for instance develope a new application in pygame, no its not a game, if I have done the same thing in ObjC or PyObj I would have to rewrite the code for windows and linux. While with pygame my code works exactly the same in windows and linux even though my main platform is macos.
Thats the appeal of most python libraries , they are cross platform. WxPython is another example, someone mentioned that "it does not exactly look natively" , do you want this to stop you from making your application available for windows and linux. Why limit yourself only on the MAC platform ? Do you think the average user will care how natively your app will look. Even macos apps do not look native , many of them introduce their own "eye candy" gui. Not that you cant make WxPython look 100% native, the way you code is always importnat.
Objc makes sense when you intend to develop for Iphone OS , as Apple thought it a great idea to exclude python (and not only python), even though they were forced to include javascript (or else websurfing would have being a nightmare on iphoneos) . Pyjamas, can make python available for iphone os as well (with no hacks or jailbroken phones), but with the obvious limitations since it translates python code to javascript, but still its a valid solution till Apple decide that excluding python from iphone os is a really bad idea.
link text
There is no harm done in studying Objective C though. You can always use the native libraries via pyobjc.
But to be absolutely sincere with you, If my app reaches a dead end with the python libraries ( a very unlikely scenario) I would rather wrap an existing cross platform C/C++ Libraries with Cython than go the objective c pyobjc route and detroy the cross platform ability of my app. The last thing I would be using is anything platoform specifc.
Now if you dont care about other platforms at all, then I guess Objective C can be a valid choice. It certainly looks ugly as hell, but I have heard that it gets much better the more you use it and there are many people that prefer it over C/C++.

Scripting language choice for initial performance [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I have a small lightweight application that is used as part of a larger solution. Currently it is written in C but I am looking to rewrite it using a cross-platform scripting language. The solution needs to run on Windows, Linux, Solaris, AIX and HP-UX.
The existing C application works fine but I want to have a single script I can maintain for all platforms. At the same time, I do not want to lose a lot of performance but am willing to lose some.
Startup cost of the script is very important. This script can be called anywhere from every minute to many times per second. As a consequence, keeping it's memory and startup time low are important.
So basically I'm looking for the best scripting languages that is:
Cross platform.
Capable of XML parsing and HTTP Posts.
Low memory and low startup time.
Possible choices include but are not limited to: bash/ksh + curl, Perl, Python and Ruby. What would you recommend for this type of a scenario?

Lua is a scripting language that meets your criteria. It's certainly the fastest and lowest memory scripting language available.

Because of your requirement for fast startup time and a calling frequency greater than 1Hz I'd recommend either staying with C and figuring out how to make it portable (not always as easy as a few ifdefs) or exploring the possibility of turning it into a service daemon that is always running. Of course this depends on how
Python can have lower startup times if you compile the module and run the .pyc file, but it is still generally considered slow. Perl, in my experience, in the fastest of the scripting languages so you might have good luck with a perl daemon.
You could also look at cross platform frameworks like gtk, wxWidgets and Qt. While they are targeted at GUIs they do have low level cross platform data types and network libraries that could make the job of using a fast C based application easier.

"called anywhere from every minute to many times per second. As a consequence, keeping it's memory and startup time low are important."
This doesn't sound like a script to me at all.
This sounds like a server handling requests that arrive from every minute to several times a second.
If it's a server, handling requests, start-up time doesn't mean as much as responsiveness. In which case, Python might work out well, and still keep performance up.
Rather than restarting, you're just processing another request. You get to keep as much state as you need to optimize performance.

When written properly, C should be platform independant and would only need a recompile for those different platforms. You might have to jump through some #ifdef hoops for the headers (not all systems use the same headers), but most normal (non-win32 API) calls are very portable.
For web access (which I presume you need as you mention bash+curl), you could take a look at libcurl, it's available for all the platforms you mentioned, and shouldn't be that hard to work with.
With execution time and memory cost in mind, I doubt you could go any faster than properly written C with any scripting language as you would lose at least some time on interpreting the script...

I concur with Lua: it is super-portable, it has XML libraries, either native or by binding C libraries like Expat, it has a good socket library (LuaSocket) plus, for complex stuff, some cURL bindings, and is well known for being very lightweight (often embedded in low memory devices), very fast (one of the fastest scripting languages), and powerful. And very easy to code!
It is coded in pure Ansi C, and lot of people claim it has one of the best C biding API (calling C routines from Lua, calling Lua code from C...).

If Low memory and low startup time are truly important you might want to consider doing the work to keep the C code cross platform, however I have found this is rarely necessary.
Personally I would use Ruby or Python for this type of job, they both make it very easy to make clear understandable code that others can maintain (or you can maintain after not looking at it for 6 months). If you have the control to do so I would also suggest getting the latest version of the interpreter, as both Ruby and Python have made notable improvements around performance recently.
It is a bit of a personal thing. Programming Ruby makes me happy, C code does not (nor bash scripting for anything non-trivial).

As others have suggested, daemonizing your script might be a good idea; that would reduce the startup time to virtually zero. Either have a small C wrapper that connects to your daemon and transmits the request back and forth, or have the daemon handle requests directly.
It's not clear if this is intended to handle HTTP requests; if so, Perl has a good HTTP server module, bindings to several different C-based XML parsers, and blazing fast string support. (If you don't want to daemonize, it has a good, full-featured CGI module; if you have full control over the server it's running on, you could also use mod_perl to implement your script as an Apache handler.) Ruby's strings are a little slower, but there are some really good backgrounding tools available for it. I'm not as familiar with Python, I'm afraid, so I can't really make any recommendations about it.
In general, though, I don't think you're as startup-time-constrained as you think you are. If the script is really being called several times a second, any decent interpreter on any decent operating system will be cached in memory, as will the source code of your script and its modules. Result: the startup times won't be as bad as you might think.
Dagny:~ brent$ time perl -MCGI -e0
real 0m0.610s
user 0m0.036s
sys 0m0.022s
Dagny:~ brent$ time perl -MCGI -e0
real 0m0.026s
user 0m0.020s
sys 0m0.006s
(The parameters to the Perl interpreter load the rather large CGI module and then execute the line of code '0;'.)

Python is good. I would also check out The Computer Languages Benchmarks Game website:
http://shootout.alioth.debian.org/
It might be worth spending a bit of time understanding the benchmarks (including numbers for startup times and memory usage). Lots of languages are compared such as Perl, Python, Lua and Ruby. You can also compare these languages against benchmarks in C.

I agree with others in that you should probably try to make this a more portable C app instead of porting it over to something else since any scripting language is going to introduce significant overhead from a startup perspective, have a much larger memory footprint, and will probably be much slower.
In my experience, Python is the most efficient of the three, followed by Perl and then Ruby with the difference between Perl and Ruby being particularly large in certain areas. If you really want to try porting this to a scripting language, I would put together a prototype in the language you are most comfortable with and see if it comes close to your requirements. If you don't have a preference, start with Python as it is easy to learn and use and if it is too slow with Python, Perl and Ruby probably won't be able to do any better.

Remember that if you choose Python, you can also extend it in C if the performance isn't great. Heck, you could probably even use some of the code you have right now. Just recompile it and wrap it using pyrex.
You can also do this fairly easily in Ruby, and in Perl (albeit with some more difficulty). Don't ask me about ways to do this though.

Can you instead have it be a long-running process and answer http or rpc requests?
This would satisfy the latency requirements in almost any scenario, but I don't know if that would break your memory footprint constraints.

At first sight, it's sounds like over engineering, as a rule of thumb I suggest fixing only when things are broken.
You have an already working application. Apparently you want to want to call the feature provided from few more several sources. It looks like the description of a service to me (maybe easier to maintain).
Finally you also mentioned that this is part of a larger solution, then you may want to reuse the language, facilities of the larger solutions. From the description you gave (xml+http) it seems quite an usual application that can be written in any generalist language (maybe a web container in java?).
Some libraries can help you to make your code portable:
Boost,
Qt
more details may trigger more ideas :)

Port your app to Ruby. If your app is too slow, profile it and rewrite the those parts in C.

Which of these scripting languages is more appropriate for pen-testing? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
First of all, I want to avoid a flame-war on languages. The languages to choose from are Perl, Python and Ruby . I want to mention that I'm comfortable with all of them, but the problem is that I can't focus just on one.
If, for example, I see a cool Perl module, I have to try it out. If I see a nice Python app, I have to know how it's made. If I see a Ruby DSL or some Ruby voodoo, I'm hooked on Ruby for a while.
Right now I'm working as a Java developer, but plan on taking CEH in the near future. My question is: for tool writing and exploit development, which language do you find to be the most appropriate?
Again, I don't want to cause a flame-war or any trouble, I just want honest opinions from scripters that know what they're doing.
One more thing: maybe some of you will ask "Why settle on one language?". To answer this: I would like to choose only one language, in order to try to master it.

You probably want Ruby, because it's the native language for Metasploit, which is the de facto standard open source penetration testing framework. Ruby's going to give you:
Metasploit's framework, opcode and shellcode databases
Metasploit's Ruby lorcon bindings for raw 802.11 work.
Metasploit's KARMA bindings for 802.11 clientside redirection.
Libcurl and net/http for web tool writing.
EventMachine for web proxy and fuzzing work (or RFuzz, which extends the well-known Mongrel webserver).
Metasm for shellcode generation.
Distorm for x86 disassembly.
BinData for binary file format fuzzing.
Second place here goes to Python. There are more pentesting libraries available in Python than in Ruby (but not enough to offset Metasploit). Commercial tools tend to support Python as well --- if you're an Immunity CANVAS or CORE Impact customer, you want Python. Python gives you:
Twisted for network access.
PaiMei for program tracing and programmable debugging.
CANVAS and Impact support.
Dornseif's firewire libraries for remote debugging.
Ready integration with WinDbg for remote Windows kernel debugging (there's still no good answer in Ruby for kernel debugging, which is why I still occasionally use Python).
Peach Fuzzer and Sully for fuzzing.
SpikeProxy for web penetration testing (also, OWASP Pantera).
Unsurprisingly, a lot of web work uses Java tools. The de facto standard web pentest tool is Burp Suite, which is a Java swing app. Both Ruby and Python have Java variants you can use to get access to tools like that. Also, both Ruby and Python offer:
Direct integration with libpcap for raw packet work.
OpenSSL bindings for crypto.
IDA Pro extensions.
Mature (or at least reasonable) C foreign function interfaces for API access.
WxWindows for UI work, and decent web stacks for web UIs.
You're not going to go wrong with either language, though for mainstream pentest work, Metasploit probably edges out all the Python benefits, and at present, for x86 reversing work, Python's superior debugging interfaces edge out all the Ruby benefits.
Also: it's 2008. They're not "scripting languages". They're programming languages. ;)

[Disclaimer: I am primarily a Perl programmer, which may be colouring my judgement. However, I am not a particularly tribal one, and I think on this particular question my argument is reasonably objective.]
Perl was designed to blend seamlessly into the Unix landscape, and that is why it feels so alien to people with a mainly-OO background (particularly the Java school of OOP). For that reason, though, it’s incredibly widely installed on machines with any kind of Unixoid OS, and many vendor system utilities are written in it. Also for the same reason, servers that have neither Python nor Ruby installed are still likely to have Perl on them, again making it important to have some familiarity with. So if your CEH activity includes extensive activity on Unix, you will have to have some amount of familiarity with Perl anyway, and you might as well focus on it.
That said, it is largely a matter of preference. There is not much to differentiate the languages; their expressive power is virtually identical. Some things are a little easier in one of the languages, some a little easier in another.
In terms of libraries I do not know how Ruby and Python compare against each other – I do know that Perl has them beat by a margin. Then again, sometimes (particularly when you’re looking for libraries for common needs) the only effect of that is that you get deluged with choices. And if you are only looking to do things in some particular area which is well covered by libraries for Python or Ruby, the mass of other stuff on CPAN isn’t necessarily an advantage. In niche areas, however, it matters, and you never know what unforeseen need you will eventually have (err, by definition).
For one-liner use on the command line, Python is kind of a non-starter.
In terms of interactive interpreter environment, Perl… uhm… well, you can use the debugger, which is not that great, or you can install one from CPAN, but Perl doesn’t ship a good one itself.
So I think Perl does have a very slight edge for your needs in particular, but only just. If you pick Ruby you’ll probably not be much worse off at all. Python might inconvenience you a little more noticeably, but it too is hardly a bad choice.

I could make an argument for all three :-)
Perl has all of CPAN - giving you a huge advantage in pulling together functionality quickly. It also has a nice flexible testing infrastructure that means you can plug lots of different automated testing styles (including tests in other languages) in the same framework.
Ruby is a lovely language to learn - and lacks some of the cruft in Perl 5. If you're doing web based testing it also has the watir library - which is trez useful (see http://wtr.rubyforge.org/)
Python - nice language and (while it's not to my personal preference) some folk find the way its structured easier to get to grips with.
Any of them (and many others) would be a great language to learn.
Instead of looking at the language - I'd look at your working environment. It's always easier to learn stuff if you have other folk around who are doing similar stuff. If you current dev/testing folk are already focussed on one of the above - I'd go for that. If not, pick the one that would be most applicable/useful to your current working environment. Chat to the rest of your team and see what they think.

That depends on the implementation, if it will be distributed I would go with Java, seeing as you know that, because of its portability. If it is just for internal use, or will be used in semi-controlled environments, then go with whatever you are the most comfortable maintaining, and whichever has the best long-term outlook.
Now to just answer the question, I would go with Perl, but I'm a linux guy so I may be a bit biased in this.

If you plan on using Metasploit for pen-testing and exploit development I would recommend ruby as mentioned previously Metasploit is written in ruby and any exploit/module development you may wish to do will require ruby.
If you will be using Immunity CANVAS for pen testing then for the same reasons I would recommend Python as CANVAS is written in python. Also allot of fuzzing frameworks like Peach and Sulley are written in Python.
I would not recommend Perl as you will find very little tools/scripts/frameworks related to pen testing/fuzzing/exploits/... in Perl.
As your question is "tool writing and exploit development" I would recommend Ruby if you choose Metasploit or python if you choose CANVAS.
hope that helps :)

Speaking as a CEH, learn the CEH material first. This will expose you to a variety of tools and platforms used to mount various kinds of attacks. Once you understand your target well, look into the capabilities of the tools and platforms already available (the previously mentioned metasploit framework is very thorough and robust). How can they be extended to meet your needs? Once you know that, you can compare the capabilities of the languages.
I would also recommend taking a look at the tools available on the BackTrack distro.

All of them should be sufficient for that. Unless you need some library that is only available in one language, I'd let personal preference guide me.

If you're looking for a scripting language that will play well with Java, you might want to look at Groovy. It has the flexibility and power of Perl (closures, built in regexes, associative arrays on every corner) but you can access Java code from it thus you have access to a huge number of libraries, and in particular the rest of the system you're developing.

metasploit is a great framework for penetration testing. It's mainly written in Ruby, so if you know that language well, maybe you can hook in there. However, to use metasploit, you don't need to know any language at all.

If you are interested in CEH, I'd take a look at Grey Hat Python. It shows some stuff that is pretty interesting and related.
That being said, any language should be fine.

Well, what kind of exploits are you thinking about? If you want to write something that needs low level stuff (ptrace, raw sockets, etc.) then you'll need to learn C. But both Perl and Python can be used. The real question is which one suits your style more?
As for toolmaking, Perl has good string-processing abilities, is closer to the system, has good support, but IMHO it's very confusing. I prefer Python: it's a clean, easy to use, easy to learn language with good support (complete language/lib reference, 3rd party libs, etc.). And it's (strictly IMHO) cool.

I'm with tqbf. I've worked with Python and Ruby. Currently I'm working with JRuby. It has all the power of Ruby with access to the Java libraries so if there is something you absolutely need a low-level language to solve you can do so with a high-level language. So far I haven't needed to really use much Java as Ruby has had the ability to do everything I've needed as an API tester.

Is Python good for big software projects (not web based)? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
Right now I'm developing mostly in C/C++, but I wrote some small utilities in Python to automatize some tasks and I really love it as language (especially the productivity).
Except for the performances (a problem that could be sometimes solved thanks to the ease of interfacing Python with C modules), do you think it is proper for production use in the development of stand-alone complex applications (think for example to a word processor or a graphic tool)?
What IDE would you suggest? The IDLE provided with Python is not enough even for small projects in my opinion.

We've used IronPython to build our flagship spreadsheet application (40kloc production code - and it's Python, which IMO means loc per feature is low) at Resolver Systems, so I'd definitely say it's ready for production use of complex apps.
There are two ways in which this might not be a useful answer to you :-)
We're using IronPython, not the more usual CPython. This gives us the huge advantage of being able to use .NET class libraries. I may be setting myself up for flaming here, but I would say that I've never really seen a CPython application that looked "professional" - so having access to the WinForms widget set was a huge win for us. IronPython also gives us the advantage of being able to easily drop into C# if we need a performance boost. (Though to be honest we have never needed to do that. All of our performance problems to date have been because we chose dumb algorithms rather than because the language was slow.) Using C# from IP is much easier than writing a C Extension for CPython.
We're an Extreme Programming shop, so we write tests before we write code. I would not write production code in a dynamic language without writing the tests first; the lack of a compile step needs to be covered by something, and as other people have pointed out, refactoring without it can be tough. (Greg Hewgill's answer suggests he's had the same problem. On the other hand, I don't think I would write - or especially refactor - production code in any language these days without writing the tests first - but YMMV.)
Re: the IDE - we've been pretty much fine with each person using their favourite text editor; if you prefer something a bit more heavyweight then WingIDE is pretty well-regarded.

You'll find mostly two answers to that – the religous one (Yes! Of course! It's the best language ever!) and the other religious one (you gotta be kidding me! Python? No... it's not mature enough). I will maybe skip the last religion (Python?! Use Ruby!). The truth, as always, is far from obvious.
Pros: it's easy, readable, batteries included, has lots of good libraries for pretty much everything. It's expressive and dynamic typing makes it more concise in many cases.
Cons: as a dynamic language, has way worse IDE support (proper syntax completion requires static typing, whether explicit in Java or inferred in SML), its object system is far from perfect (interfaces, anyone?) and it is easy to end up with messy code that has methods returning either int or boolean or object or some sort under unknown circumstances.
My take – I love Python for scripting, automation, tiny webapps and other simple well defined tasks. In my opinion it is by far the best dynamic language on the planet. That said, I would never use it any dynamically typed language to develop an application of substantial size.
Say – it would be fine to use it for Stack Overflow, which has three developers and I guess no more than 30k lines of code. For bigger things – first your development would be super fast, and then once team and codebase grow things are slowing down more than they would with Java or C#. You need to offset lack of compilation time checks by writing more unittests, refactorings get harder cause you never know what your refacoring broke until you run all tests or even the whole big app, etc.
Now – decide on how big your team is going to be and how big the app is supposed to be once it is done. If you have 5 or less people and the target size is roughly Stack Overflow, go ahead, write in Python. You will finish in no time and be happy with good codebase. But if you want to write second Google or Yahoo, you will be much better with C# or Java.
Side-note on C/C++ you have mentioned: if you are not writing performance critical software (say massive parallel raytracer that will run for three months rendering a film) or a very mission critical system (say Mars lander that will fly three years straight and has only one chance to land right or you lose $400mln) do not use it. For web apps, most desktop apps, most apps in general it is not a good choice. You will die debugging pointers and memory allocation in complex business logic.

In my opinion python is more than ready for developing complex applications. I see pythons strength more on the server side than writing graphical clients. But have a look at http://www.resolversystems.com/. They develop a whole spreadsheet in python using the .net ironpython port.
If you are familiar with eclipse have a look at pydev which provides auto-completion and debugging support for python with all the other eclipse goodies like svn support. The guy developing it has just been bought by aptana, so this will be solid choice for the future.
#Marcin
Cons: as a dynamic language, has way
worse IDE support (proper syntax
completion requires static typing,
whether explicit in Java or inferred
in SML),
You are right, that static analysis may not provide full syntax completion for dynamic languages, but I thing pydev gets the job done very well. Further more I have a different development style when programming python. I have always an ipython session open and with one F5 I do not only get the perfect completion from ipython, but object introspection and manipulation as well.
But if you want to write second Google
or Yahoo, you will be much better with
C# or Java.
Google just rewrote jaiku to work on top of App Engine, all in python. And as far as I know they use a lot of python inside google too.

I really like python, it's usually my language of choice these days for small (non-gui) stuff that I do on my own.
However, for some larger Python projects I've tackled, I'm finding that it's not quite the same as programming in say, C++. I was working on a language parser, and needed to represent an AST in Python. This is certainly within the scope of what Python can do, but I had a bit of trouble with some refactoring. I was changing the representation of my AST and changing methods and classes around a lot, and I found I missed the strong typing that would be available to me in a C++ solution. Python's duck typing was almost too flexible and I found myself adding a lot of assert code to try to check my types as the program ran. And then I couldn't really be sure that everything was properly typed unless I had 100% code coverage testing (which I didn't at the time).
Actually, that's another thing that I miss sometimes. It's possible to write syntactically correct code in Python that simply won't run. The compiler is incapable of telling you about it until it actually executes the code, so in infrequently-used code paths such as error handlers you can easily have unseen bugs lurking around. Even code that's as simple as printing an error message with a % format string can fail at runtime because of mismatched types.
I haven't used Python for any GUI stuff so I can't comment on that aspect.

Python is considered (among Python programmers :) to be a great language for rapid prototyping. There's not a lot of extraneous syntax getting in the way of your thought processes, so most of the work you do tends to go into the code. (There's far less idioms required to be involved in writing good Python code than in writing good C++.)
Given this, most Python (CPython) programmers ascribe to the "premature optimization is the root of all evil" philosophy. By writing high-level (and significantly slower) Python code, one can optimize the bottlenecks out using C/C++ bindings when your application is nearing completion. At this point it becomes more clear what your processor-intensive algorithms are through proper profiling. This way, you write most of the code in a very readable and maintainable manner while allowing for speedups down the road. You'll see several Python library modules written in C for this very reason.
Most graphics libraries in Python (i.e. wxPython) are just Python wrappers around C++ libraries anyway, so you're pretty much writing to a C++ backend.
To address your IDE question, SPE (Stani's Python Editor) is a good IDE that I've used and Eclipse with PyDev gets the job done as well. Both are OSS, so they're free to try!
[Edit] #Marcin: Have you had experience writing > 30k LOC in Python? It's also funny that you should mention Google's scalability concerns, since they're Python's biggest supporters! Also a small organization called NASA also uses Python frequently ;) see "One coder and 17,000 Lines of Code Later".

Nothing to add to the other answers, besides that if you choose python you must use something like pylint which nobody mentioned so far.

One way to judge what python is used for is to look at what products use python at the moment. This wikipedia page has a long list including various web frameworks, content management systems, version control systems, desktop apps and IDEs.
As it says here - "Some of the largest projects that use Python are the Zope application server, YouTube, and the original BitTorrent client. Large organizations that make use of Python include Google, Yahoo!, CERN and NASA. ITA uses Python for some of its components."
So in short, yes, it is "proper for production use in the development of stand-alone complex applications". So are many other languages, with various pros and cons. Which is the best language for your particular use case is too subjective to answer, so I won't try, but often the answer will be "the one your developers know best".

Refactoring is inevitable on larger codebases and the lack of static typing makes this much harder in python than in statically typed languages.

And as far as I know they use a lot of python inside google too.
Well i'd hope so, the maker of python still works at google if i'm not mistaken?
As for the use of Python, i think it's a great language for stand-alone apps. It's heavily used in a lot of Linux programs, and there are a few nice widget sets out there to aid in the development of GUI's.

Python is a delight to use. I use it routinely and also write a lot of code for work in C#. There are two drawbacks to writing UI code in Python. one is that there is not a single ui framework that is accepted by the majority of the community. when you write in c# the .NET runtime and class libraries are all meant to work together. With Python every UI library has at's own semantics which are often at odds with the pythonic mindset in which you are trying to write your program. I am not blaming the library writers. I've tried several libraries (wxwidgets, PythonWin[Wrapper around MFC], Tkinter), When doing so I often felt that I was writing code in a language other than Python (despite the fact that it was python) because the libraries aren't exactly pythonic they are a port from another language be it c, c++, tk.
So for me I will write UI code in .NET (for me C#) because of the IDE & the consistency of the libraries. But when I can I will write business logic in python because it is more clear and more fun.

I know I'm probably stating the obvious, but don't forget that the quality of the development team and their familiarity with the technology will have a major impact on your ability to deliver.
If you have a strong team, then it's probably not an issue if they're familiar. But if you have people who are more 9 to 5'rs who aren't familiar with the technology, they will need more support and you'd need to make a call if the productivity gains are worth whatever the cost of that support is.

I had only one python experience, my trash-cli project.
I know that probably some or all problems depends of my inexperience with python.
I found frustrating these things:
the difficult of finding a good IDE for free
the limited support to automatic refactoring
Moreover:
the need of introduce two level of grouping packages and modules confuses me.
it seems to me that there is not a widely adopted code naming convention
it seems to me that there are some standard library APIs docs that are incomplete
the fact that some standard libraries are not fully object oriented annoys me
Although some python coders tell me that they does not have these problems, or they say these are not problems.

Try Django or Pylons, write a simple app with both of them and then decide which one suits you best. There are others (like Turbogears or Werkzeug) but those are the most used.