Related
I am working on a big project that involves a lot of web based and AI work. I am extremely comfortable with Python, though my only concern is with concurrent programming and scaling this project to make it work on clusters. Thus, Clojure for AI and support for Java function calls and bring about concurrent programming.
Is this a good idea to do all the web-based api work with Python and let Clojure take care of most of the concurrent AI work?
Edit:
Let me explain the interaction in detail. Python would be doing most of the dirty work (scraping, image processing, improving the database and all that.) Clojure, if possible, would either deal with the data base or get the data from Python. I except something CPython sort of linking with Python and Clojure.
Edit2:
Might be a foolish question to ask, but this being a rather long term project which will evolve quite a bit and go under several iterations, is Clojure a language here to stay? Is it portable enough?
I built an embarrassingly parallel number-crunching application with a backend in Clojure (on an arbitrary number of machines) and a frontend in Ruby on Rails. I don't particularly like RoR, but this was a zero-budget project at the time and we had a Rails programmer at hand who was willing to work for free.
The Clojure part consisted of (roughly) a controller, number crunching nodes, and a server implementing a JSON-over-HTTP API which was the interface to the Rails web app. The Clojure nodes used RabbitMQ to talk to each other. Because we defined clear APIs between different parts of the application, it was easy to later rewrite the frontend in Clojure (because that better suited our needs).
If you're working on a distributed project with a long life span and continuous development effort, it could make sense to design the application as a number of separate modules that communicate through well defined APIs (json, bson, ... over AMQP, HTTP, ... or a database). That means you can get started quickly using a language you're comfortable with, and rewrite parts in another language at a later stage if necessary.
I can't see a big problem with using Python for the web apps and Clojure for the concurrent data crunching / back end code. I assume you would use something like JSON over http for the communications between the two, which should work fine.
I'd personally use Clojure for both (using e.g. the excellent Noir as a web framework and Korma for the database stuff.), but if as you say your experience is mostly in Python then it probably makes sense to stick with Python from a productivity perspective (in the short term at least).
To answer the questions regarding the future of Clojure:
It's definitely here to stay. It has a very active community and is probably one of the "hottest" JVM languages right now (alongside Scala and Groovy). It seems to be doing particularly well in the big data / analytics space
Clojure has a particular advantage in terms of library support, since it can easily make use of any Java libraries. This is a huge advantage for a new langauge from a practical perspective, since it immediately solves what is usually one of the biggest issues in getting a new language ecosystem off the ground.
Clojure is a new language that is still undergoing quite a lot of development. If you choose to use Clojure, you should be aware that you will need to put in some effort to stay current and keep your code up to date with the latest Clojure versions. I've personally not found this to be an issue, but it may come as a surprise to people used to more "stable" languages like Java.
Clojure is very portable - it will basically run anywhere that you can get a reasonably modern JVM, which is pretty much everywhere nowadays.
If you can build both sides to use Data and Pure(ish) Functions to communicate then this should work very well. wrapping your clojure functions in web services that take and retrun JSON (or more preferably clojure forms) should make them accessible to your Python based front end will no extra fuss.
Of course it's more fun to write it in Clojure all the way through. ;)
If this is a long term project than building clean Functional (as in takes and returns values) interfaces that exchange Data becomes even more important because it will give you the ability to evolve the components independently.
In such scenarios I personally like to start in the below sequence.
Divide the system into subsystems with "very clear" definition of what each system does and that definition should follow the principle of "do one thing and keep it simple". At this stage don't think about language etc.
Choose the platform (not languages) on which these subsystems will run. Ex: JVM, Python VM, NodeJs, CLR(Mono), other VMs. Try to select few platforms or if possible just one as that does make life easier down the road in terms of complexity.
Choose the language to program those platforms. This is very subjective but for JVM you can go with Clojure or Jython (in case you like Dynamic languages as I do).
As far as Clojure future is concerned, this is a language developed by "community of amazing programmers" and not by some corporation. I hope that clears your doubt about the "long term" concern of Clojure. By the way Clojure is LISP, so you can modify the language the way you want it and fix things yourself even if someone don't do that for you.
So this thread is definitely NOT a thread for why Python is better than Ruby or the inverse. Instead, this thread is for objective criticism on why you would pick one over the other to write a RESTful web API that's going to be used by many different clients, (mobile, web browsers, tablets etc).
Again, don't compare Ruby on Rails vs Django. This isn't a web app that's dependent on high level frameworks such as RoR or Django. I'd just like to hear why someone might choose one over the other to write a RESTful web API that they had to start tomorrow, completely from scratch and reasons they might go from one to another.
For me, syntax and language features are completely superfluous. The both offer an abundant amount of features and certainly both can achieve the same exact end goals. I think if someone flips a coin, it's a good enough reason to use one over the other. I'd just love to see what some of you web service experts who are very passionate about their work respond to why they would use one over the other in a very objective format.
I would say the important thing is that regardless of which you choose, make sure that your choice does not leak through your REST API. It should not matter to the client of your API which you chose.
I know Ruby, don't know python... you can see which way I'm leaning toward, right?
Choose the one you're most familiar with and most likely to get things done with the fastest.
Yeah, flip a coin. The truth is that you're going to find minimalist frameworks in either language. Heroku is a pretty strong reason to say Ruby but there may be other similar hosts for Python. But Heroku makes it stupid easy to deploy your api into the cloud whether it's Rails or some other Ruby project that uses Rack. WSGI doesn't give you this option.
As for as the actually implementation though, I'm guessing that you'll find that they're both completely competent languages and both a joy to program in.
I think they are fairly evenly matched in features. I prefer Python, but I have been using it for over a decade so I freely admit that what follows is totally biased.
IMHO Python is more mature - there are more libraries for it (although Ruby may be catching up), and the included libraries I think are better designed. The language evolution process is more mature too, with each proposed feature discussed in public via the PEPs before the decision is made to include them in a release. I get the impression that development of the Ruby language is much more ad-hoc.
Python is widely used in a lot of areas apart from web development - scientific computing, CGI rendering pipelines, distributed computing, Linux GUI tools etc. Ruby got very little attention before Rails came along, so I get the impression that most Ruby work is focused on web development. That may not be a problem if that is all you want to do with the language, but it does mean that Python has a more diverse user base and a more diverse set of libraries.
Python is faster too.
Ruby + Sinatra
Very easy to use with/as rack middleware - someone's already mentioned heroku
Either will do a great job and you'll gain in other ways from learning something new. Why not spend as couple of days with each? See how far you can get with a simple subset of the problem, then see how you feel. For bonus points report back here and answer your own question!
I am a newbie to zope and I previously worked on Django for about 2.5 years. So when I first jumped into Zope(v2) (only because my new company is using it since 7 years), I faced these questions. Please help me in understanding them.
What is the "real" purpose of zodb as such? I know what it does, but tell me one great thing that zodb does and a framework like Django (which doesn't have zodb) misses.
Update: Based on the answers, Zodb replaces the need for ORM. You can directly store the object inside the db(zodb itself).
It is said one of the zope's killer feature is the TTW(Through the Web or Developing using ZMI) philosophy. But I(and any developer) prefers File-System based development(using Version control, using Eclipse, using any favorite tool outside Zope). Then where is this TTW actually used?
This is the big one. What "EXTRA Stuff" does Zope's Acquistion gain when compared to Python/Django Inheritance.
Is it really a good move to come to Zope, from Django ?
Any site like djangosnippets.org for Zope(v2)?
First things first: current zope2 versions include all of zope3, too. And if you look at modern zope2 applications like Plone, you'll see that it uses a lot of "zope 3" (now called the "zope tool kit", ZTK) under the hood.
The real purpose of the ZODB: it is one of the few object databases (as opposed to relational SQL databases) that sees real widespread use. You can "just" store all your python objects in there without needing to use an object-relational mapper. No "select * from xyz" under the hood. And adding a new attribute on a zodb object "just" persists that change. Luxurious! Especially handy when your data cannot be handily mapped to a strict relational database. If you can map it easily: just use such a database, I've used sqlalchemy a few times in zope projects.
TTW: we've come back from that. At least, the zope2 way of TTW indeed has all the drawbacks that you fear. No version control, no outside tools, etc. Plone is experimenting (google for "dexterity") with nice explicit zope 3 ways of doing TTW development that can still be mapped back to the filesystem.
TTW: the zodb makes it easy and cheap to store all sorts of config settings in the database, so you can typically adjust a lot of things through the browser. This doesn't really count as typical TTW development, though.
Acquisition: handy trick, though it leads to a huge namespace polution. Double edged sword. To improve debuggability and maintenance we try to do without in most of the cases. The acquisition happens inside the "object graph", so think "folder structure inside the zope site". A call to "contact_form" three folders down can still find the "contact_form" on the root of the site if it isn't found somewhere in between. Double edged sword!
(And regular python object oriented inheritance happens all over the place of course).
Moving from django to zope: a really good idea for certain problems and nonsensical for other problems :-) Quite a lot of zope2/plone companies have actually done some django projects for specific projects, typically those that have 99% percent of their content in a relatively straightforward SQL database. If you're more into content management, zope (and plone) is probably better.
Additional tip: don't focus only on zope2. Zope3's "component architecture" has lots of functionality for creating bigger applications (also non-web). Look at grok (http://grok.zope.org) for a friendly packaged zope, for instance. The pure component architecture is also usable inside django projects.
On the ZODB:
Another way to ask "What is the real purpose of the ZODB?" is to ask, "Why was the ZODB originally created?"
The answer to that is the project was started very early on, around 1996. This was before the existance of MySQL or PostgreSQL, when miniSQL (a free-to-use but not free software) database was still in common use, or big money databases such as Oracle. Python provided the pickle module to serialize Python objects to disk - but serialization is lower level, it doesn't allow for features such as transactions, concurrent writes, and replication. This is what the ZODB provides.
It's still in use today in Zope because it works well. If you have no existing skillset in realational databases, it's easier to learn to use the ZODB than a relational database. It's also usable simpler use-cases, for example if you have a command-line script that needs to store some configuration information, using a relational database means having to run a database server just to store a little bit of configuraiton. You could use a config file, but the ZODB also works quite nicely because it's an embedable database. That means that the database is running in the same process as the rest of your Python code.
It's also worth noting that the API used to store objects inside containers is different between Zope 2 and Zope 3. In Zope 2, containers are stored as attributes:
root.mycontainer.myattr
In Zope 3, they use the same interface as Python standard dictionary type:
root['mycontainer']myattr
This is another reason why it can be easier to learn to use the ZODB than the Django ORM, since Django has it's own interface for it's ORM which is distinct from Python's existing interfaces.
Through-the-web (TTW):
Again, understanding the reason for TTW goes back when Zope was developed. While it seems silly to break with well known developer tools such Subversion or Mercurial, Zope was developed in the late 90s when the only free version control system was CVS. Zope 2 had it's own simple version control capabilites, and they were as good as CVS (which is to say, "they were limited and sucky."). UNIX workstations cost a lot more money back then, and had far fewer resources, so System Administrators were much more guarded and careful about how servers were managed. TTW allowed people who might not normally be able to upload code to the server with sysadmin intervation a way to do that.
With text editors, emacs and vi have had ftp-modes, and Zope 2 can listen on an FTP port. This would allow you to develop so that code was stored in the ZODB (editable TTW), but it was common to edit this code using a emacs or vi.
Today in Zope, TTW is more rarely used or promoted since it no longer makes sense to do this. Disk space is cheap, servers are (relatively) cheap, and there are lots of developer tools which expect to interact with the standard filesystem.
Acquisition:
It was a mistake. It was a very confusing feature that caused lots of unexpected things to happen. In theory there are some interesting ideas to acquisition, but in practice it's best tossed in the bin and has little practical use.
Moving from Django to Zope:
Work started on Zope 3 in 2001. This fixed a lot of the problems with Zope 2. It's a testament to the Zope community that Zope 2 is still actively and well maintained, but it's hardly state-of-the-art. Zope 2 is really only interesting to learn from a historical perspective.
Zope 3 ended up getting evolved in a few different directions, and so modern incarnations of Zope are best expressed in the form of Grok, BFG or Bobo.
Grok is closest to Zope 3, and as such is a pretty large framework - it can be rather overwhelming at times when delving through it's code base. However, just like Django, or any other full-stack framework you don't need to use every part of Grok, it can be quite easy to learn the basic and create web applications with it. It's convention-over-configuration is second to none, and it's class-based Views give it a much tighter, arguably cleaner code base than a Django web application. It's URL routing system is extremely flexible, but also arguably over-engineered.
BFG is a "pay for only what you eat" framework written by long time Zope developer Chris McDonough. As such, it's closer to Pylons in spirit, where only the parts deemed core or essential to a framework are included. It also plays very well with WSGI. It only uses a few core Zope packages.
Bobo is a "micro-framework". It's just a way to route URLs and serve up an app. It doesn't use any Zope packages, so isn't strictly in the Zope family of web frameworks. But it was written by Zope's creator, Jim Fulton, who originally called the publishing part of Zope, "Bobo". The original Bobo, written in the early 90's, mapped URLs to packages and modules, so if your source code was layed out as:
mypackage.mymodule.MyClass
You could have a URL such as:
/mypackage/mymodule/MyClass
Which was very inflexible, and was replaced with URL Traversel in Zope 2, which is fairly complex. Bobo uses Routes, so it's a middle ground between dead-simple URL resolution and complex URL resolution - about the same in complexity as Django's URL resolution machinery.
I answer without much experience on both, but I had the chance to manipulate both, so I can tell you my opinion on some of your questions.
1)What is the "real" purpose of zodb
as such? Meaning I know what it does,
but tell me one great thing that zodb
does and a framework like django(which
doesn't have zodb) misses
Load distribution via ZEO and search via ZCatalog. Django is very low level on this point of view. To achieve the same, you would have to reimplement a lot of wheels, triangular.
Something I learned quite soon is: don't mess with low level database issues. You will screw them up. It's a can of worms, Dune sized.
So why choose django ORM ? You should also consider if YAGNI. django is easy and self contained, documentation is premium, and when (if) your site will grow that much, you will do the switch to a better ORM (or to a pure OODB, in case of ZODB) later on.
2)It is said one of the zope's killer
feature is the TTW(Through the Web or
Developing using ZMI) philosophy. But
I(and any developer) prefers
File-System based development(using
Version control, using Eclipse, using
any favorite tool outside Zope). Then
where is this TTW actually used?
I cannot answer properly to this question, but I would not say that it's fundamentally bad to develop with such approach. Of course it's a change of mindset, and I tend to prefer filesystem based development as well.
4)Is it really a good move to work on
Zope, from Django ?
Zope 3 is very modular, so you are free to use many of its components from django. I would advise against it though. You can, of course, but what I found most problematic is the lack of help. There are not many people using zope components and django at the same time. Sooner or later, you will have a problem and google won't help. At that point, you will realize that if your life was a videogame, you are definitely playing it at level difficult (maybe extreme, if you will have to put your nose into the zope code).
A very good reference on ZODB is ZODB/ZEO programmer's guide. ZODB is not an ORM. Its a true object database. Python objects are persisted inside the database transparently without any worries about how to transform them into a representation suitable for database. Any pickleable Python object can be saved inside the ZODB. Relational databases are suitable for large amount of flat data (like employee records) while ZODB is best for hierarchical data (typically found in web applications). I personally use Zope 3 for my applications. I never did TTW type of work. Best part of using ZODB was the fact that I never had to worry at all about how I am going to save data and how things would change when I upgrade my software from one version to next one. For example, if I add a new attribute to a Python class, all I have to do is provide a default value as a class attribute. It then becomes automatically available to all objects created with the previous version of the same class. Removing an attribute is a simple del operation on existing objects. BTW, ZODB can be used independently in any kind of Python application and isn't coupled with just ZOPE platform. I love the fact that I don't have to worry about the nitty gritties of SQL while working on Python applications thanx to ZODB. And off course if you need a database server so that you can run multiple copies of your application backed by the same server ZEO comes to your rescue on top of ZODB.
Zope started with the idea of being an Object Publishing Environment. From that perspective mapping the URL directly to the object hierarchy in ZODB was great. The URLs simply reflect the hierarchy of objects. Now so far as figuring out the URL is considered, there is always the Rotterdam debugging interface for help. For development work, I keep the development flags on in the zope configuration and look at the contents of ZODB through the Rotterdam interface. Rotterdam skin provide a great way of introspecting the Python objects stored inside the ZODB and figuring out the URLs is much more interactive. Moreover, for major containers inside my ZODB, I register them as persistent utilities inside the site manager (Zope 3 sites and site managers). Anywhere in my code, whenever I need access to such containers, all I do is getUtility(IMyContainerType). I don't even have to remember the detailed locations of those containers inside the code base. They are once registered with the site manager and going forward available anywhere inside the code base through getUtility() calls.
And the URLs also support namespaces. For example using the ++skin++ namespace, you can anytime change the skin of your web application. Using the ++language++ namespace, you can any time change the preferred language of your user interface. Using the ++attributes++ namespace you can access individual attributes of an object. URLs are simply much more powerful and much more customizable. And you can write traversal adapters, define your own namespaces, to enhance the capabilities of your URLs. To give an example, all pages which are directly accessible from the web interface, are part of my default skin. While all pages which are invoked through background AJAX calls, are under a different skin. This way, one can implement different ways of authentication mechanisms in different skins. In main skin, one is redirected to a different login page in case of authentication failure. For AJAX pages, one could simply receive an HTTP error. This could be centrally done. Zope 3 objects have interfaces and one view can be defined for multiple interfaces. Wherever you have an object which supports the given interface, all associated views become automatically available and all such URLs are automatically valid. If you think about it, its a much more powerful than a single python file or XML file where the URLs are hard-coded. I don't really know much about DJango and J2EE so cannot say if they have equivalent capability.
ZODB is a OO-style database that doesn't need a schema definition. You can simply create (nearly) all kinds of objects, and persist them.
The TTW is sometimes annoying, but you can mount the ZOPE-object-tree using webdav. Then you can edit the templates and scripts using your favorite editor.
ZOPE is especially powerful for creating CMS-like systems, IMHO there it is still unmatched - you'd have to go through a lot to make it work equally well in Django.
And through the TTW, actually non-developers like designers have a good chance of developing e.g. templates and CSS without need for developer interaction.
+1 on Wheat's answer, above: "Zope 2 is really only interesting to learn from a historical perspective". I did Zope dev for a large site for a couple of years, 50% zope 2, 50% zope 3. Even then (this was 2 years ago) we were working to migrate everything off of zope 2. Unless you already have a lot invested in an existing Zope 2 project, there's no reason to use it; there's just not much of future there. And if you do have a big existing zope 2 project, I'd suggest taking a look at a product caled Five (a joke: 2 + 3 = 5) that aims to
allow you to integrate Zope 3
technologies into Zope 2. Among
others, it allows you to use Zope 3
interfaces, ZCML-based configuration,
adapters, browser pages (including
skins, layers, and resources),
automated add and edit forms based on
schemas, object events, as well as
Zope 3-style i18n message catalogs.
When all is said and done, Zope 3 is a very different framework from 2, and IMHO, a much better (albeit more complicated) one. TTW is optional, and not recommended for most cases. Implicit acquisition is gone.
Looks like people here have covered why you might want to use the ZODB, so I thought I'd mention one other thing about Zope 3 (or Zope 2 using Five) that's good. Zope has a very powerful system for wiring together different application components called the Zope Component Architecture (ZCA). It allows you to write components that are more or less autonomous and reusable, and which can be plugged together in a standardized way. I mostly do Django development now and I sometimes find myself missing the ZCA. In Django, the ability to write reusable components is limited and kind of ad-hoc. But, like Reinout says zope.component (like most zope packages, including the ZODB) works outside of the zope framework and could be used in a Django project.
That said, the ZCA has its drawbacks, one of which is the tedious process of registering your components in XML files; it always felt a little Java-esqe to me. One reason I really like Grok http://grok.zope.org/ is that it sits on top of zope.component and does much of that grunt work for you.
So bottom line: Zope 2 is mostly a dead end. If your employer is amenable to it, start looking at Zope 3, or at least Five. I think you'll find Zope 3 has a steep learning curve compared to Django, so it might be a good idea to come at it via Grok, which smooths out a lot of Zope 3's rougher edges. But, I think for a really large or complex web application with lots of moving parts, I'd go for Zope over Django (and I say this as someone who really likes Django a lot). For smaller projects, Django would probably be faster. Quantifying "large" and "small" in this context is hard though, and would probably require a couple of thousand more words. If you really are interested in Zope 3, the book by Philipp von Weitershausen is definitely the place to start.
I know I'll get a thousand "Depends on what you're trying to do" answers, but seriously, there really is no solid information about this online yet. Here are my assumptions - I think they're similar for alot of people right now:
It is now October 2008. I want to start writing an application for January 2009. I am willing to use beta code and such but by January, I'd like a site that doesn't have 'strange' problems. With that said, if a language is simply 10% slower than another, I don't care about those things as long as the issue is linear. My main concern is developer productivity.
I'll be using Linux, Apache, MySQL for the application.
I want the power to do things like run scp and ftp client functions with stable libraries (I only picked those two because they're not web-related but at the same time represent pretty common network protocols that any larger app might use). Technologies like OpenID and Oauth will be used as well.
Experienced web developers are readily available (i.e. I don't have to find people from financial companies and such).
Whatever the choice is is common and will be around for a while.
Here's a kicker. I'd like to be able to use advanced presentation layer tools/languages similar to HAML, SASS. I definitively want to use JQuery.
I will be creating a Facebook app and at some point doing things like dealing with SMS messages, iPhone apps, etc...
At this point, the choices for language are PHP (Cake,Symfony,Zend), Python (Django), Ruby (Merb). I'm really between Django and Merb at this point mostly because everybody else seems to be going that way.
Please don't put any technologies in here that aren't made for mainstream. I know Merb is untested mostly, but their stated goal is a solid platform and it has alot of momentum behind it so I'm confident that it's workable. Please don't answer with how great Perl is or .Net.
For Future References - these choices were already made:
Debian (Lenny) - For converting CPU cycles into something useful. Trac
0.11 - For Project Management Gliffy - For wireframes and such
Google Docs/Apps - For documentation, hosted email, etc...
Amazon ec2/S3 - For hosting, storage.
Cheers,
Adam
Django!
Look up the DjangoCon talks on Google/Youtube - Especially "Reusable Apps" (www.youtube.com/watch?v=A-S0tqpPga4)
I've been using Django for some time, after starting with Ruby/Rails. I found the Django Community easier to get into (nicer), the language documented with excellent examples, and it's modularity is awesome, especially if you're wanting to throw custom components into the mix, and not be forced to use certain things here and there.
I'm sure there are probably ways to be just as flexible with Rails or some such, but I highly encourage you to take a long look at the Django introductions, etc, at http://www.djangoproject.com/
Eugene mentioned it's now at 1.0 - and therefore will remain a stable and backward-compatible codebase well through January 2009.
Also, the automatic admin interfaces it builds are production ready, and extremely flexible.
Sorry, but your question is wrong. People are probably going to vote me down for this one but I want to say it anyway:
I wouldn't expect to get an objective answer! Why? That's simple:
All Ruby advocates will tell to use Ruby.
All Python advocates will tell to use Python.
All PHP advocates will tell to use PHP.
Insert additional languages here.
Got the idea?
I recommend you to try each of the languages you mentioned for yourself. At least a few days each. Afterwards you should have a much better foundation to make your final decision.
That said, I would choose Ruby (because I am a Ruby advocate).
All of them will get the job done.
Use the one that you and your team are most familiar with
This will have a far greater impact on the delivery times and stability of your app than any of the other variables.
it depends.
php - symfony is a great framework. downsides: php, wordy and directory heavy. propel gets annoying to use. upsides: php is everywhere and labor is cheap. well done framework, and good support. lots of plugins to make your life easier
python - django is also a great framework. downsides: python programmers can be harder to find, django even harder. changing your db schema can be somewhat difficult since there are no official migrations. doesn't quite do mvc like you'd expect. upsides: does everything you need and has the great python std library and community behind it.
ruby - i've never used merb, so I'll address rails. upsides: there is a plugin, gem, or recipe for almost anything you could want to do. easy to use. downsides: those plugins, gems, and recipes sometimes fail to work in mysterious ways. monkey patching is often evil. the community is.. vocal. opinionated software, and sometimes those opinions are wrong (lack of foreign keys). rails itself seems like a tower of cards waiting to explode and take hours of your life away.
with all of that said, I'm a freelance php/symfony and ruby/rails developer. I've worked on several projects in both languages and frameworks. My latest project is in Rails solely because of ActiveMerchant. I've been looking for a reason to develop a django app for a while. If there were an ActiveMerchant like library for django, I probably would have used it.
I would go with Django, if you are comfortable with a Python solution. It's at version 1.0 now, and is maturing nicely, with a large user base and many contributors. Integrating jQuery is no problem, and I've done it without any issues.
The only thing is, as far as I can tell, Ruby is much more popular for web development nowadays, so it's easier to find Ruby developers. I get this impression from browsing recent job advertisements - there aren't that many for Python or Django. I don't know much about Merb, so I can't give a fair comparison.
I've done enough PHP to not recommend starting a new project with it.
Based in your reasons, I would go with Ruby. I see that you want some administration tools (scp, ftp client) and Ruby has it (net/sftp and net/ftp libraries).
Also, there are great gems like God for monitoring your system, Vlad the Deployer for deploying, etc. And a lot of alternatives in Merb's field, just use whatever you find it's better for your needs (Thin, Mongrel, ebb, etc).
To get a feeling of where the Django ecosystem is at currently, you might want to check out
djangopeople.net (try djangopeople.net/us/ny for New York state)
djangogigs.com
I have to preface this with my agreeing with Orion Edwards, choose the one your team is most familiar with.
However, I also have to note the curious lack of ASP.NET languages in your list. Not to provoke the great zealot army, but where's the beef? .NET is a stable, rapid development platform and the labor pool is growing daily. VB.NET and C# are transportable skill sets, and that can mean a lot when you're building a team of developers to work on a diverse set of tasks. .NET also allows you to separate your presentation layer from your backend code, like other languages, but also allows you to expose that backend code as web service for things like your iPhone and Facebook applications.
Take every suggestion with a grain of salt, and pick what suits the application best. Do your research, and design for function and not the zealots.
Disclaimer: Once a PHP, ColdFusion and Perl developer. Flex zealot, and Adobe lover. Now writing enterprise .NET applications. ;)
Don't forget Mono, which will let you run .NET under *nix. Not that I'm saying it will be perfect, just playing devil's advocate.
Don't get stuck in the mindset of server-side page layout. Consider technologies like SproutCore, GWT or ExtJS which put the layouting code fully on the client, making the server responsible only for data marshalling and processing (and easily replaced).
And you really, really need to know which server platform you want. Don't pick one because it's the flavor of the month, pick one because you're comfortable with it. Flavors don't last, a solidly built codebase will.
Having built apps in Django, I can attest to its utility. If only all frameworks were as elegant (yes Spring, I'm looking at you).
However in terms of betting the farm on Django, one thing you need to factor in is that Python 3 will be released shortly. Python 3 is not backwards compatible and there's a risk that it will fork the language and end up slowing momentum for all Python projects while they deal with the fallout. To be fair, Ruby 2.0 is due soon too, but I don't think it will be as disruptive.
My experience with various new technologies over the last ten years leads me to recommend that you make stability of the platform a serious criterion. It's all well and good developing with the latest and greatest framework, but when you find it's moved forward a point version and suddenly the way you have done everything is deprecated, that can turn out to result in extra unnecessary work. This was particularly my experience working with rails a little ahead of version 1. For that reason alone I would avoid any platform that wasn't at least at 1.0 when you start work on it.
Ruby is great to work with and will keep your developer productivity high, but if Django is the more stable platform I would favour that for sure.
It pays not to be biased about your server setup. Any modern web framework worth it's weight in source code has a SQL abstraction layer of some sort. PostgreSQL gets much better performance, and this is coming from a former MySQL partisan.
Apache is a beast, both to configure and on your server's resources. Why not go with something light-weight, like nginx or lighttpd?
(For the record, I'm a big Django user, but as the accepted answer said, go with whatever your team knows. Quick turn-arounds are not the time to be learning new frameworks. If you're hiring a team from scratch, go with Django.)
Update: I ended up using, and loving, Django. I'm totally done with PHP - sorry about that. Future readers trying to create a new web 2.0 site (assuming they have a programming background), should greatly consider this setup:
Amazon ec2 for hosting ($80/month - not cheap but worth it if you can afford it)
Django/Python (Python is the most powerful scripting language on the planet - and Django just makes it work on the web)
Development should be done with SQLlite and the development server that comes with Django. Don't waste time with Nginx, Apache, MySQL until you're withing a few weeks of a beta.
Oh, and I now develop on a Mac, which works great for local Django development.
Finally, Pinax is a great start for Django development.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
Right now I'm developing mostly in C/C++, but I wrote some small utilities in Python to automatize some tasks and I really love it as language (especially the productivity).
Except for the performances (a problem that could be sometimes solved thanks to the ease of interfacing Python with C modules), do you think it is proper for production use in the development of stand-alone complex applications (think for example to a word processor or a graphic tool)?
What IDE would you suggest? The IDLE provided with Python is not enough even for small projects in my opinion.
We've used IronPython to build our flagship spreadsheet application (40kloc production code - and it's Python, which IMO means loc per feature is low) at Resolver Systems, so I'd definitely say it's ready for production use of complex apps.
There are two ways in which this might not be a useful answer to you :-)
We're using IronPython, not the more usual CPython. This gives us the huge advantage of being able to use .NET class libraries. I may be setting myself up for flaming here, but I would say that I've never really seen a CPython application that looked "professional" - so having access to the WinForms widget set was a huge win for us. IronPython also gives us the advantage of being able to easily drop into C# if we need a performance boost. (Though to be honest we have never needed to do that. All of our performance problems to date have been because we chose dumb algorithms rather than because the language was slow.) Using C# from IP is much easier than writing a C Extension for CPython.
We're an Extreme Programming shop, so we write tests before we write code. I would not write production code in a dynamic language without writing the tests first; the lack of a compile step needs to be covered by something, and as other people have pointed out, refactoring without it can be tough. (Greg Hewgill's answer suggests he's had the same problem. On the other hand, I don't think I would write - or especially refactor - production code in any language these days without writing the tests first - but YMMV.)
Re: the IDE - we've been pretty much fine with each person using their favourite text editor; if you prefer something a bit more heavyweight then WingIDE is pretty well-regarded.
You'll find mostly two answers to that – the religous one (Yes! Of course! It's the best language ever!) and the other religious one (you gotta be kidding me! Python? No... it's not mature enough). I will maybe skip the last religion (Python?! Use Ruby!). The truth, as always, is far from obvious.
Pros: it's easy, readable, batteries included, has lots of good libraries for pretty much everything. It's expressive and dynamic typing makes it more concise in many cases.
Cons: as a dynamic language, has way worse IDE support (proper syntax completion requires static typing, whether explicit in Java or inferred in SML), its object system is far from perfect (interfaces, anyone?) and it is easy to end up with messy code that has methods returning either int or boolean or object or some sort under unknown circumstances.
My take – I love Python for scripting, automation, tiny webapps and other simple well defined tasks. In my opinion it is by far the best dynamic language on the planet. That said, I would never use it any dynamically typed language to develop an application of substantial size.
Say – it would be fine to use it for Stack Overflow, which has three developers and I guess no more than 30k lines of code. For bigger things – first your development would be super fast, and then once team and codebase grow things are slowing down more than they would with Java or C#. You need to offset lack of compilation time checks by writing more unittests, refactorings get harder cause you never know what your refacoring broke until you run all tests or even the whole big app, etc.
Now – decide on how big your team is going to be and how big the app is supposed to be once it is done. If you have 5 or less people and the target size is roughly Stack Overflow, go ahead, write in Python. You will finish in no time and be happy with good codebase. But if you want to write second Google or Yahoo, you will be much better with C# or Java.
Side-note on C/C++ you have mentioned: if you are not writing performance critical software (say massive parallel raytracer that will run for three months rendering a film) or a very mission critical system (say Mars lander that will fly three years straight and has only one chance to land right or you lose $400mln) do not use it. For web apps, most desktop apps, most apps in general it is not a good choice. You will die debugging pointers and memory allocation in complex business logic.
In my opinion python is more than ready for developing complex applications. I see pythons strength more on the server side than writing graphical clients. But have a look at http://www.resolversystems.com/. They develop a whole spreadsheet in python using the .net ironpython port.
If you are familiar with eclipse have a look at pydev which provides auto-completion and debugging support for python with all the other eclipse goodies like svn support. The guy developing it has just been bought by aptana, so this will be solid choice for the future.
#Marcin
Cons: as a dynamic language, has way
worse IDE support (proper syntax
completion requires static typing,
whether explicit in Java or inferred
in SML),
You are right, that static analysis may not provide full syntax completion for dynamic languages, but I thing pydev gets the job done very well. Further more I have a different development style when programming python. I have always an ipython session open and with one F5 I do not only get the perfect completion from ipython, but object introspection and manipulation as well.
But if you want to write second Google
or Yahoo, you will be much better with
C# or Java.
Google just rewrote jaiku to work on top of App Engine, all in python. And as far as I know they use a lot of python inside google too.
I really like python, it's usually my language of choice these days for small (non-gui) stuff that I do on my own.
However, for some larger Python projects I've tackled, I'm finding that it's not quite the same as programming in say, C++. I was working on a language parser, and needed to represent an AST in Python. This is certainly within the scope of what Python can do, but I had a bit of trouble with some refactoring. I was changing the representation of my AST and changing methods and classes around a lot, and I found I missed the strong typing that would be available to me in a C++ solution. Python's duck typing was almost too flexible and I found myself adding a lot of assert code to try to check my types as the program ran. And then I couldn't really be sure that everything was properly typed unless I had 100% code coverage testing (which I didn't at the time).
Actually, that's another thing that I miss sometimes. It's possible to write syntactically correct code in Python that simply won't run. The compiler is incapable of telling you about it until it actually executes the code, so in infrequently-used code paths such as error handlers you can easily have unseen bugs lurking around. Even code that's as simple as printing an error message with a % format string can fail at runtime because of mismatched types.
I haven't used Python for any GUI stuff so I can't comment on that aspect.
Python is considered (among Python programmers :) to be a great language for rapid prototyping. There's not a lot of extraneous syntax getting in the way of your thought processes, so most of the work you do tends to go into the code. (There's far less idioms required to be involved in writing good Python code than in writing good C++.)
Given this, most Python (CPython) programmers ascribe to the "premature optimization is the root of all evil" philosophy. By writing high-level (and significantly slower) Python code, one can optimize the bottlenecks out using C/C++ bindings when your application is nearing completion. At this point it becomes more clear what your processor-intensive algorithms are through proper profiling. This way, you write most of the code in a very readable and maintainable manner while allowing for speedups down the road. You'll see several Python library modules written in C for this very reason.
Most graphics libraries in Python (i.e. wxPython) are just Python wrappers around C++ libraries anyway, so you're pretty much writing to a C++ backend.
To address your IDE question, SPE (Stani's Python Editor) is a good IDE that I've used and Eclipse with PyDev gets the job done as well. Both are OSS, so they're free to try!
[Edit] #Marcin: Have you had experience writing > 30k LOC in Python? It's also funny that you should mention Google's scalability concerns, since they're Python's biggest supporters! Also a small organization called NASA also uses Python frequently ;) see "One coder and 17,000 Lines of Code Later".
Nothing to add to the other answers, besides that if you choose python you must use something like pylint which nobody mentioned so far.
One way to judge what python is used for is to look at what products use python at the moment. This wikipedia page has a long list including various web frameworks, content management systems, version control systems, desktop apps and IDEs.
As it says here - "Some of the largest projects that use Python are the Zope application server, YouTube, and the original BitTorrent client. Large organizations that make use of Python include Google, Yahoo!, CERN and NASA. ITA uses Python for some of its components."
So in short, yes, it is "proper for production use in the development of stand-alone complex applications". So are many other languages, with various pros and cons. Which is the best language for your particular use case is too subjective to answer, so I won't try, but often the answer will be "the one your developers know best".
Refactoring is inevitable on larger codebases and the lack of static typing makes this much harder in python than in statically typed languages.
And as far as I know they use a lot of python inside google too.
Well i'd hope so, the maker of python still works at google if i'm not mistaken?
As for the use of Python, i think it's a great language for stand-alone apps. It's heavily used in a lot of Linux programs, and there are a few nice widget sets out there to aid in the development of GUI's.
Python is a delight to use. I use it routinely and also write a lot of code for work in C#. There are two drawbacks to writing UI code in Python. one is that there is not a single ui framework that is accepted by the majority of the community. when you write in c# the .NET runtime and class libraries are all meant to work together. With Python every UI library has at's own semantics which are often at odds with the pythonic mindset in which you are trying to write your program. I am not blaming the library writers. I've tried several libraries (wxwidgets, PythonWin[Wrapper around MFC], Tkinter), When doing so I often felt that I was writing code in a language other than Python (despite the fact that it was python) because the libraries aren't exactly pythonic they are a port from another language be it c, c++, tk.
So for me I will write UI code in .NET (for me C#) because of the IDE & the consistency of the libraries. But when I can I will write business logic in python because it is more clear and more fun.
I know I'm probably stating the obvious, but don't forget that the quality of the development team and their familiarity with the technology will have a major impact on your ability to deliver.
If you have a strong team, then it's probably not an issue if they're familiar. But if you have people who are more 9 to 5'rs who aren't familiar with the technology, they will need more support and you'd need to make a call if the productivity gains are worth whatever the cost of that support is.
I had only one python experience, my trash-cli project.
I know that probably some or all problems depends of my inexperience with python.
I found frustrating these things:
the difficult of finding a good IDE for free
the limited support to automatic refactoring
Moreover:
the need of introduce two level of grouping packages and modules confuses me.
it seems to me that there is not a widely adopted code naming convention
it seems to me that there are some standard library APIs docs that are incomplete
the fact that some standard libraries are not fully object oriented annoys me
Although some python coders tell me that they does not have these problems, or they say these are not problems.
Try Django or Pylons, write a simple app with both of them and then decide which one suits you best. There are others (like Turbogears or Werkzeug) but those are the most used.