What are pros and cons of Fabric and Plumbum python libraries for local/remote command execution? What are use cases when one library should be used and other is not? What are the differences attention should be drawn to?
background and suggested comparison methodology
(oops it's a dead post)
Both tools are fun, allow either local or remote work, but have differences in the things they are supposed to solve, i.e. "terminology", and both are basically pretty much obsolete by modern deployment/automation tooling (like ansible, and many others that chose DSL way, e.g. terraform).
Their advantage over more modern ones are lack of "opinionated" approach about the "how", and more on "what".
Suggested comparison criteria:
"Pythonness" vs. "Shellness" (i.e. how "pythonic" the user code with each is)
Special Capabilities
ROI with 2 types of maintainers of your "automation" code (ops vs. devs, let's put "QA" as something in between)
Fabric (my last work was done at 1.8 take this with a grain of time salt):
more pythonic, than shellish, this means easy to support by both old tools and new - i.e. editors, IDEs would be easy to setup
many many context processors, many decorators, very nice
easier to adopt by developers, a bit more traction would come from ops people
Plumbum
The user code can be either pythonic or shellish
"shell combinators" are a killer feature to get senior shell/perl folk onboard, but it uses dynamic imports, so editors/IDEs are a bit trickier to setup.
Due to 1. You will get 'ops' people on board easier, because of mimicking shell constructs in Plumbum, but please install good coding conventions.
Epilogue
Having worked with both toolkits (with lots of fun) and then having switched to ansible - I feel confident to claim - both tools are now superseded by ansible.
you can do most automation tasks with existing ansible modules, and what you can't - you can write a plugin or module for it (in any language), or just call shell module.
My consideration would be this:
if your team of maintainers has good level of programming skills (Esp. in python), as a requirement - you'd be ok with using either fabric, Plumbum (it has more cool hacks ;)) or Ansible.
if you have multi-level multi-team organization, I would simply bet on Ansible - it has lower learning curve, and allows to grow up easily.
Good day.
They're pretty much the same thing. The biggest win for fabric over plumbum is the ability to connect to multiple hosts in parallel, which is more or less indispensible when you're working with a non-trivial setup. fabric also offers a couple of contrib helpers that let you upload jinja templates, upload files, and transfer files back to the local system. I personally find the fabric api to be far more intuitive for working with remote servers.
YMMV, of course, but both are geared towards being very close to shell commands. That said, my team and I are focused on ansible for most configuration / deploy flows. Fabric does offer some power over ansible at the expense of having to roll your own idempotence.
Related
Is it a good idea to use pywebsocket in a production environment, since their google developer page states ...
pywebsocket is intended for testing or experimental purposes.
Moreover what would the specific drawbacks of using it be?
Are there performance drawbacks?
Is it not stable or unsecure in a certain way?
...
Since Mozilla as well as Google use it to test their websocket implementations, and it was suggested (for production) in many SO threads, I thought it to be a pretty stable basis until I read the docs.
Or am I misinterpeting something, and it is just meant to be especially helpful for testing, as well as suitable in production?
After some research it becomes clear, that pywebsockets was developed for testing browser implementations. It has (at least) the drawbacks of being neither secure nor scalable.
So in short: It is not suitable to be used in production environment!
As stated above it is still a great tool for testing your client side implementation of websockets, probably even "the best" one for that, since the websocket implementations of Chrome as well as Firefox,... are being tested with it.
As alternatives for production in python you could look at:
Twisted
Tornado
Gist
websockify
Autobahn
For a list of tools for other programming languages have a look at this SO wiki answer
We have developed a web application running on Linux that is quite popular. We now wish to release it as an appliance so customers can run it internally on their own networks.
We are unsure of the best approach. We are flexible on areas such as: the Linux distro, whether it's a hardware or software only appliance. Does anyone have any advice on the best way to go about this? Links to any good resources on the subject? Questions we should be asking ourselves? Legal considerations for a commercial app? Security considerations?
UPDATE:
It's a Python based web application. We would like the user to be able to do everything via a web interface. No command line stuff etc.
I know when Github needed to do something similar, they contracted with a company who specializes in building installers called BitRock.
If you want to develop the solution yourself, you can't go wrong building a Debian package (or RPM if you prefer). That's what most linux system admins would be comfortable with, and there are very well-known ways to give them a mix of customization/control while also making the process easy to manage on your end. That gives you and your users a very well-known update path as well.
Unless you have had very specific requests from customers, I would shy away from a turnkey-style appliance where you provide hardware. It's extra work for you and could be a turn-off to customers. Different business have different needs though, so maybe your client-base doesn't have IT and would prefer an all-in-one solution. Until you ask, you can't be sure though.
It depends on what langauge/tecnology application is written.
If it is java, release war file + tomcat/jboss.
If it is python, release eggs.
If it is php... not sure, probably just .tar.bz2.
Linux distro or virtual image mught be advantage, but I dislike using them, because they are usually does not fits to my infra (why do I have to install some custom debian-based distro to my rhel-only infrastructure?).
Which framework has the most mature, flexible, intergrated, centralized and easy-to-use plugins/extension system.
My main requirements are:
a centralized system/repository where i could find a extension i need
no need to make changes in the source code, the plugin should be easily enabled and disabled
large plugin/extension database
something like http://wordpress.org/extend/plugins/
http://www.symfony-project.org/plugins/
I can't speak for Django, but I can tell you about Rails' open source community. GitHub is the central location for all Rails open source code.
Most ruby libraries/plugins these days are packaged as "gems", which are easy to install, update, and remove. RubyGems is the place to go for these pre-packaged gems, when you care less about the code and more about dropping the functionality into your application.
There is now a new tool called RVM that keeps the gems (and even rails version) isolated from one application to the next, on your system. That way if one app uses version 1.0 of a gem, and another uses version 2.0, they don't conflict with each other.
All in all, a pretty sweet setup.
There are lots of reusable django apps around. You can find many on the CheeseShop, but even more on GitHub and BitBucket.
There is also django-packages, which is a bit like the CheeseShop, but just for django packages.
VirtualEnv is like RVM (or rather, RVM is like VirtualEnv), which is a great way to isolate your python packages (I even use it in production). It has been around for ages, and works well with pip (the best python package installer).
Both of them are mature frameworks. I don't use ruby so I don't know about the rails plugin land. Given how popular it is (and my information from my lurking time on local Ruby lists), it's pretty good.
With Django, you have (like Matthew mentioned) django-packages and a few other places. I've been working on a largish Django project and it's pretty easy to just search for something like "django facebook" on google and get what you need. The Pinax project is an integrated collection of Django apps that lets you have most things out of the box. That's another thing you might want to consider. The packaging of the plugins are using the standard Python distutils libraries so installation is a single command (or if you're using pip/virtualenv, directly off the net).
VirtualEnv and related tools are not really Django specific. They're good practice if you're doing any python development though.
You should take a step back and evaluate both languages as well in my opinion. Python and Ruby are quite different in their approach to good code and it's likely that one will fit your brain better than the other.
I currently work with .NET exclusively and would like to have a go at python. To this end I need to set up a python development environment. I guide to this would be handy. I guess I would be doing web development so will need a web server and probably a database. I also need pointers to popular ORM's, an MVC framework, and a testing library.
One of my main criteria with all this is that I want to understand how it works, and I want it to be as isolated as possible. This is important as i am wary of polluting what is a working .NET environment with 3rd party web and database servers. I am perfectly happy using SQLite to start with if this is possible.
If I get on well with this I am also likely to want to set up automated build and ci server (On a virtual machine, probably ubuntu). Any suggestions for these would be useful.
My ultimate aim if i like python is to have similar sorts of tools that i have available with .NET and to really understand the build and deployment of it all. To start with I will settle for a simple development environment that is as isolated as possible and will be easy to remove if I don't like it. I don't want to use IronPython as I want the full experience of developing a python solution using the tools and frameworks that are generally used.
It's not that hard to set up a Python environment, and I've never had it muck up my .NET work. Basically, install Python --- I'd use 2.6 rather than 3.0, which is not yet broadly accepted --- and add it to your PATH, and you're ready to go with the language. I wouldn't recommend using a Ubuntu VM as your development environment; if you're working on Windows, you might as well develop on Windows, and I've had no significant problems doing so. I go back and forth from Windows to Linux with no trouble.
If you have an editor that you're comfortable with that has basic support for Python, I'd stick with it. If not, I've found Geany to be a nice, light, easy-to-use editor with good Python support, though I use Emacs myself because I know it; other people like SCITE, NotePad++, or any of a slew of others. I'd avoid fancy IDEs for Python, because they don't match the character of the language, and I wouldn't bother with IDLE (included with Python), because it's a royal pain to use.
Suggestions for libraries and frameworks:
Django is the standard web framework, but it's big and you have to work django's way; I prefer CherryPy, which is also actively supported, but is light, gives you great freedom, and contains a nice, solid webserver that can be replaced easily with httpd.
Django includes its own ORM, which is nice enough; there's a standalone one for Python, though, which is even nicer: SQL Alchemy
As far as a testing library goes, pyunit seems to me to be the obvious choice
Good luck, and welcome to a really fun language!
EDIT summary: I originally recommended Karrigell, but can't any more: since the 3.0 release, it's been continuously broken, and the community is not large enough to solve the problems. CherryPy is a good substitute if you like a light, simple framework that doesn't get in your way, so I've changed the above to suggest it instead.
Well, if you're thinking of setting up an Ubuntu VM anyway, you might as well make that your development environment. Then you can install Apache and MySQL or Postgres on that VM just via the standard packaging tools (apt-get install), and there's no danger of polluting your Windows environment.
You can either do the actual development on your Windows machine via your favourite IDE, using the VM as a networked drive and saving the code there, or you can just use the VM as a full desktop environment and do everything there, which is what I would recommend.
Install the pre-configured ActivePython release from activestate.
Among other features, it includes the PythonWin IDE (Windows only) which makes it easy to explore Python interactively.
The recommended reference is Dive Into Python, mentioned many times on similar SO discussions.
You should install python 2.4, python 2.5, python 2.6 and python 3.0, and add to your path the one you use more often (Add c:\Pythonxx\ and c:\Pythonxx\Scripts).
For every python 2.x, install easy_install; Download ez_setup.py and then from the cmd:
c:\Python2x\python.exe x:\path\to\ez_setup.py
c:\Python2x\Scripts\easy_install virtualenv
Then each time you start a new project create a new virtual environment to isolate the specific package you needs for your project:
mkdir <project name>
cd <project name>
c:\Python2x\Scripts\virtualenv --no-site-packages .\v
It creates a copy of python and its libraries in .v\Scripts and .\v\Lib. Every third party packages you install in that environment will be put into .\v\Lib\site-packages. The -no-site-packages don't give access to the global site package, so you can be sure all your dependencies are in .\v\Lib\site-packages.
To activate the virtual environment:
.\v\Scripts\activate
For the frameworks, there are many. Django is great and very well documented but you should probably look at Pylons first for its documentions on unicode, packaging, deployment and testing, and for its better WSGI support.
For the IDE, Python comes with IDLE which is enough for learning, however you might want to look at Eclipse+PyDev, Komodo or Wingware Python IDE. Netbean 6.5 has beta support for python that looks promising (See top 5 python IDE).
For the webserver, you don't need any; Python has its own and all web framework come with their own. You might want to install MySql or ProgreSql; it's often better to develop on the same DB you will use for production.
Also, when you have learnt Python, look at Foundations of Agile Python Development or Expert Python Programming.
Using Python on Windows
SO: Python tutorial for total beginners?
Take a look at Pylons, read about WSGI and Paste.
There's nice introductory Google tech talk about them: ReUsable Web Components with Python and Future Python Web Development.
Here's my answer to similar question:
Django vs other Python web frameworks?
NOTE: I included a lot of links to frameworks, projects and what-not, but as a new user I was limited to 1 link per answer. If someone else with enough reputation to edit wants/can edit them into this answer instead of the footnotes, I'd be grateful.
There are some Python IDE's such as Wing IDE[1], I believe some people use Eclipse[2] with a python plugin[3] as well. A lot of people in the #python channel of FreeNode seem to prefer vim, emacs, nano and similar text editors in favor of IDE's. My personal preffered editor is Vim, but if you've mostly done .NET development on windows, presumably with the usual Visual X IDE's, vim and emacs will probably cause you culture shock and you'd be better of using an IDE.
Nearly all python web frameworks* support the WSGI standard[4], most of the large web servers have some sort of plugin to support WSGI, the others support WSGI via fast cgi or plain cgi.
The Zope[5] and Django[6] frameworks have their own ORM's, of other ORM's the two most well known appear to be SQL Alchemy[7] and SQL Object[8]. I only have experience with the former, but both support all possible sane database choices, including SQLite which is installed together with Python and hence perfectly suited to testing and experimenting without polluting your .NET environment with 3rd part web servers and database servers.
The builtin unittest[9] and pyunit[10] frameworks seem to be the preffered solutions for unit testing, but I don't have much experience with these.
bpython[11] and ipython[12] offer enhanced interactive python shells which can greatly help speed up and testing small bits of code and hence worth looking in to.
As for a list of well known and often used web frameworks, look into the following frameworks**:
Twisted[13] is a generic networking framework, which supports almost every single protocol under the sun.
Pylons[14] is light-weight framework aimed at being as flexible as possible and leaving all the choices about what ORM, templating language and what-not to you.
CherryPy[15] tries to provide an interface to expose Python objects to the web.
Django[6] attempts to be an all-in-one solution, builtin template system, ORM, admin pages and internationalization. While the previous frameworks have more DIY wiring together various frameworks work involved with them.
Zope[5] is aimed to be suitable for large enterprise applications, I've heard nothing but good things about it, but consensus seems to be that for smaller you're probably better off with one of the simpler and smaller frameworks.
TurboGears[16] is the framework I know the least about, but it seems to be mostly competition for Django.
This is everything I can think of right now, I'll edit and add stuff if I can think of it. I hope this helps you some in the wonderful world of python.
* - The main exception would be Apache's mod_python, which you should avoid for exactly that reason, use mod_wsgi instead.
** - Word of warning, I have not personally used these frameworks this is just a very short impression I have gotten from talking to other people about each framework, it may be wildly inaccurate. (If anyone has any corrections, do comment and I'll try to edit and fix this answer).
(The http:// is missing since they're recognized as links otherwise)
[1] www.wingware.com/
[2] www.eclipse.org/
[3] pydev.sourceforge.net/
[4] wsgi.org/wsgi/
[5] www.zope.org/
[6] www.djangoproject.com/
[7] www.sqlalchemy.org/
[8] www.sqlobject.org/
[9] docs.python.org/library/unittest.html
[10] pyunit.sourceforge.net/pyunit.html
[11] www.bpython-interpreter.org/
[12] ipython.scipy.org/
[13] twistedmatrix.com/trac/
[14] pylonshq.com/
[15] www.cherrypy.org/
[16] turbogears.org/
Environment?
Here is the simplest solution:
Install Active Python 2.6. Its the Python itself, but comes with some extra handy useful stuff, like DiveintoPython chm.
Use Komodo Edit 5. It is among the good free editor you can use for Python.
Use IDLE. Its the best simplest short snippet editor, with syntax highlighting and auto complete unmatched by most other IDEs. It comes bundled with python.
Use Ipython. Its a shell that does syntax highlighting and auto complete, bash functions, pretty print, logging, history and many such things.
Install easy_install and/or pip for installing various 3rd party apps easily.
Coming from Visual Studio and .Net it will sound a lot different, but its an entirely different world.
For the framework, django works the best. Walk thro the tutorial and you will be impressed enough. The documentation rocks. The community, you have to see for yourself, to know how wonderful it is!!
Python has build in SQL like database and web server, so you wouldn't need to install any third party apps. Remember Python comes with batteries included.
If you've worked with Eclipse before you could give Pydev a try
I'm wondering if there's such a thing as Django-like ease of web app development combined with good deployment, debugging and other tools?
Django is a very productive framework for building content-heavy sites; the best I've tried and a breath of fresh air compared to some of the Java monstrosities out there. However it's written in Python which means there's little real support in the way of deployment/packaging, debugging, profilers and other tools that make building and maintaining applications much easier.
Ruby has similar issues and although I do like Ruby much better than I like Python, I get the impression that Rails is roughly in the same boat at Django when it comes to managing/supporting the app.
Has anyone here tried both Django and Grails (or other web frameworks) for non-trivial projects? How did they compare?
You asked for someone who used both Grails and Django. I've done work on both for big projects. Here's my Thoughts:
IDE's:
Django works really well in Eclipse, Grails works really well in IntelliJ Idea.
Debugging:
Practically the same (assuming you use IntelliJ for Grails, and Eclipse for Python). Step debugging, inspecting variables, etc... never need a print statement for either. Sometimes django error messages can be useless but Grails error messages are usually pretty lengthy and hard to parse through.
Time to run a unit test:
django: 2 seconds.
Grails: 20 seconds (the tests themselves both run in a fraction of a second, it's the part about loading the framework to run them that takes the rest... as you can see, Grails is frustratingly slow to load).
Deployment:
Django: copy & paste one file into an apache config, and to redeploy, just change the code and reload apache.
Grails: create a .war file, deploy it on tomcat, rinse and repeat to redeploy.
Programming languages:
Groovy is TOTALLY awesome. I love it, more so than Python. But I certainly have no complaints.
Plugins:
Grails: lots of broken plugins (and can use every java lib ever).
Django: a few stable plugins, but enough to do most of what you need.
Database:
Django: schema migrations using South, and generally intuitive relations.
Grails: no schema migrations, and by default it deletes the database on startup... WTF
Usage:
Django: startups (especially in the Gov 2.0 space), independent web dev shops.
Grails: enterprise
Hope that helps!
However it's written in Python which
means there's little real support in
the way of deployment/packaging,
debugging, profilers and other tools
that make building and maintaining
applications much easier.
Python has:
a great interactive debugger, which makes very good use of Python REPL.
easy_install anv virtualenv for dependency management, packaging and deployment.
profiling features comparable to other languages
So IMHO you shouldn't worry about this things, use Python and Django and live happily :-)
Lucky for you, newest version of Django runs on Jython, so you don't need to leave your whole Java ecosystem behind.
Speaking of frameworks, I evaluated this year:
Pylons (Python)
webpy (Python)
Symfony (PHP)
CakePHP (PHP)
None of this frameworks comes close to the power of Django or Ruby on Rails. Based on my collegue opinion I could recommend you kohana framework. The downside is, it's written in PHP and, as far as I know, PHP doesn't have superb tools for debugging, profiling and packaging of apps.
Edit: Here is a very good article about packaging and deployment of Python apps (specifically Django apps). It's a hot topic in Django community now.
The statement that grails deletes the database on start-up is completely wrong. It's behavior on start-up is completely configurable and easy to configure. I generally use create-drop when running an app in dev mode. I use update when I run in test and production.
I also love the bootstrap processing that lets me pre-configure test users, data, etc by environment in Grails.
I'd love to see someone who has really built and deployed some commercial projects comment on the pros / cons. Be a really interesting read.
Grails.
Grails just looks like Rails (Ruby),but it uses groovy which is simpler than java. It uses java technology and you can use any java lib without any trouble.
I also choose Grails over simplicity and there are lots of java lib (such as jasper report, jawr etc) and I am glad that now they join with SpringSource which makes their base solid.
I have two friends who originally started writing an application using Ruby on Rails, but ran into a number of issues and limitations. After about 8 weeks of working on it, they decided to investigate other alternatives.
They settled on the Catalyst Framework, and Perl. That was about 4 months ago now, and they've repeatedly talked about how much better the application is going, and how much more flexibility they have.
With Perl, you have all of CPAN available to you, along with the large quantity of tools included. I'd suggest taking a look at it, at least.
The "good deployment" issue -- for Python -- doesn't have the Deep Significance that it has for Java.
Python deployment for Django is basically "move the files". You can run straight out of the subversion trunk directory if you want to.
You can, without breaking much of a sweat, using the Python distutils and build yourself a distribution kit that puts your Django apps into Python's site-packages. I'm not a big fan of it, but it's really easy to do.
Since my stuff runs in Linux, I have simple "install.py" scripts that move stuff out of the Subversion directories into /opt/this and /opt/that directories. I use an explicit path settings in my Apache configuration to name those directories where the applications live.
Patching can be done by editing the files in place. (A bad policy.) I prefer to edit in the SVN location and rerun my little install to be sure I actually have all the files under control.
cakephp.org
Cakephp is really good, really close to ruby on rails (1.2). It is in php, works very well on shared hosts and is easy to implement.
The only downside is that the documentation is somewhat lacking, but you quickly get it and quickly start doing cool stuff.
I totally recommend cakephp.
Personally I made some rather big projects with Django, but I can compare only with said "montrosities" (Spring, EJB) and really low-level stuff like Twisted.
Web frameworks using interpreted languages are mostly in its infancy and all of them (actively maintained, that is) are getting better with every day.
By "good deployment" are you comparing it with Java's EAR files, which allow you to deploy web applications by uploading a single file to a J2EE server? (And, to a lesser extent, WAR files; EAR files can have WAR files for dependent projects)
I don't think Django or Rails have gotten quite to that point yet, but I could be wrong... zuber pointed out an article with more details on the Python side.
Capistrano may help out on the Ruby side.
Unfortunately, I haven't really worked with either Python or Ruby that much, so I can't help out on profilers or debuggers.