I once did a cursory search and found no good CVS bindings for Python. I wanted to be able to write helper scripts to do some fine-grained manipulation of the repository and projects in it. I had to resort to using popen and checking stdout and stderr and then parsing those. It was messy and error-prone.
Are there any good quality modules for CVS integration for Python? Which module do you prefer and why?
While I am at it, is there a good Subversion integration module for Python? My understanding is that Subversion has a great API for such things.
For cvs, pyCVS may be worth a look.
For svn, there is pysvn, which is pretty good.
Tailor, a Python program which lets different version control systems interoperate, simply calls the external programs cvs and svn when working with repositories of those formats. This seems pretty ugly, but reduces Tailor's dependencies from "requires ____ bindings" to "requires working system".
Related
I'm interested in what pitfalls can be (except Python is not installed in target system) when using Python for deb package flow control scripts (preinst, postinst, etc.). Will it be practical to implement those scripts in Python, not in sh?
As I understand it's at least possible.
The only reason this isn't commonly done, afaik, is that it's not convention, and Python isn't usually more useful or straightforward than plain shell script for the sorts of things that maintainer scripts do. When it is more useful, you can often break out the Python-needing functionality into a separate Python script which is called by the maintainer scripts.
It can help to follow convention in this sort of situation, since there are a lot of helpful tools and scripts (e.g., Lintian, Debhelper) which generally assume that maintainer scripts use bash. If they don't, it's ok, but those tools may not be as useful as they would be otherwise. The only other issue I think you need to be aware of is that if your preinst or postrm scripts need Python, then Python needs to be a pre-dependency (Pre-Depends) of your package instead of just a Depends.
That said, I've found it useful to use Python in a maintainer script before.
I've just built a Jython project that uses both some Python module imports and some Java jars. On my own computer, since I just wanted the thing done, I've gotten things to work in a very hacky way by hardcoding sys.path and installing every module and jar I wanted separately. This is definitely not something I want to keep for a release version. I've read about being able to package everything up into one standalone Jython jar, and that sounds pretty good to me. Is there any reason I shouldn't do this? If not, is there a guide on the best way to do this someone can point me to? I'm running the whole thing through PIG, so having a callable Jython jar would be ideal.
I know some similar questions to this already exist on SO, but the answers to those seem pretty old, and the documentation given (for MavenJython, for example) is pretty poor. I've already looked at MavenJython, and Jip, but I can't really decide between the two, and I'm not really finding sufficient information for either. An ideal answer to this question what I should use, why I should use it, and give a brief demo of how one would use it. A link to any of those would also be awesome.
Thanks!
Please elaborate on how the MavenJython documentation can be improved. Things have not changed since 2011 (you probably have seen this answer), which is not that long ago. There is the website, but as a programmer you probably just want to read the source code of the jython-compile-maven-plugin-test project, which is only 200 lines of code. It is a good idea to use this package as a starting point for your own project.
Distribution philosophies
The way Java software distribution (and Windows software distribution) usually works is that you package everything you need. So yes, a standalone Jython jar would be appropriate. The drawback is that every software may use a different Jython version, the benefit being that this might be what you want (updates may break things). This is the MavenJython approach too, packaging everything in the correct versions.
Python and Linux software distribution just installs packages, checking compatibility at install time. This is the jip approach, which assumes you already have Jython, and whoever installs software will resolve compability issues by installing the correct versions.
Differences
I can not say much about jip though, I have not used it. From what I see in the demos jip is meant to provide Python packages access to Maven Java libraries. It also seems to be capable of producing maven packages from Jython code. So you can probably achieve your goal using either MavenJython or jip. Just try.
The deliverables built using MavenJython distribute Jython, while jip does not.
If you want to instruct programmers who already use jython and are unfamiliar with Maven, who want to use your library to fetch your jython library package, jip might be the way to go.
If you want to write Jython libraries for programmers and distribute them, you can use either MavenJython or jip.
If you have a software package that is going to end up as a deliverable to customers, which happens to also use Jython code and Jython packages, perhaps also providing in-program scripting to the user, go with MavenJython. It allows you to create a standalone executable.
Pig use case
For running jython through pig it is enough to install jython and put the jython sources in your path -- see the embed python section of the pig manual. jip can be appropriate for installing jython packages locally, but is not necessary if you only want to run your code. If you however want to distribute software which uses pig, and install pig, jython and your code on a clients computer, MavenJython can do that for you.
I recently started learning Python. Not yet ventured into coding.
During one of my learning sessions, i came accross the term Jython.
I googled it & got some information.
I would like to know if anyone has implemented any real-world program using Jython.
Most of the time, Jython isn't used directly to write full read-world programs, but a lot of programs actually embed Jython to use it as a scripting language.
The official Jython website gives a list of projects, some written in Jython, others using Jython for scripting:
http://wiki.python.org/jython/JythonUsers
I am writing a full application in Jython at the moment, and would highly recommend it. Having all of the Java libraries at your disposal is very handy, and the Python syntax and language features actually make using some of them easier than it is in Java (I'm mostly talking about Swing here).
Check out the chapter on GUI Applications from the Jython book. It does a lot of comparisons like 'Look at all this Java code, and now look at it reduced to Python code of half the length!'.
The only caveats I've found are:
Jython development tends to run slightly behind Python, which can be annoying if you find a cool way of doing something in Python, only to discover it's not supported in the current Jython version.
Occasionally you might have hiccups with the interface between Python and Java (I have a couple of unsolved problems here and here, although there are always workarounds for this kind of thing).
Distribution is not as simple as it could be, although once you figure out how to do it, it's fairly painless. I recommend following the method here. It essentially consists of:
Exploding jython.jar and adding your own modules into it.
Writing and compiling a small Java class that creates a Python interpreter and loads up your Python modules.
Creating an executable .jar file consisting of the jython.jar modules, your own Python modules, and the Java class.
Jython really shines for dependency injection.
You know those pesky variables you have to give your program, like
file system paths
server names
ports
Jython provides a really nice way of injecting those variables by putting them in a script. It works equally well for injecting java dependencies, as well.
WebSphere and WebLogic use it as their default scripting engine for administrative purposes.
A lot of other Oracle products ship it as part of their "oracle_commons" module (Oracle Universal Installer, Oracle HTTP Server etc). It's mostly version 2.2 being deployed though, which is a bit old and clunky.
There is a list of application that uses jython at http://wiki.python.org/jython/JythonUsers
I've got some experience with Bash, which I don't mind, but now that I'm doing a lot of Windows development I'm needing to do basic stuff/write basic scripts using
the Windows command-line language. For some reason said language really irritates me, so I was considering learning Python and using that instead.
Is Python suitable for such things? Moving files around, creating scripts to do things like unzipping a backup and restoring a SQL database, etc.
Python is well suited for these tasks, and I would guess much easier to develop in and debug than Windows batch files.
The question is, I think, how easy and painless it is to ensure that all the computers that you have to run these scripts on, have Python installed.
Summary
Windows: no need to think, use Python.
Unix: quick or run-it-once scripts are for Bash, serious and/or long life time scripts are for Python.
The big talk
In a Windows environment, Python is definitely the best choice since cmd is crappy and PowerShell has not really settled yet. What's more Python can run on several platform so it's a better investment. Finally, Python has a huge set of library so you will almost never hit the "god-I-can't-do-that" wall. This is not true for cmd and PowerShell.
In a Linux environment, this is a bit different. A lot of one liners are shorter, faster, more efficient and often more readable in pure Bash. But if you know your quick and dirty script is going to stay around for a while or will need to be improved, go for Python since it's far easier to maintain and extend and you will be able to do most of the task you can do with GNU tools with the standard library. And if you can't, you can still call the command-line from a Python script.
And of course you can call Python from the shell using -c option:
python -c "for line in open('/etc/fstab') : print line"
Some more literature about Python used for system administration tasks:
The IBM lab point of view.
A nice example to compare bash and python to script report.
The basics.
The must-have book.
Sure, python is a pretty good choice for those tasks (I'm sure many will recommend PowerShell instead).
Here is a fine introduction from that point of view:
http://www.redhatmagazine.com/2008/02/07/python-for-bash-scripters-a-well-kept-secret/
EDIT: About gnud's concern: http://www.portablepython.com/
Are you aware of PowerShell?
Anything is a good replacement for the Batch file system in windows. Perl, Python, Powershell are all good choices.
#BKB definitely has a valid concern. Here's a couple links you'll want to check if you run into any issues that can't be solved with the standard library:
Pywin32 is a package for working with low-level win32 APIs (advanced file system modifications, COM interfaces, etc.)
Tim Golden's Python page: he maintains a WMI wrapper package that builds off of Pywin32, but be sure to also check out his "Win32 How Do I" page for details on how to accomplish typical Windows tasks in Python.
Python is certainly well suited to that. If you're going down that road, you might also want to investigate SCons which is a build system itself built with Python. The cool thing is the build scripts are actually full-blown Python scripts themselves, so you can do anything in the build script that you could otherwise do in Python. It makes make look pretty anemic in comparison.
Upon rereading your question, I should note that SCons is more suited to building software projects than to writing system maintenance scripts. But I wouldn't hesitate to recommend Python to you in any case.
As a follow up, after some experimentation the thing I've found Python most useful for is any situation involving text manipulation (yourStringHere.replace(), regexes for more complex stuff) or testing some basic concept really quickly, which it is excellent for.
For stuff like SQL DB restore scripts I find I still usually just resort to batch files, as it's usually either something short enough that it actually takes more Python code to make the appropriate system calls or I can reuse snippets of code from other people reducing the writing time to just enough to tweak existing code to fit my needs.
As an addendum I would highly recommend IPython as a great interactive shell complete with tab completion and easy docstring access.
I've done a decent amount of scripting in both Linux/Unix and Windows environments, in Python, Perl, batch files, Bash, etc. My advice is that if it's possible, install Cygwin and use Bash (it sounds from your description like installing a scripting language or env isn't a problem?). You'll be more comfortable with that since the transition is minimal.
If that's not an option, then here's my take. Batch files are very kludgy and limited, but make a lot of sense for simple tasks like 'copy some files' or 'restart this service'. Python will be cleaner, easier to maintain, and much more powerful. However, the downside is that either you end up calling external applications from Python with subprocess, popen or similar. Otherwise, you end up writing a bunch more code to do things that are comparatively simple in batch files, like copying a folder full of files. A lot of this depends on what your scripts are doing. Text/string processing is going to be much cleaner in Python, for example.
Lastly, it's probably not an attractive alternative, but you might also consider VBScript as an alternative. I don't enjoy working with it as a language personally, but if portability is any kind of concern then it wins out by virtue of being available out of the box in any copy of Windows. Because of this I've found myself writing scripts that were unwieldy as batch files in VBScript instead, since I can't usually depend on Python or Perl or Bash being available on Windows.
Python, along with Pywin32, would be fine for Windows automation. However, VBScript or JScript used with the Windows Scripting Host works just as well, and requires nothing additional to install.
I've been using a lot of Windows Script Files lately. More powerful than batch scripts, and since it uses Windows scripting, there's nothing to install.
As much as I love python, I don't think it a good choice to replace basic windows batch scripts.
I can't see see someone having to import modules like sys, os or getopt to do basic things you can do with shell like call a program, check environment variable or an argument.
Also, in my experience, goto is much easier to understand to most sysadmins than a function call.
I have started on a personal python application that runs on the desktop. I am using wxPython as a GUI toolkit. Should there be a demand for this type of application, I would possibly like to commercialize it.
I have no knowledge of deploying "real-life" Python applications, though I have used py2exe in the past with varied success. How would I obfuscate the code? Can I somehow deploy only the bytecode?
An ideal solution would not jeopardize my intellectual property (source code), would not require a direct installation of Python (though I'm sure it will need to have some embedded interpreter), and would be cross-platform (Windows, Mac, and Linux). Does anyone know of any tools or resources in this area?
Thanks.
You can distribute the compiled Python bytecode (.pyc files) instead of the source. You can't prevent decompilation in Python (or any other language, really). You could use an obfuscator like pyobfuscate to make it more annoying for competitors to decipher your decompiled source.
As Alex Martelli says in this thread, if you want to keep your code a secret, you shouldn't run it on other people's machines.
IIRC, the last time I used cx_Freeze it created a DLL for Windows that removed the necessity for a native Python installation. This is at least worth checking out.
Wow, there are a lot of questions in there:
It is possible to run the bytecode (.pyc) file directly from the Python interpreter, but I haven't seen any bytecode obfuscation tools available.
I'm not aware of any "all in one" deployment solution, but:
For Windows you could use NSIS(http://nsis.sourceforge.net/Main_Page). The problem here is that while OSX/*nix comes with python, Windows doesn't. If you're not willing to build a binary with py2exe, I'm not sure what the licensing issues would be surrounding distribution of the Python runtime environment (not to mention the technical ones).
You could package up the OS X distribution using the "bundle" format, and *NIX has it's own conventions for installing software-- typically a "make install" script.
Hope that was helpful.
Maybe IronPython can provide something for you? I bet those .exe/.dll-files can be pretty locked down. Not sure how such features work on mono, thus no idea how this works on Linux/OS X...
I have been using py2exe with good success on Windows. The code needs to be modified a bit so that the code analysis picks up all modules needed, but apart from that, it works.
As for Linux, there are several important distribution formats:
DEB (Debian, Ubuntu and other derivatives)
RPM (RedHat, Fedora, openSuSE)
DEBs aren't particularly difficult to make, especially when you're already using distutils/setuptools. Some hints are given in the policy document, examples for packaging Python applications can be found in the repository.
I don't have any experience with RPM, but I'm sure there are enough examples to be found.
Try to use scraZ obfuscator (http://scraZ.me).
This is obfuscator for bytecode, not for source code.
Free version have good, but not perfect obfuscation methods.
PRO version have very very strong protection for bytecode.
(after bytecode obfuscation a decompilation is impossible)