We Keep Coding

monkey-patching/decorating/wrapping an entire python module

I would like, given a python module, to monkey patch all functions, classes and attributes it defines. Simply put, I would like to log every interaction a script I do not directly control has with a module I do not directly control. I'm looking for an elegant solution that will not require prior knowledge of either the module or the code using it.
I found several high-level tools that help wrapping, decorating, patching etc... and i've went over the code of some of them, but I cannot find an elegant solution to create a proxy of any given module and automatically proxy it, as seamlessly as possible, except for appending logic to every interaction (record input arguments and return value, for example).

in case someone else is looking for a more complete proxy implementation
Although there are several python proxy solutions similar to those OP is looking for, I could not find a solution that will also proxy classes and arbitrary class objects, as well as automatically proxy functions return values and arguments. Which is what I needed.
I've got some code written for that purpose as part of a full proxying/logging python execution and I may make it into a separate library in the future. If anyone's interested you can find the core of the code in a pull request. Drop me a line if you'd like this as a standalone library.
My code will automatically return wrapper/proxy objects for any proxied object's attributes, functions and classes. The purpose is to log and replay some of the code, so i've got the equivalent "replay" code and some logic to store all proxied objects to a json file.

Programmatically modifying someones AppDelegate - categories, subclass?

I am working on a framework installer script. The script needs to modify the users AppDelegate file and inject a few lines of code at the beginning or end of the applicationDidFinishLaunching and applicationWillTerminatate methods.
Some options I've thought about:
Parse the source code, and insert lines at correct positions. (Can be difficult to get right and work for everyone's code, just about equivalent to writing a compiler...)
Subclass the AppDelegate file (is this possible?)
Categories??
Which of these is the best option? Any other suggestions?

If you really need to make this something that modifies the AppDelegate with no intervention at all from the developer, and you can modify the xcodeproj and the nib but not the source, there is a way to do it.
First, make sure your classes get compiled in, and an instance of your class gets created in the nib.
Now, here's what you do:
Define a -[AHHackClass applicationDidFinishLaunching] method that does your extra stuff, then calls the [self originalApplicationDidFinishLaunching].
In -[AHHackClass awakeFromNib:], use objc runtime calls to copy the -[AHHackClass applicationDidFinishLaunching] method to the application delegate as -[originalApplicationDidFinishLaunching], then use method swizzling to swap the two methods' implementations.
Do the same to swizzle applicationWillTerminate.
See JRSwizzle for some code that makes the method swizzling much easier, and MethodSwizzling at CocoaDev for some background.
However, there may be a much easier way to do this: Does your extra stuff really need to be called from the app delegate's applicationDidFinishLaunching and applicationWillTerminate methods? Can't you just set up to listen for notifications in your awakeFromNib and handle things there?
And if, for some reason, you can't do that, can you just put a line in the instructions to the developer to call your method from their applicationDidFinishLaunching method?

One solution I am currently considering:
Add NewAppDelegate.m/h file that subclasses AppDelegate.
This subclass, does what I want, and then calls the super methods.
Find/replace AppDelegate with NewAppDelegate.m.h in main.m
This seems pretty simple and robust. Thoughts on this? Will this work for all/most projects?

How to properly manage application configurations

What is the most universal and best application configurations management method? I want to have these properties in order to have "good configuration management":
A list of all available properties and their default values in one
place.
A list of properties which can be changed by an app user, also in one
place.
When I retrieve a specific property, it's value is returned from the
2nd list (user changeable configs) or if it's not there, from the
first list.
So far, what I did was hard coding the 1st list as an object (more specific as a dict), wrote .conf file used by ConfigParser to make an app user to easily change some of the properties (2nd list), and wrote a public method on the config object to retrieve a property by it's name or if it's not there, raise an exception. In the end, one object was responsible for managing all the stuff (parsing file, raising exception, overriding properties etc.) But I was wondering, if there's a built-in library which does more or less the same thing, or even a better way to manage configuration, which takes into account all the KISS, DRY and other principles (I'm not always successful to do that with this method)?
Thanks in advance.

Create a default settings module which contains your desired default settings. Create a second module intended to be used by the the user with a from default_settings import * statement at the top, and instructing the user to write any replacements into this module instead.
Python is rather expressive, so in most cases, if you can expect the user to understand it on any level, you can use a Python module itself as the configuration file.

Are verbose __init__ methods in Python bad?

I have a program that I am writing in Python that does the following:
The user enters the name of a folder. Inside that folder a 8-15 .dat files with different extensions.
The program opens those dat files, enters them into a SQL database and then allows the user to select different changes made to the database. Then the database is exported back to the .dat files. There are about 5-10 different operations that could be performed.
The way that I had planned on designing this was to create a standard class for each group of files. The user would enter the name of the folder and an object with certain attributes (file names, dictionary of files, version of files (there are different versions), etc) would get created. Determining these attributes requires opening a few of these files, reading file names, etc.
Should this action be carried out in the __init__ method? Or should this action be carried our in different instance methods that get called in the __init__ method? Or should these methods be somewhere else, and only be called when the attribute is required elsewhere in the program?
I have already written this program in Java. And I had a constructor that called other methods in the class to set the object's attributes. But I was wondering what standard practice in Python would be.

Well, there is nothing special about good OOP practices in Python. Decomposition of one big method into a bunch of small ones is great idea both in Java and in Python. Among other things small methods gives you an opportunity to write different constructors:
class GroupDescriptor(object):
def __init__(self, file_dictionary):
self.file_dict = file_dictionary
self.load_something(self.file_dict['file_with_some_info'])
#classmethod
def from_filelist(cls, list_of_files):
file_dict = cls.get_file_dict(list_of_files)
return cls(file_dict)
#classmethod
def from_dirpath(cls, directory_path):
files = self.list_dir(directory_path)
return cls.from_filelist(files)
Besides, I don't know how it is in Java but in Python you don't have to worry about exceptions in constructor because they are finely handled. Therefore, it is totally normal to work with such exception-prone things like files.

It looks the action you are describing are initialization, so it'd be perfectly ok to put them into __init__. On the other hand, these actions seem to be pretty expensive, and probably useful in the other part of a program, so you might want to encapsulate them in some separate function.

There's no problem with having a long __init__ method, but I would avoid it simply because its more difficult to test. My approach would be to create smaller methods which are called from __init__. This way you can test them and the initialization separately.
Whether they should be called when needed or run up front really depends on what you need them to do. If they are expensive operations, and are usually not all needed, then maybe its better to only call them when needed. On the other hand, you might want to run them up front so that there is no lag when the attributes are required.
Its not clear from your question whether you actually need a class though. I have no experience with Java, but I understand that everything in it is a class. In python it is perfectly acceptable to just have a function if that's all that's required, and to only create classes when you need instances and other classy things.

The __init__ method is called when the object is instantiated.
Coming from a C++ background I believe its not good to do actual work other than initialization in the constructor.

Use user provided python code during runtime

I'm developing a system that operates on (arbitrary) data from databases. The data may need some preprocessing before the system can work with it. To allow the user the specify possibly complex rules I though of giving the user the possibility to input Python code which is used to do this task. The system is pure Python.
My plan is to introduce the tables and columns as variables and let the user to anything Python can do (including access to the standard libs). Now to my problem:
How do I take a string (the user entered), compile it to Python (after adding code to provide the input data) and get the output. I think the easiest way would be to use the user-entered data a the body of a method and take the return value of that function a my new data.
Is this possible? If yes, how? It's unimportant that the user may enter malicious code since the worst thing that could happen is, that he screws up his own system, which is thankfully not my problem ;)

Python provides an exec() statement which should do what you want. You will want to pass in the variables that you want available as the second and/or third arguments to the function (globals and locals respectively) as those control the environment that the exec is run in.
For example:
env = {'somevar': 'somevalue'}
exec(code, env)
Alternatively, execfile() can be used in a similar way, if the code that you want executed is stored in its own file.
If you only have a single expression that you want to execute, you can also use eval.

Is this possible?
If it doesn't involve time travel, anti-gravity or perpetual motion the answer to this question is always "YES". You don't need to ask that.
The right way to proceed is as follows.
You build a framework with some handy libraries and packages.
You build a few sample applications that implement this requirement: "The data may need some preprocessing before the system can work with it."
You write documentation about how that application imports and uses modules from your framework.
You turn the framework, the sample applications and the documentation over to users to let them build these applications.
Don't waste time on "take a string (the user entered), compile it to Python (after adding code to provide the input data) and get the output".
The user should write applications like this.
from your_framework import the_file_loop
def their_function( one_line_as_dict ):
one_line_as_dict['field']= some stuff
the_file_loop( their_function )
That can actually be the entire program.
You'll have to write the_file_loop, which will look something like this.
def the_file_loop( some_function ):
with open('input') as source:
with open('output') as target:
for some_line in source:
the_data = make_a_dictionary( some_line )
some_function( the_data )
target.write( make_a_line( the_data ) )
By creating a framework, and allowing users to write their own programs, you'll be a lot happier with the results. Less magic.

2 choices:
You take his input and put it in a file, then you execute it.
You use exec()

If you just want to set some local values and then provide a python shell, check out the code module.
You can start an instance of a shell that is similar to the python shell, as well as initialize it with whatever local variables you want. This would assume that whatever functionality you want to use the resulting values is built into the classes you are passing in as locals.
Example:
shell = code.InteractiveConsole({'foo': myVar1, 'bar': myVar2})

What you actually want is exec, since eval is limited to taking an expression and returning a value. With exec, you can have code blocks (statements) and work on arbitrarily complex data, passed in as the globals and locals of the code.
The result is then returned by the code via some convention (like binding it to result).

well, you're describing compile()
But... I think I'd still implement this using regular python source files. Add a special location to the path, say '~/.myapp/plugins', and just __import__ everything there. Probably you'll want to provide some convenient base classes that expose the interface you're trying to offer, so that your users can inherit from them.