We Keep Coding

How to avoid Python script name being regarded as namespace with Doxygen

I am now making documentations with Doxygen for Python scrips. Suppose the name of the script is all_the_best.py, and at the beginning of the scripts, I document it as follows:
##
# #namespace scripts.python
# This is a python script
import os
...
I suppose that the script will belong to scripts.python namespace in the Namespaces tab of generated HTML file. However, I found that in the Namespaces tab, not only scripts.python is available but also all_the_best appears. Any ideas on how to avoid it? Thanks.

I tried a lot of things, and in my case I had to put EXTRACT_ALL = NO in the Doxyfile to have the second instance removed.

How to abstract 3rd party implementation details from the core functionality?

Here is an example of the problem and requirement .
Scenario : We have a Web App , The Controller has a function that does HTML to PDF conversion . We have to switch to various engines depending on the requirement and advancement in Technology . This means we need to keep changing the core file . It can even be a web service that does my job and not implemented locally .
Overall the current architecture in pseudo Python is :
Controller
def requestEntry():
"""Entry point"""
html = REQUEST['html'] , css = REQUEST['css']
createPDF(html,css)
def createPDF(html,css):
"""Sigh"""
#Hardcoding Begines
configured_engine = getXMLOption('configured_engine')
if configured_engine == "SuperPDFGen":
#Returns SuperGen's PDF Blob
supergen = SuperGen(html,css)
return supergen.pdfout()
if configured_engine = "PISA":
pisa = PISA(html,css)
return pisa.pdf_out()
The problem here is every new engine requirement is a code change in the controller .And suppose we find a new technology we have to upgrade the core software version .
Solving it :
One way I can think of is to define a simple class like this :
def createPdf(data,engine):
pdf_render = PDFRender(html,css,engine)
return pdf_render.pdf_out()
So now we keep out PDFRender astracted - core version need not be changed any more for a implemetation change , but we need to code the engines with a If ladder in the PDFRender Class .
To extend this idea , i could have a convention engine would be the module name too . But this own't adapt to a URL if its given as Engine .
def createPdf(data,engine):
#Convert the string called engine to a Module ! Sigh
import engine Engine!!
pdf_render = engine.Engine(data)
return pdf_render()
Are there any suggested paradigm's or a plugin adapter mechanism for this? It should take a URL or a Python Module as input and get the job done . New implementations should be standalone and pluggable to the existing code with no version changes in core feature . How can I do that ? I think the core should talk in terms of service (either a python module , sub process or a url ) this is what I trying to achieve .

Add a package where you stick your renderers, one Python module for each, the module being named after the "engine" and the renderer class being named "PDFRenderer". Then in the package's __init__ scan the modules in the directory, import the PDFRenderer class from each module, and build a enginename:PDFRenderer mapping, and use it to get the PDFRenderer class for the engine.

It looks like right now you have to change a couple of lines of code each time you change pdf renderer. The code is simple and easy to understand.
Sure, you can abstract it all out, and use configuration files, and add a plugin architecture. But what will you gain? Instead of changing a couple of lines of code, you'll instead change a plugin or a configuration file, and that machinery will need to be maintained: for example, if you decide you need to pass in some flags to one of the renderers, then you have to add that functionality to your configuration file or whatever abstraction you've chosen.
How many times do you think you'll change pdf renderer? My guess would be at most once a year. Keep your code simple, and don't build complex solutions until you really need them.

Python: Mako template lookups per app

I'm using cherrypy with Mako as a template engine.
I want Mako to lookup different directories based on what app is being requested.
I.e.
I have three 'apps': Site, Admin and Install.
They all have their own template folder, structure looking something like:
/template
/template/site
/template/admin
/template/install
/template/system
/system contains some system wide templates, like 404 pages, etc.
I'm using Twiseless as a reference whilst trying to get to grips with cherrypy / mako, but I'm stuck with how to do this.
Read on for a brief overview of how I've tried to do this, but a warning: I think I'm going about this completely the wrong way! :) So, if you have any ideas/pointers, it might be a good idea to save yourself the trouble of reading any further than this.
In my main file, server.py, I do something like:
from libs.plugins.template import MakoTemplatePlugin
engine = cherrypy.engine
makoTemplate = MakoTemplatePlugin(engine, self.base_dir)
setTemplateDirs(makoTemplate, self.template_path)
MakoTemplatePlugin is a slightly modified version of the plugin by the same name found in Twiseless, linked above.
What this code does is set the TemplateLookup to use the default template directories from my global config file. i.e.
/template
/template/system
Then, each time an app is loaded, I call a function (setTemplateDirs) to update the directories where Mako searches.
I thought this would work, but it doesn't. Initially I made the error of creating a new instance of MakoTemplatePlugin for each app. This just resulted in them all being called on each page load, starting with the first one instantiated, containing just the basic, non-app specific directories.
As this was called first, it was triggering a 404 error, as it was searching in the wrong folders.
I instead made sure to pass a reference to the MakeTemplatePlugin to all of my apps. I thought if I ran setTemplateDirs each time each app is called, this would solve the problem... but it doesn't.
I don't know where to put the function so it will run every time a page is requested...
e.g.
# /apps/site/app.py
import somemodule.setTemplateDirs
class Site(object, params):
def __init__(self):
self.params = params
self.makoTemplate = params['makoTemplate']
self.base_path = params['base_path']
setTemplateDirs(self.makoTemplate, self.base_path, '', '/')
#cherrypy.expose
#cherrypy.tools.render(template='index.html')
def index(self):
pass
This obviously just works when the application is first loaded... I tried moving the update function call into a seperate method update and tried calling that for each page, e.g:
#cherrypy.exposed
#cherrypy.tools.render(template='index.html')
#update
def index(self):
pass
But this just gives me config related errors.
Rather than to continue to mess about with this, there must be an easier way.
How would you do it?
Thanks a lot,
Tom

I got this working. Thanks to stephan for providing the link to the mako tool example: http://tools.cherrypy.org/wiki/Mako.
I just modified that slightly to get it working.
If anyone's wondering, the basis of it is that you define tools.mako.directories in your global config, you can then override that in individual app config files.
e.g.
server.conf
...
tools.mako.directories: ['', 'system']
...
site.conf
...
tools.mako.directories: ['site', 'system']
...
I did some extra work to translate the relative URIs to absolute paths, but the crux of it is explained above.

Inclusion of external documentation into a Sphinx project

We maintain a fairly large documentation using Sphinx in SVN.
As part of the generated output we would like to include the release notes of related Python modules as primary content (not as hyperlink!). The release notes of the external modules are also maintained in SVN. Is there some Sphinx-ish way to pull in the parts of the documentation from other (SVN) sources? Ok, using SVN externals is a way to solve the problem but perhaps not the smartest way...any better options?

The two options I can think of are:
Add an svn:externals link to the remote project (which you already know about).
Extend Sphinx with a custom directive to include files from remote subversion repositories.
I'm no expert on Sphinx internals but was able to cobble together a quick extension which embeds files from a remote subversion repository.
The extension adds an svninclude directive which takes 1 argument, the url of the repository where your docs are located. It checks this repository out into a temp directory _svncache located in the project root, and then proceeds to read the contents of each file and insert them into the parser's state machine.
Here is the code for the svninclude.py extension. It is oversimplified and has no error checking at the moment. If you plan to implement this let me know and I can provide some additional tips if you get stuck:
import os, re, subprocess, sys
from docutils import nodes, statemachine
from docutils.parsers.rst import directives
from sphinx.util.compat import Directive, directive_dwim
class SvnInclude(Directive):
has_content = True
required_arguments = 1
optional_arguments = 0
final_argument_whitespace = False
def _setup_repo(self, repo):
env = self.state.document.settings.env
path = os.path.normpath(env.doc2path(env.docname, base=None))
cache = os.path.join(os.path.dirname(path), '_svncache')
root = os.path.join(cache, re.sub('[\W\-]+', '_', repo))
if not os.path.exists(root):
os.makedirs(root)
subprocess.call(['svn', 'co', repo, root])
return root
def run(self):
root = self._setup_repo(self.arguments[0])
for path in self.content:
data = open(os.path.join(root, path), 'rb').read()
lines = statemachine.string2lines(data)
self.state_machine.insert_input(lines, path)
return []
def setup(app):
app.add_directive('svninclude', directive_dwim(SvnInclude))
Here is an example of the markup you'd include in your index.rst (or other file):
.. svninclude:: http://svn.domain.com/svn/project
one.rst
doc/two.rst
Where the paths one.rst and doc/two.rst are relative to the subversion url, for example http://svn.domain.com/svn/project/one.rst.
You'd of course want to package up the svninclude.py and get it installed in your Python path. Here's what I did to test it:
Added 'svninclude' to the extensions list in source/conf.py.
Placed svninclude.py in the project root.
Then ran:
% PYTHONPATH=. sphinx-build -b html ./source ./build

AppEngine and Django: including a template file

As the title suggests, I'm using Google App Engine and Django.
I have quite a bit of identical code across my templates and would like to reduce this by including template files. So, in my main application directory I have the python handler file, the main template, and the template I want to include in my main template.
I would have thought that including {% include "fileToInclude.html" %} would work on its own but that simply doesn't include anything. I assume I have to set something up, maybe using TEMPLATE_DIRS, but can't figure it out on my own.
EDIT:
I've tried:
TEMPLATE_DIRS = (os.path.join(os.path.dirname(__file__), 'templates'), )
But to no avail. I'll try some other possibilities too.

First, you should consider using template inheritance rather than the include tag, which is often appropriate but sometimes far inferior to template inheritance.
Unfortunately, I have no experience with App Engine, but from my experience with regular Django, I can tell you that you need to set your TEMPLATE_DIRS list to include the folder from which you want to include a template, as you indicated.

I found that it works "out of the box" if I don't load Templates first and render them with a Context object. Instead, I use the standard method shown in the AppEngine tutorial.

I am having the same problem and tracked it down into the ext.webapp package. In template.py, you'll find this comment on line 33:
Django uses a global setting for the directory in which it looks for templates.
This is not natural in the context of the webapp module, so our load method
takes in a complete template path, and we set these settings on the fly
automatically. Because we have to set and use a global setting on every
method call, this module is not thread safe, though that is not an issue
for applications.
See line 92 in the same file. You can see the template dirs being squashed:
directory, file_name = os.path.split(abspath)
new_settings = {
'TEMPLATE_DIRS': (directory,),
'TEMPLATE_DEBUG': debug,
'DEBUG': debug,
}
UPDATE: Here is the workaround which worked for me - http://groups.google.com/group/google-appengine/browse_thread/thread/c3e0e4c47e4f3680/262b517a723454b6?lnk=gst&q=template_dirs#262b517a723454b6

I've done the following to get around using includes:
def render(file, map={}):
return template.render(
os.path.join(os.path.dirname(__file__), '../templates', file), map)
table = render("table.html", {"headers": headers, "rows": rows})
final = render("final.html", {"table": table})
self.response.out.write(final)