Is it it possible to get the names of the variables in a Mako Template before rendering?
from mako.template import Template
bar = Template("${foo}")
# something like:
# >> print bar.fields()
# ['foo']
Use case:
We have configuration files whereby we specify the metadata from the database to be displayed on a webpage. The client can pick one of a few hundred different pieces of named metadata. There are N slots the client can configure but we don't know in advance which pieces of metadata a particular client would like to have filled in on the form. Because if this when rendering the form we need to know, in advance, what variable names we need to pass for this clients template.
We had thought of having a consistent dictionary with all possible values and passing that in each time but it was unworkable as new available fields are added frequently to the underlying pool of available metadata the client could pick.
Because of this we had hoped to use Mako to template the config files but I can't figure out how to determine with the field values in the template would be so that I can build a full-formed Context to pass to the template.
Unfortunately there is no easy way to get the names of the variables from a template object.
Fortunately there is the mako.codegen._Identifiers class and the sole purpose of its objects is to keep track of the variables during the compilation process.
Unfortunately it is buried deep beneath the Mako API surface, and it is gone after the compilation is done.
Fortunately you can get it without setting up everything Mako sets up when it compiles templates. All you need is the parse tree that you can get by using mako.lexer.Lexer.
Anyway here is the code:
from mako import lexer, codegen
lexer = lexer.Lexer("${foo}", '')
node = lexer.parse()
# ^ The node is the root element for the parse tree.
# The tree contains all the data from a template
# needed for the code generation process
# Dummy compiler. _Identifiers class requires one
# but only interested in the reserved_names field
compiler = lambda: None
compiler.reserved_names = set()
identifiers = codegen._Identifiers(compiler, node)
# All template variables can be found found using this
# object but you are probably interested in the
# undeclared variables:
# >>> print identifiers.undeclared
# set(['foo'])
Chasing Mako variables is no fun. I knocked together this little function to extract the variables from a template - use & improve as you like.
def ListMakoVariables(template):
'''
Extract Mako variables from template.
'''
start = 'XXXX${'
stop = '}YYYY'
makovars = []
splitReady = template.replace('${',start).replace('}',stop)
startParts = splitReady.split('XXXX')
for startStr in startParts:
if '}' in startStr:
makovars.append(startStr.split('YYYY')[0])
vars = set(makovars)
return vars, makovars
FWIW, makovars are in-order, vars are unique but not in order.
Related
I am trying to make use of a column's value as a radio button's choice using below code
Forms.py
#retreiving data from database and assigning it to diction list
diction = polls_datum.objects.values_list('poll_choices', flat=True)
#initializing list and dictionary
OPTIONS1 = {}
OPTIONS = []
#creating the dictionary with 0 to no of options given in list
for i in range(len(diction)):
OPTIONS1[i] = diction[i]
#creating tuples from the dictionary above
#OPTIONS = zip(OPTIONS1.keys(), OPTIONS1.values())
for i in OPTIONS1:
k = (i,OPTIONS1[i])
OPTIONS.append(k)
class polls_form(forms.ModelForm):
#retreiving data from database and assigning it to diction list
options = forms.ChoiceField(choices=OPTIONS, widget = forms.RadioSelect())
class Meta:
model = polls_model
fields = ['options']
Using a form I am saving the data or choices in a field (poll_choices), when trying to display it on the index page, it is not reflecting until a server restart.
Can someone help on this please
of course "it is not reflecting until a server restart" - that's obvious when you remember that django server processes are long-running processes (it's not like PHP where each script is executed afresh on each request), and that top-level code (code that's at the module's top-level, not in a function) is only executed once per process when the module is first imported. As a general rule: don't do ANY db query at a module's top-level or at the top-level of a class statement - at best you'll get stale data, at worse it will crash your server process (if you're doing query before everything has been properly setup by django, or if you're doing query based on a schema update before the migration has been applied).
The possible solutions are either to wait until the form's initialisation to setup your field's choices, or to pass a callable as the formfield's choices options, cf https://docs.djangoproject.com/en/2.1/ref/forms/fields/#django.forms.ChoiceField.choices
Also, the way you're building your choices list is uselessly complicated - you could do it as a one-liner:
OPTIONS = list(enumerate(polls_datum.objects.values_list('poll_choices', flat=True))
but it's also very brittle - you're relying on the current db content and ordering for the choice value when you should use the polls_datum's pk instead (which is garanteed to be stable).
And finally: since you're working with what seems to be a related model, you may want to use a ModelChoiceField instead.
For future reference:
What version of Django are you using?
Have you read up on the documentation of ModelForms? https://docs.djangoproject.com/en/2.1/topics/forms/modelforms/
I'm not sure what you're trying to do with diction to dictionary to tuple. I think you could skip a step there and your future self will thank you for that.
Try to follow some tutorials and understand why certain steps are being taken. I can see from your code that you're rather new to coding or Python and there's room for improvement. Not trying to talk you down, but I'm trying to push you into the direction of becoming a better developer ;-)
REAL ANSWER:
That being said, I think the solution is to write the loading of the data somewhere in your form model, rather than 'loose' in forms.py. See bruno's answer for more information on this.
If you want to reload the data on each request that loads the form, you should create a function that gets called every time the form is loaded (for example in the form's __init__ function).
I'm creating an app. that will generate math problems. They're specific problems where some parameters can be altered. Each problem will be different, and require a different method to solve (all of which will be programatically implemented).
For example:
models.py
import random
from django.db import models
class Problem(models.Model):
unformattedText = models.TextField()
def __init__(self, unformattedText, genFunction, *args, **kwargs):
super(Problem, self).__init__(*args, **kwargs)
self.unformatedText = unformattedText
self.genFunction = genFunction
def genQAPair():
self.genFunction(self.unformattedText)
views.py
def genP1(text):
num_1 = random.randrange(0, 100)
num_2 = random.randrange(0, 100)
text.format((num_1, num_2))
return {'question':text, 'answer':num_1 - num_2}
def genP2(text, lim=4):
num_1 = random.randrange(0, lim)
text.format(num_1)
return {'question':text, 'answer':num_1*40}
p1 = Problem(
unformattedText='Sally has {} apples. Frank takes {}. How many apples does Sally have?',
genFunction=genP1
)
p1.save()
p2 = Problem(
unformattedText='John jumps {} feet into the air. How long does it take for him to age?',
genFunction=genP2
)
p2.save()
When I try this, the function isn't actually saved. Django just saves the integer 1. When I initiate an instance of the model, the function is there as intended, but apparently only 1 is saved to the database.
Bonus question: I'm actually beginning to question whether or not I even need Django models for this. I'm using Django because it's super easy to get everything onto a webpage. Is there a better way to do this? (Maybe store the text of each problem in a JSON file and the generating functions in some separate script.)
The persistence layer for a Django application is the database, and the database schema is specified by your model definitions. In this case you've only defined a single field in your model, unformattedText; you haven't specified any storage for the corresponding function. Your self.genFunction = genFunction is just creating an attribute on an object in memory; it won't be persisted.
There are various possible ways to store the function. You could store it as raw text; you could store it as a pickle blob; you could store the function path and name (e.g. "my.path.to.problems.genP1"); or do something else. In any case, you'll need to create a database field for that information.
Here is a rough outline of an example solution using the function path:
models.py
class Problem(models.Model):
unformattedText = models.TextField()
genPath = models.TextField()
views.py
import importlib
def problem_view(request, problem_id):
problem = Problem.objects.get(id=problem_id)
gen_path, gen_name = problem.genPath.rsplit(".", 1)
gen_module = importlib.import_module(gen_path)
gen_function = getattr(gen_module, gen_name)
context = gen_function(problem.unformattedText)
return render(request, 'template.html', context)
Only you can determine if you need to use a database at all. If you only have a few fixed questions then you could just stuff everything into a Python file and be done with it. But there are advantages to using Django's models, including the ability to use the admin.
There are a couple of options, depending on the actual task. I ranged them starting with the most safe option to the most dangerous (but flexible):
1. Store function identifiers:
You can store genP1 and genP2 as 'genP1' and 'genP2' - i.e. by name (or you can use any other unique identifier).
Pros:
You can validate user input and execute trusted code only, because in this case you control almost everything.
You can easily debug your functions, because they are part of your system.
Cons:
You need to define all your functions in the code. That means if you want to add new function, you need to redeploy your application.
If you are storing function names, you need to manually import module (or package) with the functions and call them.
If you are storing identifiers, you need to define a mapping {identifier: path to actual function}.
2. Use DSL
You can write your own DSL (or use existing)
Pros:
You can add new functions at runtime without redeploying application.
You can control which code can user execute.
You can see source code for your functions.
Cons:
It is hard to write safe and flexible DSL, especially if you want to call some python code from it.
It is hard to debug huge functions.
3. Serialize them
You can serialize functions using pickle
Pros:
You can add new functions at runtime without redeploying application.
Easier than writing own DSL.
Cons:
Unsafe - you must not execute untrusted code. If you allow users to create their own functions, serialization is not your way - define (or use existing) safe DSL instead.
It might be impossible to show source python code for the serialized function. For more information: How can I get the source code of a Python function?
It is hard to debug huge functions.
4. Just store actual Python code
Just store the python source code in the DB as a string.
Pros:
You can add new functions at runtime without redeploying application.
You can see source code without any additional processing.
Easier than writing own DSL.
Cons:
Unsafe - you must not execute untrusted code. If you allow users to create their own functions, storing source code is not your way - define (or use existing) safe DSL instead.
It is hard to debug huge functions.
I'm trying to get a list of variables from a Jinja2 template.
test1.j2:
some-non-relevant-content
{{var1}}
{% include 'test2.j2' %}
test2.j2:
another-text
{{var2}}
I can get variables from test1 easily:
env = Environment(loader=FileSystemLoader(searchpath='./Templates'))
src_t = env.loader.get_source(env, 'test1.j2')[0]
parsed_t = env.parse(source=src_t)
t_vars = meta.find_undeclared_variables(ast=parsed_t)
Problem is, I can only get variables from the parent template with get_source.
Obviously, I can not feed class template object to parse method as well.
Is there any way to build the full list? {'var1', 'var2'} in my case.
Ideally by using Jinja2 API. Minimum custom code.
Found a way to code that without a big pain.
meta.find_referenced_templates helps to load all child templates when applied recursively. When done, it's trivial to get variables from all templates in a single list.
I have a base template for when a user is logged in, and on that base template, I need to add user specific options in a drop down menu. This drop down menu with options must be constant across all handlers, i.e., any time the base template is invoked (extended) with a child template.
Other than performing the necessary DB query, assigning the query results to a variable, and passing that variable to every handler (there are many), how can I consolidate this into one query and one variable, which gets passed directly to the base template? I am using jinja2 templates as well.
I would hate to do something so cumbersome in exchange for something far more simple and maintainable.
Any ideas? Thanks.
EDIT
So I still haven't found anything that's exactly what I'm looking for; however, I decided to at least make some headway in the interim. So, I made a custom decorator that takes a view's returned dict() and appends the appropriate data to it. For example:
def get_base_data(func):
def wrapper(request):
d = func(request)
user_id = request.user.id # used in query
contact_group_data = ContactGroups.query.filter(...criteria...).all()
d['contact_group_data'] = contact_group_data
return d
return wrapper
Now, I can at least decorate each method very concisely and simply by putting:
#view_config(...)
#get_base_data
def my_handler(request):
pass # rest of code...
This is one of most inobvious things in Pyramid and took a while to find for me, too.
You can modify the global template context in BeforeRender event.
http://docs.pylonsproject.org/projects/pyramid/en/latest/narr/hooks.html#using-the-before-render-event
Alternatively, you could use class based views, inherit all your views from one base view class which has get_base_data(), then the class instance is passed to the template context to all your views and then you could extract the data with {{ view.get_base_data }}.
http://ruslanspivak.com/2012/03/02/class-based-views-in-pyramid/
I vouch for the latter approach as it is more beautiful, predictable and easier to maintain engineering wise.
I've looked at documentation, and have searched Google extensively, and haven't found a solution to my problem.
This is my readRSS function (note that 'get' is a method of Kenneth Reitz's requests module):
def readRSS(name, loc):
linkList = []
linkTitles = list(ElementTree.fromstring(get(loc).content).iter('title'))
linkLocs = list(ElementTree.fromstring(get(loc).content).iter('link'))
for title, loc in zip(linkTitles, linkLocs):
linkList.append((title.text, loc.text))
return {name: linkList}
This is one of my MongoAlchemy classes:
class Feed(db.Document):
feedname = db.StringField(max_length=80)
location = db.StringField(max_length=240)
lastupdated = datetime.utcnow()
def __dict__(self):
return readRSS(self.feedname, self.location)
As you can see, I had to call the readRSS function within a function of the class, so I could pass self, because it's dependent on the fields feedname and location.
I want to know if there's a different way of doing this, so I can save the readRSS return value to a field in the Feed document. I've tried assigning the readRSS function's return value to a variable within the function __dict__ -- that didn't work either.
I have the functionality working in my app, but I want to save the results to the Document to lessen the load on the server (the one I am getting my RSS feed from).
Is there a way of doing what I intend to do or am I going about this all wrong?
I found out the answer. I needed to make use of a computed_field decorator, where the first argument was the structure of my return value and deps was a set which contained the fields that this field was dependent on. I then passed the dependent fields into a function's arguments and there you have it.
#fields.computed_field(db.KVField(db.StringField(), db.ListField(db.TupleField(db.StringField()))), deps=[feedname, location])
def getFeedContent(a=[feedname, location]):
return readRSS(a['feedname'], a['location'])
Thanks anyway, everyone.