We Keep Coding

Is there a way to map data from form to insert in to database without explicitly defining each variable?

I have made a really long form with the help of colander alchemy and deform.
This form has 100 or so fields and currently the only way I know to add the data back to the database once form is submitted is to explicitly re-define each variable and then add that to the database but there must be a better way.
#my schema
class All(colander.MappingSchema):
setup_schema(None,atr)
atrschema =atr.__colanderalchemy__
setup_schema(None,chemicals)
chemicalsschema =chemicals.__colanderalchemy__
setup_schema(None,data_aquisition)
data_aquisitionschema =data_aquisition.__colanderalchemy__
setup_schema(None,depositor)
depositorschema =depositor.__colanderalchemy__
setup_schema(None,dried_film)
dried_filmschema =dried_film.__colanderalchemy__
form = All()
form = deform.Form(form,buttons=('submit',))
# this is how I get it to work by redefining each field but there must be a better way
if 'submit' in request.POST:
prism_material = request.params['prism_material']
angle_of_incidence_degrees =
request.params['angle_of_incidence_degrees']
number_of_reflections = request.params['number_of_reflections']
prism_size_mm = request.params['prism_size_mm']
spectrometer_ID = 6
page = atr (spectrometer_ID=spectrometer_ID,prism_size_mm=prism_size_mm,number_of_reflections=number_of_reflections,angle_of_incidence_degrees=angle_of_incidence_degrees,prism_material=prism_material)
request.dbsession.add(page)
Would like to somehow just be able to remap all of that 'multi dictionary' that is returned back to the database?

So, you have a dict (request.params) and want to pass the key-value pars from that dict to a function? Python has a way to do that using **kwargs syntax:
if 'submit' in request.POST:
page = Page(spectrometer_ID=6,**request.params)
request.dbsession.add(page)
(this works also because SQLAlchemy provides a default constructor which assigns the passed values to the mapped columns, no need to define it manually)
Of course, this is a naive approach which will only work for the simplest use-cases - for example, it may allow passing parameters not defined in your schema which may create a security problem; the field names in your schema must match the field names in your SQLAlchemy model; it may not work with lists (i.e. multiple values with the same name which you can access via request.params.get_all(name)).

Flask + WTForms, dynamically generated list of fields

I am making a Flask application that is essentially form-based and so I'm using WTForms and Flask-wtf.
I am currently refactoring my code so my whole form uses WTForms and there is a very dynamic part of one of the forms that I am unable to implement using WTForms. I have no clue how to do it, my initial ideas didn't work, I can't find references or tutorials covering my problem and so this is why I ask for help.
So, the form in question allows users to submit objects that consist of:
A label (StringField, easy)
A Description (TextAreaField, also easy; although I had trouble to make a default value work)
A list of property of the form (predicate, object), where predicate is taken from a pre-built list and object can basically be anything but each predicate will generate a specific object (for instance, the predicate "related to" will expect another object (that comes from a dropdown list) and the predicate "resource" will expect a http link of some sort). This list can be empty.
As you can guess I have trouble with the list. The way the code works right now, I get the label and description using wtforms, and the property list is generated using a config constant (that is used throughout the code so I only have one place to edit if I want to add new properties) and a dynamic menu in javascript that creates (here, for predicates) fields, that I can then get using flask.request.form object in the view function. All the hidden fields for predicates have the same name attribute and all the hidden fields for objects have the same name attribute.
Here is what the view of the form looks like, initialized with a few properties:
http://i.imgur.com/bfMG95s.png
Under the "Propriétés" label you have a dropdown to select the predicate, the second field is displayed or hidden depending on the selected predicate (can be a dropdown or a text field), and it is only when you click on "Ajouter propriété" ("Add property") that a new line is added in the tab below and the fields are generated.
I'd like not to have to change anything on this side because it works very well, makes the form very intuitive and is basically exactly what I want it to be from the user's end.
This is what my custom Form looks like right now (it doesn't work and properties stays empty whatever the number of fields I submit with the form):
class PropertyForm(Form):
property_predicate = HiddenField(
validators=[AnyOf(values=app.config["PROPERTY_LIST"].keys())]
)
property_object = HiddenField(
validators=[DataRequired()]
)
class CategoryForm(Form):
"""
Custom form class for creating a category with the absolute minimal
attributes (label and description)
"""
label = StringField(
"Nom de la categorie (obligatoire)",
validators=[DataRequired()]
)
description = TextAreaField(
"Description de la categorie (obligatoire)",
validators=[DataRequired()]
)
properties = FieldList(FormField(PropertyForm),validators=[Optional()])
And here is what I'd love to do in my views.py code (that I am currently refactoring):
def cat_editor():
cat_form = CategoryForm()
if request.method == "GET":
# Do GET stuff and display the form
return render_template("cateditor.html", form=cat_form, varlist=template_var_list)
else if request.method == "POST":
if cat_form.validate_on_submit():
# Get values from form
category_label = cat_form.label.data
category_description = cat_form.description.data
category_properties = cat_form.properties.data
# Do POST stuff and compute things
return redirect(url_for("index"))
else:
# form didn't validate so we return the form so the template can display the errors
return render_template("cateditor.html", form=cat_form,
template_var_list = template_var_list)
The basic structure works perfectly, it's just that damn dynamic list I can't get to work properly.
Getting label and description from the WTForms CategoryForm instance works fine, but properties always return an empty list. Ideally I'd love to be able to get a list of the form [(predicate1, property1), (predicate2, object2) ... ] when calling cat_form.properties.data (this is why I have a FieldList of FormFields with two HiddenField in each) but I'd have no problem having to build such a list from two list as long as it's using WTForms. Any idea? Thanks a lot :)

I found out what the problem was by playing around with FieldList objects and append_entry() to see what HTML code would Flask-wtf generate if I was to make a prepopulated property list.
My Javascript was generating hidden fields with all the same name, as from what I understood that WTForms is able to aggregate fields with the same name to create lists. Problem is, those similarly named fields were part of a FormField itself nested in a FieldList object name properties.
In order for the WTForms Form object to discern a set of hidden fields from another, when you nest FormFields inside a FieldList it prefixes the FormFields field names with "FieldList_name-index-". Which means what WTForms was expecting was something like
<input type="hidden", name="properties-0-property_predicate" value=...>
<input type="hidden", name="properties-0-property_object" value=...>
<input type="hidden", name="properties-1-property_predicate" value=...>
<input type="hidden", name="properties-1-property_object" value=...>
<input type="hidden", name="properties-2-property_predicate" value=...>
<input type="hidden", name="properties-2-property_object" value=...>
I modified my javascript so it generates the appropriate names. Now when I call cat_form.properties.data I have something that looks like:
[{"property_predicate": "comment", "property_object":"bleh"},
{"property_predicate": "comment", "property_object": "bleh2"}]
And that is exactly what I need. For some reason the form doesn't validate but at least I know how to make WTForms extract data my javascript-generated hidden fields, which is what the problem was.
Edit: Form validation happens because you have to insert a CSRF hidden input with your csrf to every subform you generate with the FormField.

Use jQuery for the more dynamic elements/ behavior in your form(s). Note that form fields have a hidden property (or method, depending e.g., if you're using bootstrap), allowing you to render everything you might need, but only show fields when these are necessary, and hiding them otherwise. Dynamically adding fields is a bit harder, but not really impossible. Is there a limit to the number of fields associated with properties? if yes, i'd just render the maximum number of fields (as long as it's reasonable, up to 5 seems OK, when you get to double digits as a maximum number of properties a user can add, rendering a bunch of fields you'll never use gets to be inelegant).
Here's a good place to see how that would work. Of course, you have another problem of choosing when to hide or show relevant fields, but that can also be handled by a javascript/jQuery script, using jQuery's .change() event. Something like this:
$("#dropdown").change(function () {
var chosen_val = $(this).val();
if (chosen_val == 'banana'){$('#property1').show();} else {$('#property1').hide();}
});
This code will probably not work, and is definitely lacking proper logic but should give you an idea of how to approach this issue using jQuery. Note that 'property1' field is always there, waiting to be shown if the user chooses the right dropdown value.

Pass an entire inital object in an edit form (data pre-population) in Django

I have a very complicated form and I choose to not use ModelForm since I needed flexibility and control over the fields. Since I am not using ModelForm, I can't simply do something like instance=order, where order = Order.objects.get(pk=1).
Currently I am pre-populating every field with initial in the forms.py as oppose to the views.py like this
self.fields['work_type'] = forms.ChoiceField(choices=Order.WORK_TYPE_CHOICES, initial=order.work_type)
But I was wondering if I could pass the entire order object to a form or do I have to declare initial to every field?
Is there a way to do something like
order_form = OrderEditForm(data=request.POST, initial=order)
in views.py?

I have a very complicated form and I choose to not use ModelForm since
I needed flexibility and control over the fields
Everything that you can do using a Form, you can do in a ModelForm such as adding new fields or over-riding attributes on the fields etc.
But I was wondering if I could pass the entire order object to a form
or do I have to declare initial to every field?
You can pass the order object into the form but you will still have to populate each field individually either in the forms or in the view function.
So in your view you would do something like this:
intial = {'order_number': order.number, 'order_id': order.id}
form = OrderForm(initial=initial)

The easiest way to prepopulate data to a form is passing a dictionary as first argument to de form constructor.
order_form = OrderEditForm(order.__dict__())
where __dict__() is a method that passes "order" object attributes to a dictionary with each attribute's name as a key and their content as value.
An example of how to "invent" such a method could be something like:
order_initial = Order.objects.filter(pk=order.pk).values()[0]
and then construct the form with:
order_form = OrderEditForm(order_initial)
Look at this example (how they populate values at "post" time):

For future reference to other people:
I have since found out after reading SO's comments and answers that it's better to use ModelForm even if you end up explicitly defining every field manually (using something like self.fields['foo'] = forms.CharField()).
In any case, if you are trying to pass a dictionary of current values in a form then the best (built-in) way to convert a model to a dictionary is actually using model_to_dict:
from django.forms.models import model_to_dict
order = Order.objects.get(pk=1)
dictionary = model_to_dict(order)
form = OrderEditForm(dictionary)
I got the solution from this blog. I hope this will be helpful for someone.

Django custom (multi)widget input validation

What is the correct method for validating input for a custom multiwidget in each of these cases:
if I want to implement a custom Field?
if I want to use an existing database field type (say DateField)?
The motivation for this comes from the following two questions:
How do I use django's multi-widget?
Django subclassing multiwidget
I am specifically interested in the fact that I feel I have cheated. I have used value_from_datadict() like so:
def value_from_datadict(self, data, files, name):
datelist = [widget.value_from_datadict(data, files, name + '_%s' % i) for i, widget in enumerate(self.widgets)]
try:
D = date(day=int(datelist[0]), month=int(datelist[1]), year=int(datelist[2]))
return str(D)
except ValueError:
return None
Which looks through the POST dictionary and constructs a value for my widget (see linked questions). However, at the same time I've tacked on some validation; namely if the creation of D as a date object fails, I'm returning None which will fail in the is_valid() check.
My third question therefore is should I be doing this some other way? For this case, I do not want a custom field.
Thanks.

You validate your form fields just like any other fields, implementing the clean_fieldname method in your form. If your validation logic spreads across many form fields (which is nto the same as many widgets!) you put it in your form's clean() method.
http://docs.djangoproject.com/en/1.2/ref/forms/validation/

According to the documentation, validation is the responsibility of the field behind the widget, not the widget itself. Widgets should do nothing but present the input for the user and pass input data back to the field.
So, if you want to validate data that's been submitted, you should write a validator.
This is especially important with MultiWidgets, as you can have more than one aspect of the data error out. Each aspect needs to be returned to the user for consideration, and the built in way to do that is to write validators and place them in the validators attribute of the field.
Contrary to the documentation, you don't have to do this per form. You can, instead, extend one of the built in forms and add an entry to default_validators.
One more note: If you're going to implement a MultiWidget, your form is going to pass some sort of 'compressed' data back to it to render. The docs say:
This method takes a single “compressed” value from the field and returns a list of “decompressed” values. The input value can be assumed valid, but not necessarily non-empty.
-Widgets
Just make sure you're handling that output correctly and you'll be fine.

Django Form Preview - How to work with 'cleaned_data'

Thanks to Insin for answering a previous question related to this one.
His answer worked and works well, however, I'm perplexed at the provision of 'cleaned_data', or more precisely, how to use it?
class RegistrationFormPreview(FormPreview):
preview_template = 'workshops/workshop_register_preview.html'
form_template = 'workshops/workshop_register_form.html'
def done(self, request, cleaned_data):
# Do something with the cleaned_data, then redirect
# to a "success" page.
registration = Registration(cleaned_data)
registration.user = request.user
registration.save()
# an attempt to work with cleaned_data throws the error: TypeError
# int() argument must be a string or a number, not 'dict'
# obviously the fk are python objects(?) and not fk_id
# but how to proceed here in an easy way?
# the following works fine, however, it seems to be double handling the POST data
# which had already been processed in the django.formtools.preview.post_post
# method, and passed through to this 'done' method, which is designed to
# be overidden.
'''
form = self.form(request.POST) # instansiate the form with POST data
registration = form.save(commit=False) # save before adding the user
registration.user = request.user # add the user
registration.save() # and save.
'''
return HttpResponseRedirect('/register/success')
For quick reference, here's the contents of the post_post method:
def post_post(self, request):
"Validates the POST data. If valid, calls done(). Else, redisplays form."
f = self.form(request.POST, auto_id=AUTO_ID)
if f.is_valid():
if self.security_hash(request, f) != request.POST.get(self.unused_name('hash')):
return self.failed_hash(request) # Security hash failed.
return self.done(request, f.cleaned_data)
else:
return render_to_response(self.form_template,
{'form': f, 'stage_field': self.unused_name('stage'), 'state': self.state},
context_instance=RequestContext(request))

I've never tried what you're doing here with a ModelForm before, but you might be able to use the ** operator to expand your cleaned_data dictionary into the keyword arguments expected for your Registration constructor:
registration = Registration (**cleaned_data)
The constructor to your model classes take keyword arguments that Django's Model meta class converts to instance-level attributes on the resulting object. The ** operator is a calling convention that tells Python to expand your dictionary into those keyword arguments.
In other words...
What you're doing currently is tantamount to this:
registration = Registration ({'key':'value', ...})
Which is not what you want because the constructor expects keyword arguments as opposed to a dictionary that contains your keyword arguments.
What you want to be doing is this
registration = Registration (key='value', ...)
Which is analogous to this:
registration = Registration (**{'key':'value', ...})
Again, I've never tried it, but it seems like it would work as long as you aren't doing anything fancy with your form, such as adding new attributes to it that aren't expected by your Registration constructor. In that case you'd likely have to modify the items in the cleaned_data dictionary prior to doing this.
It does seem like you're losing out on some of the functionality inherent in ModelForms by going through the form preview utility, though. Perhaps you should take your use case to the Django mailing list and see if there's a potential enhancement to this API that could make it work better with ModelForms.
Edit
Short of what I've described above, you can always just extract the fields from your cleaned_data dictionary "by hand" and pass those into your Registration constructor too, but with the caveat that you have to remember to update this code as you add new fields to your model.
registration = Registration (
x=cleaned_data['x'],
y=cleaned_data['y'],
z=cleaned_data['z'],
...
)