We Keep Coding

Updating and saving dict in django with post requests

I am trying to make a responsive tournament bracket with python/django and using $.post requests to update a tournament dict - which I pass to a 'bracket' template as a dictionary, render, then update by $.posting the passed in variable to a nother view, which updates, saves to server, then redirects to 'bracket' view.
I am just starting to make some progress, but having issues with reformatting of the bracket object.
Little more detail
The bracket is initialized in python (in my views.py) in a bracket view, but I am calling in the view a Tournament class that I got from here.
the Tournament class takes a list of players and then generates a dictionary corresponding to the games with the method t.generate_bracket().
I kind of restructure this and then pass it into a bracket view for displaying - I display the restructured array in my template but also pass in the un-restructured one
I have little radio buttons that are bound to a $.post 'update_bracket' view. In the JS $.post, I send the array to the view, where I will call a t.play_game() method to update the bracket. Then - instead of a JSON or HTTP response and subsequent re-population of the bracket client side (which I am not good enough at frontend to do), I save the updated bracket to a JSONField on an employee model (extends the logged in user) and HTTPResponseRedirect to the initial 'bracket' view, where I have a check to see if the Employee has a bracket saved to them, then use that value saved to the JSONField as the new bracket dict to be displayed - instead of generating from scratch with t.generate_bracket()
Here are my bracket view, my update bracket view, the bracket.html with json $.post() to update_bracket and some of the Tournament and other classes I am utilizing
Code
Initial bracket view
#login_required#(login_url='/accounts/login/')
def bracket(request):
'''
:param request:
:return:
'''
emp = Employee.objects.get(user = request.user)
emp_bracket = emp.guess
players = [p.id for p in Pet.objects.all()]
t = Tournament(players)
if not emp_bracket:
t.generate_bracket('single', 0)
else:
t.games = emp_bracket
tournament_games = t.games
nested_tournament_games = {}
nested_tournament_games['rounds']={}
for g_id,g in tournament_games.items():
#....I do restructuring of the tournament_games dict
context = {'arr':tournament_games, 'nested':nested_tournament_games['rounds']}#{'latest_question_list': latest_question_list}
return render(request, 'madness/bracket.html', context)
.post in bracket template
$('input[type=radio][name=bStatus]').change(function() {
alert(this.value);
$.post('{% url "update_bracket" %}',
{ bracketData: JSON.stringify("{{arr}}") },
function(data, status, xhr) {
console.log(JSON.stringify(data));
var nested = JSON.stringify(data);
}).done(function() { })
.fail(function(jqxhr, settings, ex) { alert('failed, ' + ex); });
});
update_bracket view
#csrf_exempt
def update_bracket(request):
bracketData = request.POST['bracketData']
print(json.loads(bracketData))
#I plan on using the play_game() method here, but first I have to just get the basics down
emp = Employee.objects.get(user = request.user)
emp.guess = bracketData
emp.save()
# return HttpResponse(json.loads(bracketData))
return HttpResponseRedirect(reverse('bracket'))
Tournament class I am using
class Tournament:
def __init__(self, players):
self.players = players
self.player_count = len(players)
self.games = {}
...
def generate_bracket(self, bracket_class, randomize):
if bracket_class != "single":
print("Illegal bracket class")
quit()
self.n_size = int(math.ceil(math.log(self.player_count, 2)))
self.underflow = int(math.pow(2, self.n_size) - self.player_count)
if randomize:
random.shuffle(self.players)
for i in range(self.underflow):
self.players.append(None)
self.num_of_rounds = int(math.log(len(self.players), 2))
self.games = generate_bracket_rec(self.players, self.num_of_rounds)
...
def generate_bracket_rec(players, num_of_rounds):
games_map = {}
next_players = []
assert len(players) > 0
if len(players) == 1:
return games_map
gm=1
for i in range(int(len(players) / 2)):
player1_id = i
player2_id = len(players) - 1 - i
if players[player2_id] is None:
next_players.append(players[player1_id])
else:
round=str(1+num_of_rounds-int(math.log(len(players), 2)))
game_id = "round_"+round + "_game_" + str(gm)#str(i + 1)
g = Game((players[player1_id], players[player2_id]), game_id)
g.round, g.game = round, gm
games_map[g.id] = g#{'players':g.players, 'winner':g.winner, 'loser':g.loser, 'done':g.done}
# next_players.append((g, "winner")) #the winner of that game
# next_players.append('winner of %s and %s' % (g.players))
next_players.append('winner of Round %s Game %s' % (g.round, g.game))
gm += 1
d = {}
d.update(games_map)
d.update(generate_bracket_rec(next_players, num_of_rounds).items())
return d
Errors in formatting
I have tried different ways of dealing with the JSON tournament object, as it stands now the error I have is happening in js in $.post - Uncaught SyntaxError: Invalid or unexpected token, so code is not even reaching the update_bracket view.
DevTools shows
$.post('/update_bracket',
{ bracketData: JSON.stringify("{'round_1_game_1': round_1_game_1: 2 vs 3
Done: False, 'round_2_game_1': round_2_game_1: 1 vs winner of Round 1 Game 1
Done: False}") },
Does anyone know how I need to be dealing with this tournament object in the lifecycle
python (generated as dict)-->template (passed in as context var) -->python (from .post of that passed-in-template-var)-->save to django JSONField-->retrieved from JSONField (restart of cycle)
EDIT - JSON Decode error
After following advice of what to do with the passing-to-template, I get a new error in update_bracket. I am able to send the data there, but then when I try to loads() it I get JSONDecode error. Do you know how I can deal with this?
request.POST['bracketData']
'{"round_1_game_1": {"players": [2, 3], "winner": null, "loser": null, "done": false}, "round_2_game_1": {"players": [1, "winner of Round 1 Game 1"], "winner": null, "loser": null, "done": false}}'
json.loads(request.POST['bracketData'])
Traceback (most recent call last):
File "C:\Program Files\JetBrains\PyCharm Community Edition 2019.2.1\helpers\pydev\_pydevd_bundle\pydevd_exec2.py", line 3, in Exec
exec(exp, global_vars, local_vars)
File "<input>", line 1, in <module>
File "C:\Users\aiden\AppData\Local\Programs\Python\Python37-32\lib\json\__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "C:\Users\aiden\AppData\Local\Programs\Python\Python37-32\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\aiden\AppData\Local\Programs\Python\Python37-32\lib\json\decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)

When you do {{ arr }} in your template, you're just telling python to print the arr object as string. Since arr is a dictionary (it's tournament_games), you're printing something like {'key': 'value'} which isn't a JSON string (JSON strings need double-quotes ").
And then you wrap it in "" making it a javascript string and your javascript calls JSON.stringify() on it. But that doesn't work, because stringify() expects a javascript object, not a string.
Since all you want is post a JSON string, create that string in python and print that in your template. In your context:
'arr': json.dumps(tournament_games)
and in your template:
bracketData: "{{ arr }}"

How to decode dataTables Editor form in python flask?

I have a flask application which is receiving a request from dataTables Editor. Upon receipt at the server, request.form looks like (e.g.)
ImmutableMultiDict([('data[59282][gender]', u'M'), ('data[59282][hometown]', u''),
('data[59282][disposition]', u''), ('data[59282][id]', u'59282'),
('data[59282][resultname]', u'Joe Doe'), ('data[59282][confirm]', 'true'),
('data[59282][age]', u'27'), ('data[59282][place]', u'3'), ('action', u'remove'),
('data[59282][runnerid]', u''), ('data[59282][time]', u'29:49'),
('data[59282][club]', u'')])
I am thinking to use something similar to this really ugly code to decode it. Is there a better way?
from collections import defaultdict
# request.form comes in multidict [('data[id][field]',value), ...]
# so we need to exec this string to turn into python data structure
data = defaultdict(lambda: {}) # default is empty dict
# need to define text for each field to be received in data[id][field]
age = 'age'
club = 'club'
confirm = 'confirm'
disposition = 'disposition'
gender = 'gender'
hometown = 'hometown'
id = 'id'
place = 'place'
resultname = 'resultname'
runnerid = 'runnerid'
time = 'time'
# fill in data[id][field] = value
for formkey in request.form.keys():
exec '{} = {}'.format(d,repr(request.form[formkey]))

This question has an accepted answer and is a bit old but since the DataTable module seems being pretty popular among jQuery community still, I believe this approach may be useful for someone else. I've just wrote a simple parsing function based on regular expression and dpath module, though it appears not to be quite reliable module. The snippet may be not very straightforward due to an exception-relied fragment, but it was only one way to prevent dpath from trying to resolve strings as integer indices I found.
import re, dpath.util
rxsKey = r'(?P<key>[^\W\[\]]+)'
rxsEntry = r'(?P<primaryKey>[^\W]+)(?P<secondaryKeys>(\[' \
+ rxsKey \
+ r'\])*)\W*'
rxKey = re.compile(rxsKey)
rxEntry = re.compile(rxsEntry)
def form2dict( frmDct ):
res = {}
for k, v in frmDct.iteritems():
m = rxEntry.match( k )
if not m: continue
mdct = m.groupdict()
if not 'secondaryKeys' in mdct.keys():
res[mdct['primaryKey']] = v
else:
fullPath = [mdct['primaryKey']]
for sk in re.finditer( rxKey, mdct['secondaryKeys'] ):
k = sk.groupdict()['key']
try:
dpath.util.get(res, fullPath)
except KeyError:
dpath.util.new(res, fullPath, [] if k.isdigit() else {})
fullPath.append(int(k) if k.isdigit() else k)
dpath.util.new(res, fullPath, v)
return res
The practical usage is based on native flask request.form.to_dict() method:
# ... somewhere in a view code
pars = form2dict(request.form.to_dict())
The output structure includes both, dictionary and lists, as one could expect. E.g.:
# A little test:
rs = jQDT_form2dict( {
'columns[2][search][regex]' : False,
'columns[2][search][value]' : None,
'columns[2][search][regex]' : False,
} )
generates:
{
"columns": [
null,
null,
{
"search": {
"regex": false,
"value": null
}
}
]
}
Update: to handle lists as dictionaries (in more efficient way) one may simplify this snippet with following block at else part of if clause:
# ...
else:
fullPathStr = mdct['primaryKey']
for sk in re.finditer( rxKey, mdct['secondaryKeys'] ):
fullPathStr += '/' + sk.groupdict()['key']
dpath.util.new(res, fullPathStr, v)

I decided on a way that is more secure than using exec:
from collections import defaultdict
def get_request_data(form):
'''
return dict list with data from request.form
:param form: MultiDict from `request.form`
:rtype: {id1: {field1:val1, ...}, ...} [fieldn and valn are strings]
'''
# request.form comes in multidict [('data[id][field]',value), ...]
# fill in id field automatically
data = defaultdict(lambda: {})
# fill in data[id][field] = value
for formkey in form.keys():
if formkey == 'action': continue
datapart,idpart,fieldpart = formkey.split('[')
if datapart != 'data': raise ParameterError, "invalid input in request: {}".format(formkey)
idvalue = int(idpart[0:-1])
fieldname = fieldpart[0:-1]
data[idvalue][fieldname] = form[formkey]
# return decoded result
return data

django returns none on web page

when I run this function in Django my output is none. what is wrong with the news() function?
Code:
import feedparser
from django.http import HttpResponse
def news():
YahooContent = feedparser.parse ("http://news.yahoo.com/rss/")
for feed in YahooContent.entries:
print feed.published
print feed.title
print feed.link + "\n"
return
def html(request):
html = "<html><body> %s </body></html>" % news()
return HttpResponse(html)
Error:
webpage shows None

You are printing the results, not returning them. In fact, the return statement will return None, just like all methods that don't have a return statement.
You should build the string in your method itself, like this:
def html(request):
head = '<html><body>'
foot = '</body></html>'
entries = []
for entry in feedparser.parse("http://news.yahoo.com/rss/").entries:
entries.append('{}<br />{}<br />{}<br />'.format(entry.published,
entry.title, entry.link))
return HttpResponse(head+''.join(entries)+foot)
Can you explain your code a little bit?
Imagine you have a list of "entries", like this:
entries = [a, b, c]
Each entry has a .published, .title, .link attribute that you want to print as a list in HTML.
You can do this easily by looping through it and using the print statement:
print('<ul>')
for entry in entries:
print('<li>{0.published} - {0.title} - {0.link}</li>'.format(entry))
print('</ul>')
However, what we need here is to send this data to the browser as a HTML response. We can build the HTML string by replacing print with a string that we keep adding to, like this:
result = '<ul>'
for entry in entries:
result += '<li>{0.published} - {0.title} - {0.link}</li>'.format(entry)
result += '</ul>'
This will work but is inefficient and slow, it is better to collect the strings in a list, and then join them together. This is what I did in my original answer:
result = ['<ul>']
for entry in entries:
result.append('<li>{0.published} - {0.title} - {0.link}</li>'.format(entry))
result.append('</li>')
Then I just joined them all up with a header and a footer, and then combined each individual string in the list with a space, and returned it as the response:
return HttpResponse('<html><body>'+''.join(result)+'</body></html>')

obviously your news() method returns nothing..
The right way should be:
def news():
YahooContent = feedparser.parse ("http://news.yahoo.com/rss/")
result = ''
for feed in YahooContent.entries:
result += feed.published + '\n'
result += feed.title + '\n'
result += feed.link + "\n"
return result
def html(request):
html = "<html><body> %s </body></html>" % news()
return HttpResponse(html)

TypeError: 'NoneType' object is not iterable - Python

I know this question has been asked before but I can't seem to get mine working. Can someone help me out with this?
import sys
class Template:
def processVariable(self, template, data):
first = template.find('{{')
second = template.find('}}')
second = second + 2
out = template[first+second]
assert[out.startswith('{{')]
out = out.strip('{}')
rest = template[second:]
if out in data:
newstring = data[out]
return(rest, newstring)
else:
print "Invalid Data Passed it"
t = Template()
vars = {
'name': 'Jane',
'course': 'CS 1410',
'adjective': 'shimmering'
}
(rest, out) = t.processVariable('{{name}} is in {{course}}', vars)
This is the error I am getting:
File "template.py", line 28, in <module>
(rest, out) = t.processVariable('{{name}} is in {{course}}', vars)
TypeError: 'NoneType' object is not iterable
I understand the NoneType but is it because of my for loop or did I just miss something simple? Thanks in advance!
My code will be passed into a field so that my teacher can run a code against it and it will pass or fail. This is what his code will run:
import Candidate
t = Candidate.Template()
vars = {
'name': 'Jane',
'course': 'CS 1410',
'adjective': 'shimmering'
}
(rest, out) = t.processVariable('{{course}} is {{adjective}}', vars)
print 'rest is: [' + rest + ']'
print 'out is : [' + out + ']
what it should return is:
rest is: [ is in {{course}}]
out is : [Jane]
it will return Yes or No if it worked.

You didn't return anything from your else part, and hence your function will by default return None.
Ad of course, you cannot unpack a None into two variables. Try returning a tuple from your else part like this:
if out in data:
newstring = data[out]
return (rest, newstring)
else:
print "Invalid Data Passed it"
return (None, None)

There are errors in your code.
This:
out = template[first+second]
assert[out.startswith('{{')]
Should be:
out = template[first:second]
assert out.startswith('{{')
And:
else:
print "Invalid Data Passed it"
Should probably be:
else:
raise ValueError('Invalid data')
Also this functionality is already in Python:
from string import Template
t = Template('$name is in $course')
vars = {
'name': 'Jane',
'course': 'CS 1410',
'adjective': 'shimmering'
}
print(t.substitute(vars))
Output:
Jane is in CS 1410

trying to automate translation on babelfish with python

I have modified a python babelizer to help me to translate english to chinese.
## {{{ http://code.activestate.com/recipes/64937/ (r4)
# babelizer.py - API for simple access to babelfish.altavista.com.
# Requires python 2.0 or better.
#
# See it in use at http://babel.MrFeinberg.com/
"""API for simple access to babelfish.altavista.com.
Summary:
import babelizer
print ' '.join(babelizer.available_languages)
print babelizer.translate( 'How much is that doggie in the window?',
'English', 'French' )
def babel_callback(phrase):
print phrase
sys.stdout.flush()
babelizer.babelize( 'I love a reigning knight.',
'English', 'German',
callback = babel_callback )
available_languages
A list of languages available for use with babelfish.
translate( phrase, from_lang, to_lang )
Uses babelfish to translate phrase from from_lang to to_lang.
babelize(phrase, from_lang, through_lang, limit = 12, callback = None)
Uses babelfish to translate back and forth between from_lang and
through_lang until either no more changes occur in translation or
limit iterations have been reached, whichever comes first. Takes
an optional callback function which should receive a single
parameter, being the next translation. Without the callback
returns a list of successive translations.
It's only guaranteed to work if 'english' is one of the two languages
given to either of the translation methods.
Both translation methods throw exceptions which are all subclasses of
BabelizerError. They include
LanguageNotAvailableError
Thrown on an attempt to use an unknown language.
BabelfishChangedError
Thrown when babelfish.altavista.com changes some detail of their
layout, and babelizer can no longer parse the results or submit
the correct form (a not infrequent occurance).
BabelizerIOError
Thrown for various networking and IO errors.
Version: $Id: babelizer.py,v 1.4 2001/06/04 21:25:09 Administrator Exp $
Author: Jonathan Feinberg <jdf#pobox.com>
"""
import re, string, urllib
import httplib, urllib
import sys
"""
Various patterns I have encountered in looking for the babelfish result.
We try each of them in turn, based on the relative number of times I've
seen each of these patterns. $1.00 to anyone who can provide a heuristic
for knowing which one to use. This includes AltaVista employees.
"""
__where = [ re.compile(r'name=\"q\">([^<]*)'),
re.compile(r'td bgcolor=white>([^<]*)'),
re.compile(r'<\/strong><br>([^<]*)')
]
# <div id="result"><div style="padding:0.6em;">??</div></div>
__where = [ re.compile(r'<div id=\"result\"><div style=\"padding\:0\.6em\;\">(.*)<\/div><\/div>', re.U) ]
__languages = { 'english' : 'en',
'french' : 'fr',
'spanish' : 'es',
'german' : 'de',
'italian' : 'it',
'portugese' : 'pt',
'chinese' : 'zh'
}
"""
All of the available language names.
"""
available_languages = [ x.title() for x in __languages.keys() ]
"""
Calling translate() or babelize() can raise a BabelizerError
"""
class BabelizerError(Exception):
pass
class LanguageNotAvailableError(BabelizerError):
pass
class BabelfishChangedError(BabelizerError):
pass
class BabelizerIOError(BabelizerError):
pass
def saveHTML(txt):
f = open('page.html', 'wb')
f.write(txt)
f.close()
def clean(text):
return ' '.join(string.replace(text.strip(), "\n", ' ').split())
def translate(phrase, from_lang, to_lang):
phrase = clean(phrase)
try:
from_code = __languages[from_lang.lower()]
to_code = __languages[to_lang.lower()]
except KeyError, lang:
raise LanguageNotAvailableError(lang)
html = ""
try:
params = urllib.urlencode({'ei':'UTF-8', 'doit':'done', 'fr':'bf-res', 'intl':'1' , 'tt':'urltext', 'trtext':phrase, 'lp' : from_code + '_' + to_code , 'btnTrTxt':'Translate'})
headers = {"Content-type": "application/x-www-form-urlencoded","Accept": "text/plain"}
conn = httplib.HTTPConnection("babelfish.yahoo.com")
conn.request("POST", "http://babelfish.yahoo.com/translate_txt", params, headers)
response = conn.getresponse()
html = response.read()
saveHTML(html)
conn.close()
#response = urllib.urlopen('http://babelfish.yahoo.com/translate_txt', params)
except IOError, what:
raise BabelizerIOError("Couldn't talk to server: %s" % what)
#print html
for regex in __where:
match = regex.search(html)
if match:
break
if not match:
raise BabelfishChangedError("Can't recognize translated string.")
return match.group(1)
#return clean(match.group(1))
def babelize(phrase, from_language, through_language, limit = 12, callback = None):
phrase = clean(phrase)
seen = { phrase: 1 }
if callback:
callback(phrase)
else:
results = [ phrase ]
flip = { from_language: through_language, through_language: from_language }
next = from_language
for i in range(limit):
phrase = translate(phrase, next, flip[next])
if seen.has_key(phrase): break
seen[phrase] = 1
if callback:
callback(phrase)
else:
results.append(phrase)
next = flip[next]
if not callback: return results
if __name__ == '__main__':
import sys
def printer(x):
print x
sys.stdout.flush();
babelize("I won't take that sort of treatment from you, or from your doggie!",
'english', 'french', callback = printer)
## end of http://code.activestate.com/recipes/64937/ }}}
and the test code is
import babelizer
print ' '.join(babelizer.available_languages)
result = babelizer.translate( 'How much is that dog in the window?', 'English', 'chinese' )
f = open('result.txt', 'wb')
f.write(result)
f.close()
print result
The result is to be expected inside a div block . I modded the script to save the html response . What I found is that all utf8 characters are turned to nul . Do I need take special care in treating the utf8 response ?

I think you need to use:
import codecs
codecs.open
instead of plain open, in your:
saveHTML
method, to handle utf-8 docs. See the Python Unicode Howto for a complete explanation.