In the following example from wsgi.org is cur_named copied:
def __call__(self, environ, start_response):
script_name = environ.get('SCRIPT_NAME', '')
path_info = environ.get('PATH_INFO', '')
for regex, application in self.patterns:
match = regex.match(path_info)
if not match:
continue
extra_path_info = path_info[match.end():]
if extra_path_info and not extra_path_info.startswith('/'):
# Not a very good match
continue
pos_args = match.groups()
named_args = match.groupdict()
cur_pos, cur_named = environ.get('wsgiorg.routing_args', ((), {}))
new_pos = list(cur_pos) + list(pos_args)
new_named = cur_named.copy() # Why copy()?
new_named.update(named_args)
environ['wsgiorg.routing_args'] = (new_pos, new_named)
environ['SCRIPT_NAME'] = script_name + path_info[:match.end()]
environ['PATH_INFO'] = extra_path_info
return application(environ, start_response)
return self.not_found(environ, start_response)
Why not to call ur_named.update(named_args) directly?
Do you know where cur_named dict came from? Just imaging something like the following:
SOME_CONFIG = {
'some_key': ((..., ...), {...}),
...
}
environ['wsgiorg.routing_args'] = SOME_CONFIG['some_key']
Now when you update new_named in-place you are actually updating inner dictionary inside SOME_CONFIG which will bring your data to other requests. The safe way is to copy dictionary unless you are sure it's not needed.
Related
I am trying to parse some LUA functions with pyparsing. It also works for almost everything I need except one case where the last parameter is just a word.
So this is my code. This is my first parser using pyparsing but I did my best to structure it logically:
To explain my comments within the code:
trigger_async(<object>, <name>, <param>)
trigger_async(<parameters>)
<param> = <name> = <type>
def parse_events_from_text(text):
variable = Word(alphanums + "-_.:()")
# Entity on which the event will be triggered
obj = variable.setResultsName("object")
# Name of the event
name = (quotedString + Optional(Suppress("..") + variable)).setResultsName("name")
# Parameter List of the event
paramName = variable.setResultsName("name")
paramType = variable.setResultsName("type")
param = Group(paramName + Suppress("=") + paramType).setResultsName("parameter")
paramList = Group(Optional(Suppress("{") + ZeroOrMore(delimitedList(param)) + Suppress("}"))).setResultsName("parameters")
function_parameter = obj | name | paramList
# Function Start
trigger = "trigger"
async = Optional("_async")
# Function Call
functionOpening = Combine(trigger + async + "(").setResultsName("functionOpening")
functionCall = ZeroOrMore(Group(functionOpening + delimitedList(function_parameter) + Suppress(")")))
resultsList = functionCall.searchString(text)
results = []
for resultsL in resultsList:
if len(resultsL) != 0:
if resultsL not in results:
results.append(resultsL)
return results
So the parser was written for those kinds of events:
trigger(self._entity, 'game:construction:changed', { entity = target })`
trigger_async(entity, 'game:heal:healer_damaged', { healer = entity })`
trigger_async(entity, 'game:heal:healer_damaged', { healer = entity, entity = target, test = party})`
trigger_async(entity, 'game:heal:healer')`
trigger(entity.function(), 'game:heal:healer', {})`
But the problem is if there aren't any curly braces:
trigger(entity, 'game:heal:healer', entity.test)
it won't work because of my declared variable
variable = Word(alphanums + "-_.:()")
where braces are allowed so the parser is confused with the last one which is "missing" for the function end. If I would write
trigger(entity,'game:heal:healer',entity.test))
it would work.
I sat down and wanted to rewrite the parser but I dont know how? Somehow I must tell that it is only valid if the variable has 1 open bracket and 1 closing bracket like so:
trigger(entity,'game:heal:healer',entity.test(input))
else don't eat up that closing brace.
trigger(entity,'game:heal:healer',entity.test) <-- Variable, don't eat it!
I have a file containing a text like this:
loadbalancer {
upstream application1 {
server 127.0.0.1:8082;
server 127.0.0.1:8083;
server 127.0.0.1:8084;
}
upstream application2 {
server 127.0.0.1:8092;
server 127.0.0.1:8093;
server 127.0.0.1:8094;
}
}
Does anyone know, how could I extract variables like below:
appList=["application1","application2"]
ServerOfapp1=["127.0.0.1:8082","127.0.0.1:8083","127.0.0.1:8084"]
ServerOfapp2=["127.0.0.1:8092","127.0.0.1:8093","127.0.0.1:8094"]
.
.
.
and so on
If the lines you want always start with upstream and server this should work:
app_dic = {}
with open('file.txt','r') as f:
for line in f:
if line.startswith('upstream'):
app_i = line.split()[1]
server_of_app_i = []
for line in f:
if not line.startswith('server'):
break
server_of_app_i.append(line.split()[1][:-1])
app_dic[app_i] = server_of_app_i
app_dic should then be a dictionary of lists:
{'application1': ['127.0.0.1:8082', '127.0.0.1:8083', '127.0.0.1:8084'],
'application2': ['127.0.0.1:8092', '127.0.0.1:8093', '127.0.0.1:8094']}
EDIT
If the input file does not contain any newline character, as long as the file is not too large you could write it to a list and iterate over it:
app_dic = {}
with open('file.txt','r') as f:
txt_iter = iter(f.read().split()) #iterator of list
for word in txt_iter:
if word == 'upstream':
app_i = next(txt_iter)
server_of_app_i=[]
for word in txt_iter:
if word == 'server':
server_of_app_i.append(next(txt_iter)[:-1])
elif word == '}':
break
app_dic[app_i] = server_of_app_i
This is more ugly as one has to search for the closing curly bracket to break. If it gets any more complicated, regex should be used.
If you are able to use the newer regex module by Matthew Barnett, you can use the following solution, see an additional demo on regex101.com:
import regex as re
rx = re.compile(r"""
(?:(?P<application>application\d)\s{\n| # "application" + digit + { + newline
(?!\A)\G\n) # assert that the next match starts here
server\s # match "server"
(?P<server>[\d.:]+); # followed by digits, . and :
""", re.VERBOSE)
string = """
loadbalancer {
upstream application1 {
server 127.0.0.1:8082;
server 127.0.0.1:8083;
server 127.0.0.1:8084;
}
upstream application2 {
server 127.0.0.1:8092;
server 127.0.0.1:8093;
server 127.0.0.1:8094;
}
}
"""
result = {}
for match in rx.finditer(string):
if match.group('application'):
current = match.group('application')
result[current] = list()
if current:
result[current].append(match.group('server'))
print result
# {'application2': ['127.0.0.1:8092', '127.0.0.1:8093', '127.0.0.1:8094'], 'application1': ['127.0.0.1:8082', '127.0.0.1:8083', '127.0.0.1:8084']}
This makes use of the \G modifier, named capture groups and some programming logic.
This is the basic method:
# each of your objects here
objText = "xyz xcyz 244.233.233.2:123"
listOfAll = re.findall(r"/\b(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?):[0-9]{1,5}/g", objText)
for eachMatch in listOfAll:
print "Here's one!" % eachMatch
Obviously that's a bit rough around the edges, but it will perform a full-scale regex search of whatever string it's given. Probably a better solution would be to pass it the objects themselves, but for now I'm not sure what you would have as raw input. I'll try to improve on the regex, though.
I believe this as well can be solved with re:
>>> import re
>>> from collections import defaultdict
>>>
>>> APP = r'\b(?P<APP>application\d+)\b'
>>> IP = r'server\s+(?P<IP>[\d\.:]+);'
>>>
>>> pat = re.compile('|'.join([APP, IP]))
>>>
>>>
>>> scan = pat.scanner(s)
>>> d = defaultdict(list)
>>>
>>> for m in iter(scan.search, None):
group = m.lastgroup
if group == 'APP':
keygroup = m.group(group)
continue
else:
d[keygroup].append(m.group(group))
>>> d
defaultdict(<class 'list'>, {'application1': ['127.0.0.1:8082', '127.0.0.1:8083', '127.0.0.1:8084'], 'application2': ['127.0.0.1:8092', '127.0.0.1:8093', '127.0.0.1:8094']})
Or similarly with re.finditer method and without pat.scanner:
>>> for m in re.finditer(pat, s):
group = m.lastgroup
if group == 'APP':
keygroup = m.group(group)
continue
else:
d[keygroup].append(m.group(group))
>>> d
defaultdict(<class 'list'>, {'application1': ['127.0.0.1:8082', '127.0.0.1:8083', '127.0.0.1:8084'], 'application2': ['127.0.0.1:8092', '127.0.0.1:8093', '127.0.0.1:8094']})
I believe to use jinja2 with bottle one simply uses jinja2_template instead of template:
e.g. bottle.jinja2_template("mytemplate", dict(name=value, name2=value2))
However if one needs the i18n jinja extension how is that best specified to also do
....install_gettext_translations(
? Is that done automatically with
bottle.jinja2_template("mytemplate", dict(name=value, name2=value2), template_lookup=['templates'],'template_settings'= {'extensions':['jinja2.ext.i18n'],'autoescape': True }))
? Thanks.
Upon further reflection, I think I may need to overide the prepare method in class Jinja2Template to add the env.install_gettext_translations( ???
More info ,if I were doing ....install_gettext_translations( manually, perhaps:
tenv = Environment(extensions=['jinja2.ext.i18n'])
tenv.install_gettext_translations(gettransobj())
import gettext
import locale
def gettransobj():
loc = locale.getlocale()
# change to reflect where your mo files are
mofilename = "res/messages_%s.mo" % locale.getdefaultlocale()[0][0:2]
try:
trans = gettext.GNUTranslations(open( mofilename, "rb" ) )
except IOError:
trans = gettext.NullTranslations()
return trans
OR for babel translations obj something like
.....install_gettext_translations(gettransobj(),newstyle=True)
import babel
import locale
def gettransobj():
loc = locale.getlocale()
mofilename = "res/messages_%s.mo" % locale.getdefaultlocale()[0][0:2]
trans = babel.support.Translations(open( mofilename, "rb" ) )
If this code is somewhat correct, not sure where to put it? Not very familiar with jinja2. Just once at top of program or per bottle.jinja2_template call.
On a different note, if someone needs to do extraction using babel, see jinja2.ext.babel_extract
Another approach is getting trans obj with something like:
return gettext.translation(domain, localedir=localedir,languages=languages, codeset='utf-8')
from jinja2 import FileSystemBytecodeCache, Environment
bcc = FileSystemBytecodeCache('/tmp', '%s.cache')
template_settings = {'filters': {
'tojson': json_util.dumps
},
'bytecode_cache': bcc,
'extensions': ['jinja2.ext.i18n'],
'languages': ['en_US']
}
import bottle
from bottle import Jinja2Template
from babel.support import Translations
TEMPLATE_PATH = bottle.TEMPLATE_PATH
DEBUG = bottle.DEBUG
TEMPLATES = {}
class Jinja2BabelTemplate(Jinja2Template):
def prepare(self, filters=None, tests=None,
languages=['en_US'],
globals={}, **kwargs):
from jinja2 import Environment, FunctionLoader
self.env = Environment(loader=FunctionLoader(self.loader), **kwargs)
#BABEL: this is where we load and install our translations from babel
translations = Translations.load('translations', languages)
self.env.install_gettext_translations(translations)
if filters: self.env.filters.update(filters)
if tests: self.env.tests.update(tests)
if globals: self.env.globals.update(globals)
if self.source:
self.tpl = self.env.from_string(self.source)
else:
self.tpl = self.env.get_template(self.filename)
def jinja_template(*args, **kwargs):
tpl = args[0] if args else None
adapter = Jinja2BabelTemplate
lookup = kwargs.pop('template_lookup', TEMPLATE_PATH)
languages = kwargs.pop('languages', None)
tplid = (id(languages), tpl,)
if tplid not in TEMPLATES or DEBUG:
settings = kwargs.pop('template_settings', {})
if languages: settings.update({'languages': languages})
if isinstance(tpl, adapter):
TEMPLATES[tplid] = tpl
if settings: TEMPLATES[tplid].prepare(**settings)
elif "\n" in tpl or "{" in tpl or "%" in tpl or '$' in tpl:
TEMPLATES[tplid] = adapter(source=tpl, lookup=lookup, **settings)
else:
TEMPLATES[tplid] = adapter(name=tpl, lookup=lookup, **settings)
if not TEMPLATES[tplid]:
abort(500, 'Template (%s) not found' % tpl)
for dictarg in args[1:]: kwargs.update(dictarg)
return TEMPLATES[tplid].render(kwargs)
template = functools.partial(jinja_template,
template_settings=template_settings)
#and now you should be able to use the template function above with different languages
#get('/home')
def home():
data = {}
return template('home.html', data, languages=['en_US'])
def images_custom_list(args, producer_data):
tenant, token, url = producer_data
url = url.replace(".images", ".servers")
url = url + '/' + 'detail'
output = do_request(url, token)
output = output[0].json()["images"]
custom_images_list = [custom_images for custom_images in output
if custom_images["metadata"].get('user_id', None)]
temp_image_list = []
for image in custom_images_list:
image_temp = ( { "status": image["status"],
"links": image["links"][0]["href"],
"id": image["id"], "name": image["name"]} )
temp_image_list.append(image_temp)
print json.dumps(temp_image_list, indent=2)
def image_list_detail(args, producer_data):
tenant, token, url = producer_data
url = url.replace(".images", ".servers")
uuid = args['uuid']
url = url + "/" + uuid
output = do_request(url, token)
print output[0]
I am trying to make the code more efficient and clean looking by utilizing the Python's function decoration. Since these 2 functions share the same first 2 lines, how could I make a function decorator with these 2 lines and have these 2 functions be decorated it?
here's a way to solve it:
from functools import wraps
def fix_url(function):
#wraps(function)
def wrapper(*args, **kwarg):
kwarg['url'] = kwarg['url'].replace(".images", ".servers")
return function(*args, **kwarg)
return wrapper
#fix_url
def images_custom_list(args, tenant=None, token=None, url=None):
url = url + '/' + 'detail'
output = do_request(url, token)
output = output[0].json()["images"]
custom_images_list = [custom_images for custom_images in output
if custom_images["metadata"].get('user_id', None)]
temp_image_list = []
for image in custom_images_list:
image_temp = ( { "status": image["status"],
"links": image["links"][0]["href"],
"id": image["id"], "name": image["name"]} )
temp_image_list.append(image_temp)
print json.dumps(temp_image_list, indent=2)
#fix_url
def image_list_detail(args, tenant=None, token=None, url=None):
uuid = args['uuid']
url = url + "/" + uuid
output = do_request(url, token)
print output[0]
sadly for you, you may notice that you need to get rid of producer_data, but have it split in multiple arguments because you cannot factorize that part of the code, as you'll anyway need to split it again in each of the functions. I chose to use keyword arguments (by setting a default value to None), but you could use positional arguments as well, your call.
BTW, note that it's not making the code more efficient, though it's helping in making it a bit more readable (you know that you're changing the URL the same way for both methods, and when you fix the URL changing part, it's done the same way everywhere), but it's making 2 more function calls each time you call the function, so it's in no way more "efficient".
N.B.: It's basically based over #joel-cornett's example (I wouldn't have used #wraps otherwise, just plain old double function decorator), I just specialized it. (I don't think he deserves a -1)
Please at least +1 his answer or accept it.
But I think a simpler way to do it would be:
def fix_url(producer_data):
return (producer_data[0], producer_data[1], producer_data[2].replace(".images", ".servers"))
def images_custom_list(args, producer_data):
tenant, token, url = fix_url(producer_data)
# stuff ...
def image_list_detail(args, producer_data):
tenant, token, url = fix_url(producer_data)
# stuff ...
which uses a simpler syntax (no decorator) and does only one more function call.
Like this:
from functools import wraps
def my_timesaving_decorator(function):
#wraps(function)
def wrapper(*args, **kwargs):
execute_code_common_to_multiple_function()
#Now, call the "unique" code
#Make sure that if you modified the function args,
#you pass the modified args here, not the original ones.
return function(*args, **kwargs)
return wrapper
I'm fairly new to programming. I'm trying to write two class methods that will take a string, '{{name}} is in {{course}}' , and replace {{name}} and {{course}} with their respective Key values in a dictionary. So:
t = Template()
vars = {
'name': 'Jane',
'course': 'CS 1410'
}
out = t.process('{{name}} is in {{course}}', vars)
print 'out is: [' + out + ']'
Would print:
Jane is in CS 1410
My code goes as:
class Template:
def processVariable(self, template, data):
print template
assert(template.startswith('{{'))
start = template.find("{{")
end = template.find("}}")
out = template[start+2:end]
assert(out != None)
assert(out in data)
return data[out]
def process(self, template, data):
output = ""
check = True
while check == True:
start = template.find("{{")
end = template.find("}}")
output += template[:start]
output += self.processVariable(template[start:end+2], data)
template = template.replace(template[:end+2], "")
for i in template:
if i == "}}":
check = True
output += template
return output
t = Template()
vars = {
'name': 'Jane',
'course': 'CS 1410'
}
out = t.process('{{name}} is in {{course}}', vars)
print 'out is: [' + out + ']'
When I run the code, I get the following output:
{{name}}
{{course}}
Traceback (most recent call last):
File "C:some/filepath/name.py", line 46, in <module>
out = t.process('{{name}} is in {{course}}', vars)
File "C:some/filepath/name.py", line 28, in process
output += self.processVariable(template[start:end+2], data)
File "C:some/filepath/name.py", line 8, in processVariable
assert(template.startswith('{{'))
AssertionError
I just don't understand why im getting that assertion error if template is '{{course}}'
Edit:
The purpose making the code this way, was to bring in any dictionary and string, so that I can create a simple social network. Otherwise much simpler methods would be proficient.
You weren't actually getting the assertion error when template was {{course}}, which you can see for yourself if you change the process method to include some simple print statements, e.g.:
def process(self, template, data):
# ...
output += template[:start]
print "Processing, template is currently:"
print template
output += self.processVariable(template[start:end+2], data)
# ...
return output
The actual problem was that check never became false. You can replace your if test with something like this, and then your function runs fine:
if not '}}' in template:
check = False
Marius beat me to the answer to your question, but I just wanted to point out an easier way to do (almost) the same thing. Ofcourse, if you're just trying to learn than the hard way is usually better.
vars = {
'name': 'Jane',
'course': 'CS 1410'
}
out = '{name} is in {course}'.format(**vars)
print out