We Keep Coding

Django Rest framework - I am trying to convert a property received in Response object to JSON object and iterate through it. But response is string

In views.py VENDOR_MAPPER is list of dictionary each dictionary has id, name, placeholder and autocommit key. I also tried sending json instead of Response object.
resp_object = {}
resp_object['supported_vendors'] = VENDOR_MAPPER
resp_object['vendor_name'] = ""
resp_object['create_vo_entry'] = False
resp_object['generate_signature_flag'] = False
resp_object['branch_flag'] = False
resp_object['trunk_flag'] = False
resp_object['branch_name'] = ""
resp_object['advisory'] = ""
data = {'data': resp_object}
return Response(data)
On home.html I am accessing the vendors_supported which is list and iterate through it, however instead of object i am getting string as type of variable.
var supported_vendors = "{{data.supported_vendors|safe}}";
console.log(supported_vendors);
console.log("Supported_vendors ", supported_vendors);
console.log("Supported_vendors_type:", typeof(supported_vendors));
data.supported_vendors|safe (django template tagging) is used to remove the unwanted characters in the response i have also tried without safe, but still the type was string
also tried converted as well as parse the response but type is shown as string
var supported_vendors = "{{data.supported_vendors}}";
console.log(JSON.parse(supported_vendors));
console.log(JSON.stringify(supported_vendors));
Output generated, i have printed the response type and values i get, also converting using JSON.parse and JSON.stringify did not work and output every time was string
[1]: https://i.stack.imgur.com/DuSMb.png
I want to convert the property into javascript object and perform some computations

You can try this instead ,
return HttpResponse(json.dumps(data),
content_type="application/json")

I got the answer:
var supported_vendors = "{{data.supported_vendors}}";
Converted the above line to
var supported_vendors = {{data.supported_vendors}};
removed quotes from the variable

add parameters inside url

i want to build some function that read a url from txt file, then save it to some variable, then add some values inside the url between another values
example of the url: https://domains.livedns.co.il/API/DomainsAPI.asmx/NewDomain?UserName=apidemo#livedns.co.il&Password=demo
lets say i want to inject some values between UserName and Password and save it into file again and use it later.
i started to write the function and play with urllib parser but i still doesnt understand how to do that.
what i tried until now:
def dlastpurchase():
if os.path.isfile("livednsurl.txt"):
apikeyfile = open("livednsurl.txt", "r")
apikey = apikeyfile.read()
url_parse = urlsplit(apikey)
print(url_parse.geturl())
dlastpurchase()
Thanks in advance for every tip and help

A little bit more complex example that I believe you will find interesting and also enjoy improving it (while it takes care of some scenarios, it might be lacking in some). Also functional to enable reuse in other cases. Here we go
assuming we have a text file, named 'urls.txt' that contains this url
https://domains.livedns.co.il/API/DomainsAPI.asmx/NewDomain?UserName=apidemo#livedns.co.il&Password=demo
from os import error
from urllib.parse import urlparse, parse_qs, urlunparse
filename = 'urls.txt'
function to parse the url and return its query parameters as well as the url object, which will be used to reconstruct the url later on
def parse_url(url):
"""parse a given url and return its query parameters
Args:
url (string): url string to parse
Returns:
parsed (tupple): the tupple object returned by urlparse
query_parameters (dictionary): dictionary containing the query parameters as keys
"""
try :
# parse the url and get the queries parameters from there
parsed = urlparse(url)
# parse the queries and return the dictionary containing them
query_result = parse_qs(parsed.query)
return (query_result, parsed)
except(error):
print('something failed !!!')
print(error)
return False
function to add a new query parameter or to replace an existing one
def insert_or_replace_word(query_dic, word,value):
"""Insert a value for the query parameters of a url
Args:
query_dic (object): the dictionary containing the query parameters
word (string): the query parameter to replace or insert values for
value (string): the value to insert or use as replacement
Returns:
result (string):the result of the insertion or replacement
"""
try:
query_dic[word] = value
return query_dic
except (error):
print('Something went wrong {0}'.format(error))
function to format the query parameter and get them ready to reconstruct the new url
def format_query_strings(query_dic):
"""format the final query dictionaries ready to be used to construct a new url and construct the new url
Args:
query_dic (dictionary): final query dictionary after insertion or update
"""
final_string = ''
for key, value in query_dic.items():
#unfortunatly, query params from parse_qs are in list, so remove them before creating the final string
if type(value) == list:
query_string = '{0}={1}'.format(key, value[0])
final_string += '{0}&'.format(query_string)
else:
query_string = '{0}={1}'.format(key, value)
final_string += '{0}&'.format(query_string)
# this is to remove any extra & inserted at the end of the loop above
if final_string.endswith('&'):
final_string = final_string[:len(final_string)-1]
return final_string
we check out everything works by reading in text file, performing above operation and then saving the new url to a new file
with open(filename) as url:
lines = url.readlines()
for line in lines:
query_params,parsed = parse_url(line)
new_query_dic = insert_or_replace_word(query_params,'UserName','newUsername')
final = format_query_strings(new_query_dic)
#here you have to pass an iterable of lenth 6 in order to reconstruct the url
new_url_object = [parsed.scheme,parsed.netloc,parsed.path,parsed.params,final,parsed.fragment]
#this reconstructs the new url
new_url = urlunparse(new_url_object)
#create a new file and append the link inside of it
with open('new_urls.txt', 'a') as new_file:
new_file.writelines(new_c)
new_file.write('\n')

You don't have to use fancy tools to do that. Just split the url based on "?" Character. Then, split the second part based on "&" character. Add your new params to the list you have, and merge them with the base url you get.
url = "https://domains.livedns.co.il/API/DomainsAPI.asmx/NewDomain?UserName=apidemo#livedns.co.il&Password=demo"
base, params = url.split("?")
params = params.split("&")
params.insert(2, "new_user=yololo&new_passwd=hololo")
for param in params:
base += param + "&"
base = base.strip("&")
print(base)
I did it like this since you asked for inserting to a specific location. But url params are not depends on the order, so you can just append at the end of the url for ease. Or, you can edit the parameters from the list I show.

Passing value to nested JSON in Python?

I try to pass a variable to nested JSON in Python script.
Script as below,
import requests, request
group = request.form['grp']
zon = request.form['zone']
load = { "extra_vars": {
"g_name": "' +str(group)+ '",
"z_name": "' +str(zon)+ '"
}
}
----
--
-
However when i post the value to the API, it seem i post word '+str(group)+' and '+str(zon)+' instead the actual value that assign under declared variable.
Since i'm very new in Python programming, does passing value to nested JSON is allow in Python?

Try the following:
group = request.form['grp']
zon = request.form['zone']
load = { "extra_vars": {
"g_name": f"{group}",
"z_name": f"{zon}"
}
}

You can pass variables into a string using f-strings and brackets around your variable (note {group}):
>>> group = "my_group"
>>> {"g_name": f"'{group}'"}
{'g_name': "'my_group'"}
Or doing simple string concatenation also, which is what you almost done in your code (but just did not properly close the ' character using "'":
>>> "'" + str(group) + "'"
"'my_group'"
All in all here's your code adapted:
load = { "extra_vars": {
"g_name": f"'{group}'",
"z_name": f"'{zon}'"
}
}

Convert boto3 BaseCondition to its string representation

I am trying to convert boto3 dynamoDB conditional expressions (using types from boto3.dynamodb.conditions) to its string representation. Of course this could be hand coded but naturally I would prefer to be able to find something developed by AWS itself.
Key("name").eq("new_name") & Attr("description").begins_with("new")
would become
"name = 'new_name' and begins_with(description, 'new')"
I have been checking in the boto3 and boto core code but so far no success, but I assume it must exist somewhere in the codebase...

In the boto3.dynamodb.conditions module there is a class called ConditionExpressionBuilder. You can convert a condition expression to string by doing the following:
condition = Key("name").eq("new_name") & Attr("description").begins_with("new")
builder = ConditionExpressionBuilder()
expression = builder.build_expression(condition, is_key_condition=True)
expression_string = expression.condition_expression
expression_attribute_names = expression.attribute_name_placeholders
expression_attribute_values = expression.attribute_value_placeholders
I'm not sure why this isn't documented anywhere. I just randomly found it looking through the source code at the bottom of this page https://boto3.amazonaws.com/v1/documentation/api/latest/_modules/boto3/dynamodb/conditions.html.
Unfortunately, this doesn't work for the paginator format string notation, but it should work for the Table.query() format.

From #Brian's answer with
ConditionExpressionBuilder I had to add the Dynamodb's {'S': 'value'} type notation before the query execution.
I changed it with expression_attribute_values[':v0'] = {'S': pk_value}, where :v0 is the first Key/Attr in the condition. Not sure but should work for next values (:v0, :v1, :v2...).
Here is the full code, using pagination to retrieve only part of data
from boto3.dynamodb.conditions import Attr, Key, ConditionExpressionBuilder
from typing import Optional, List
import boto3
client_dynamodb = boto3.client("dynamodb", region_name="us-east-1")
def get_items(self, pk_value: str, pagination_config: dict = None) -> Optional[List]:
if pagination_config is None:
pagination_config = {
# Return only first page of results when no pagination config is not provided
'PageSize': 300,
'StartingToken': None,
'MaxItems': None,
}
condition = Key("pk").eq(pk_value)
builder = ConditionExpressionBuilder()
expression = builder.build_expression(condition, is_key_condition=True)
expression_string = expression.condition_expression
expression_attribute_names = expression.attribute_name_placeholders
expression_attribute_values = expression.attribute_value_placeholders
# Changed here to make it compatible with dynamodb typing
python expression_attribute_values[':v0'] = {'S': pk_value}
paginator = client_dynamodb.get_paginator('query')
page_iterator = paginator.paginate(
TableName="TABLE_NAME",
IndexName="pk_value_INDEX",
KeyConditionExpression=expression_string,
ExpressionAttributeNames=expression_attribute_names,
ExpressionAttributeValues=expression_attribute_values,
PaginationConfig=pagination_config
)
for page in page_iterator:
resp=page
break
if ("Items" not in resp) or (len(resp["Items"]) == 0):
return None
return resp["Items"]
EDIT:
I used this question to get string representation for Dynamodb Resource's query, which is not compatible (yet) with dynamodb conditions, but then I found a better solution from Github (Boto3)[https://github.com/boto/boto3/issues/2300]:
Replace paginator with the one from meta
dynamodb_resource = boto3.resource("dynamodb")
paginator = dynamodb_resource.meta.client.get_paginator('query')
And now I can simply use Attr and Key

Django get url regex by name

I have a case where I have defined some Django url patterns and now I want to retrieve the regular expression associated with a given pattern. I want that because I want to pass these regular expressions to the client so I can check urls in client as well ( I'm talking about browser side history manipulation ) and fire appropriate handlers ( in JavaScript ) when there is a match.
For example if I have:
# urls.py
urlpatterns = patterns("",
url(r"^$", Index.as_view(), name="index"),
url(r"^user/", include("User.urls", namespace="User")),
)
# User/urls.py
urlpatterns = patterns("",
url(r"^profile/(?P<slug>.*)$", GetProfile.as_view(), name="get_profile")
)
then I need the following function:
>>> get_regex("User:get_profile")
'^user/profile/(?P<slug>.*)$'
( or however Django translates it ). Note that I'm using namespaces. Any ideas? Django1.5.
Also I've managed to write a function that returns the urlpattern object associated with a passed name, however doing url.regex.pattern returns '^profile/(?P<slug>.*)$. So as you can see there is no leading ^user/.

There are several javascript reverse implementations out there.
http://djangojs.readthedocs.org/en/latest/djangojs.html#reverse-urls
https://github.com/version2/django-js-reverse
It's not the regex, but you could test the urls in your client code just like you do in the server, so it's even better in my opinion.
EDIT: Since you need to ignore URL arguments, you could get an idea from the source of django-js here. It already removes optional URL arguments, so it's probably very similar to what you describe.
The code iterates over every pattern removing the ?P from each argument subregex so you could just replace them with .*.
The point is you have in that source every regex you could possibly need to do your implementation. See the global patterns in lines 24-29.

So I've tried few things and finally I came up with my own solution. First I convert urlpatterns into a form which JavaScript understands:
import re
converter = re.compile(r"\?P<.*?>")
def recursive_parse(urlpatterns, lst):
for pattern in urlpatterns:
obj = {
"pattern": converter.sub("", pattern.regex.pattern)
}
if hasattr(pattern, "name") and pattern.name is not None:
obj["name"] = pattern.name
if hasattr(pattern, "namespace"):
obj["namespace"] = pattern.namespace
if hasattr(pattern, "url_patterns"):
if "urls" not in obj:
obj["urls"] = []
recursive_parse(pattern.url_patterns, obj["urls"])
lst.append(obj)
def generate_paths(urlpatterns):
paths = []
recursive_parse(urlpatterns, paths)
return paths
Then I call generate_paths(urlpatterns), JSON-stringify the result and pass it to JavaScript (note that in JavaScript I have to convert regular expressions as strings to RegExp objects). In JavaScript I have
var recursive_check = function(url, patterns, names, args) {
var l = patterns.length;
for (var i = 0; i < l; i++) {
var pat = patterns[i],
match = pat.pattern.exec(url);
pat.lastIndex = 0;
if (match) {
names.push(pat.namespace || pat.name);
var f = match.shift(),
url = url.replace(f, ""),
ml = match.length;
for (var j = 0; j < ml; j++) {
args.push(match[j]);
}
if (pat.urls) {
recursive_check(url, pat.urls, names, args);
}
break;
}
}
};
var fire_handler = function(url) {
var names = [], args = [];
recursive_check(url, patterns, names, args);
// do something...
};
Now in // do something... I can do something with names and args. For example I can keep a dictionary of named handlers, I can search for a handler (based on names) and call it with args.
That's the solution that works for me. Converting urlpatterns to JavaScript patterns might not be perfect (since converter seems to be a bit too simplified) but it works in most simple cases.

Try this:
from django.core.urlresolvers import get_resolver
resolver = get_resolver(None)
url = resolver.reversed_dict.getlist('get_profile')
if url:
pattern = url[0][1]

Not an answer but might be useful to someone else looking at this.
The following generates a list of all, complete url patterns in the Django project, including for nested URLRegexResolvers, based on #Freakish's code.
import re
from django.core.urlresolvers import get_resolver
converter = re.compile(r"\?P<.*?>")
def trim_leading_caret(s):
return s[1:] if s.startswith('^') else s
def recursive_parse(urlpatterns, lst, prefix=None):
for pattern in urlpatterns:
path = (prefix or '') + trim_leading_caret(converter.sub("", pattern.regex.pattern))
if hasattr(pattern, "url_patterns"):
recursive_parse(pattern.url_patterns, lst, path)
else:
lst.append('^' + path)
def generate_paths(urlpatterns):
paths = []
recursive_parse(urlpatterns, paths)
return paths
generate_paths(get_resolver(None))

As far as I understood, you want to be able to return the regex expression (and not the url) of a given view.
This is my sketch of solution:
The function url returns an instance of RegexURLResolver. This class does store the regex, because it calls LocaleRegexProvider on __init__ (in this line and this line).
So, I think that you can
reverse search the view plus namespace
get the tuple of that view from the tuple of tuples urlpatterns
return _regex of the first argument, LocaleRegexProvider._regex (or regex()), of the tuple respective to the view.
I'm not sure this works (didn't tested), neither that it is the best solution, but at least you have some links on where Django stores the regex.