How to, provide URL parameters using github-flask for {+path}? - python

First, I am a total n00b to Python. I am using github-flask, and flask obviously, to pull data from the GitHub API. I am trying to use the contents_url and retrieve a file. The URL from the GitHub API is something like:
// json
{
...
"contents_url": "https://api.github.com/repos/<org_name>/<repo_name>/contents/{+path}"
...
}
... and when I try and give that to the github-flask instance I get a TypeError, "TypeError: request() got an unexpected keyword argument 'path'" using:
# python
contents = github.get(repo['contents_url'], path='.gitignore')
I'm pretty sure I am missing something simple. I don't have to resort to string manipulation do I?

Python's recommended string interpolation is the .format method. Your code will work with that with just a few minor changes:
contents = github.get(repo['contents_url'].format(path='.gitignore'))
But you'll also have to change your contents_url slightly:
https://api.github.com/repos/<org_name>/<repo_name>/contents/{path}
Just be careful - .format interpolates based on curly braces, so any literal curly braces need to be escaped. More information is available here: https://docs.python.org/3/library/string.html#formatstrings
Edit: As you mentioned in the comment below, the URL is coming directly from GitHub's API and you can't/shouldn't change it. It turns out they're using RFC 6570 URL templates (see https://developer.github.com/v3/#hypermedia). If you use the uritemplate library I suggested below, the code will look like this:
from uritemplate import expand
# ...
contents = github.get(expand(repo['contents_url'], {'path': '.gitignore'}))

Related

Python Iterating Array of URI to retrieve JSON

I have a dynamically created array of URIs that I want to iterate through for retrieval of json data, which I'll then use to search a particular field for a specific value. Unfortunately, I keep getting syntax errors.
for i in list_url_id:{
var t = requests.get(base_url+i+'?limit='+str(count),auth=HTTPBasicAuth(uname,pw)).json()
print(t)
}
If I do a print(i) in the loop, it prints the full URL out properly. I'm lost.
EDIT:
base_url is a URL similar to https://www.abcdef.com:1443
the URI in list_url_id is a URI similar to /v1/messages/list/0293842
I have no issue (as mentioned) concatenating them into a print operation, but when used for the string for requests.get, it returns a nondescript syntax error
Python sees the code inside the bracket as a dictionary, that's probably causing the syntax errors.
Indentation is used in Python for traversing for loops.
for i in list_url_id:
t = requests.get(base_url+i+'?limit='+str(count),auth=HTTPBasicAuth(uname,pw)).json()
print(t)
This should work for you and note that var is also removed as it is the wrong syntax. Python variables do not need an explicit declaration to reserve memory space. The declaration happens automatically when you assign a value to a variable.

Finding documentation on names returned by dir()

If you run the following code:
from flask import Flask
import unittest
dir(Flask(__name__).test_client())
The following is output to terminal:
There are a number of names returned that I cannot find documentation on (all of the names that are not surrounded by double underscores).
I have found indirect reference to post here (if you search for 'self.app.post' you'll see it referenced). Note: this link describes using .post with the following keywords: data and follow_redirects. It does not mention that you can also use the keywords content_type and headers. Perhaps the only reason that these keyword options are not intuitively obvious to me is because I'm new to this...
Does anyone know where documentation on these names resides? (I can't find it in flask/python/unittest documentation anywhere - perhaps I am looking in the wrong place?)
edit: with the help of the answers, I found this documentation.
For any Python Module, Class, Method (all of these in Python are object indeed), you can view the doc by:
>>> a_module.__doc__
>>> a_class.__doc__
>>> a_method.__doc__
To see more detailed documents, you can use help command:
>>> help(a_method)
You can always check the docstring of the method - comments that developers left when they wrote the code. You can check any object or method you need. For example:
Flask.__doc__
unittest.__doc__
dir.__doc__
dir.__doc__.__doc__
You can also query
Flask(__name__).test_client().post.__doc__
Flask(__name__).test_client().preserve_context.__doc__
But you'll notice that not all methods would be documented. For example:
Flask(__name__).test_client().open.__doc__
For more about this you can also see http://legacy.python.org/dev/peps/pep-0257/
Using help() gives you the same information but formatted, e.g.:
help(Flask)
help(unittest)
help(dir)
help(dir.__doc__)

What is the difference between json.JSONDecoder().decode() and json.loads()

I am using urllib2 to grab the html of a url and then a regex to extract a JSON that I need from there. I want to get the usual "dictionary of dictionaries" Python object and both of the following work:
my_json #a correctly formatted json string
json_dict1 = json.JSONDecoder().decode(my_json)
json_dict2 = json.loads(my_json)
What is the difference and which is better in what circumstances (besides mine, but that one in particular)?
json.loads() essentially creates a json.JSONDecoder() instance and calls decode on it. As such your first line is exactly the same thing as the second line. See the json.loads() source code.
The module offers you flexibility; a simple function API or a full OO API that you can subclass if needed.

How can I ensure that my Python regular expression outputs a dictionary?

I'm using Beej's Python Flickr API to ask Flickr for JSON. The unparsed string Flickr returns looks like this:
jsonFlickrApi({'photos': 'example'})
I want to access the returned data as a dictionary, so I have:
photos = "jsonFlickrApi({'photos': 'test'})"
# to match {'photos': 'example'}
response_parser = re.compile(r'jsonFlickrApi\((.*?)\)$')
parsed_photos = response_parser.findall(photos)
However, parsed_photos is a list, not a dictionary (according to type(parsed_photos). It outputs like:
["{'photos': 'test'}"]
How can I ensure that my parsed data ends up as a dictionary type?
If you're using Python 2.6, you can just use the JSON module to parse JSON stuff.
import json
json.loads(dictString)
If you're using an earlier version of Python, you can download the simplejson module and use that.
Example:
>>> json.loads('{"hello" : 4}')
{u'hello': 4}
You need to use a JSON parser to convert the string representation to actual Python data structure. Take a look at the documentation of the json module in the standard library for some examples.
In other words you'd have to add the following line at the end of your code
photos = json.loads(parsed_photos[0])
PS. In theory you could also use eval to achieve the same effect, as JSON is (almost) compatible with Python literals, but doing that would open a huge security hole. Just to let you know.

Passing a Python list using JSON and Django

I'm trying to send a Python list in to client side (encoded as JSON). This is the code snippet which I have written:
array_to_js = [vld_id, vld_error, False]
array_to_js[2] = True
jsonValidateReturn = simplejson.dumps(array_to_js)
return HttpResponse(jsonValidateReturn, mimetype='application/json')
How do I access it form client side? Can I access it like the following?
jsonValidateReturn[0]
Or how do I assign a name to the returned JSON array in order to access it?
Actually I'm trying to convert a server side Ajax script that returns an array (see Stack Overflow question Creating a JSON response using Django and Python that handles client side POST requests, so I wanted the same thing in return with Python, but it didn't go well.
The JSON array will be dumped without a name / assignment.
That is, in order to give it a name, in your JavaScript code you would do something like this:
var my_json_data_dump = function_that_gets_json_data();
If you want to visualize it, for example, substitute:
var my_json_data_dump = { 'first_name' : Bob, 'last_name': smith };
Also, like Iganacio said, you're going to need something like json2.js to parse the string into the object in the last example. You could wrap that parsing step inside of function_that_gets_json_data, or if you're using jQuery you can do it with a function like jQuery.getJSON().
json2.js is still nice to have, though.
In response to the comment (I need space and markup):
Yes, of course. All the Python side is doing is encoding a string representation (JSON) for you. You could do something like 'var blah = %s' % json.dumps(obj_to_encode) and then on the client side, instead of simply parsing the response as JSON, you parse it as JavaScript.
I wouldn't recommend this for a few reasons:
You're no longer outputting JSON. What if you want to use it in a context where you don't want the variable name, or can't parse JavaScript?
You're evaluating JavaScript instead of simply parsing JSON. It's an operation that's open to security holes (if someone can seed the data, they might be able to execute a XSS attack).
I guess you're facing something I think every Ajax developer runs in to. You want one place of truth in your application, but now you're being encouraged to define variables and whatnot in JavaScript. So you have to cross reference your Python code with the JavaScript code that uses it.
I wouldn't get too hung up on it. I can't see why you would absolutely need to control the name of the variable from Python in this manner. If you're counting on the variable name being the same so that you can reference it in subsequent JavaScript or Python code, it's something you might obviate by simply restructuring your code. I don't mean that as a criticism, just a really helpful (in general) suggestion!
If both client and server are in Python, here's what you need to know.
Server. Use a dictionary to get labels on the fields. Write this as the response.
>>> import json
>>> json.dumps( {'vld_id':1,'vls_error':2,'something_else':True} )
'{"vld_id": 1, "something_else": true, "vls_error": 2}'
Client. After reading the response string, create a Python dictionary this way.
>>> json.loads( '{"vld_id": 1, "something_else": true, "vls_error": 2}' )
{u'vld_id': 1, u'something_else': True, u'vls_error': 2}

Categories