I'm trying to simulate some data in to the datastore to emulate POSTs.
What I'm looking for is a way to post named arguments but as one argument. So I can use the same method as a normal POST would do.
The method I want to invoke get params in two ways.
def HandlePost(params):
params.get('name')
params.get_all('collection')
class SavePartHandler(webapp.RequestHandler):
def post(self):
HandlePost(self.request)
I'm trying to find out what type the self.request are but can't find it anywhere in appengines source code.
My goal is to simulate multiple POST to fill the datastore by the methods users would.
EDIT:
Or is the anyway to change the behavior of dict so it could use the get_all method?
EDIT 2:
I'm using appengins webapp.
Out of curiosity is there a way to invoke a dummy webapp.RequestHandler and populate it with arguments? I'm browsing the source code to see how it's done and made a new instance of it but can't find how to populate it.
EDIT 3:
Updated method name so it's not confused with webapp request handlers.
Thanks to pthulin I'm almost there. Only thing left is how to simulate data that has the same key. Since using a dict will override other keys with the same name. Sometimes in HTML forms a post can contain multiple values for the same key that we get with self.request.get_all('key'). So how to create a dict (or something equal) that supports multiple key=value with same key.
..fredrik
I think what you want to do is prepare a webapp Request object and then call the post method of your handler:
from google.appengine.ext.webapp import Request
from google.appengine.ext.webapp import Response
from StringIO import StringIO
form = 'msg=hello'
handler.response = Response()
handler.request = Request({
'REQUEST_METHOD': 'POST',
'PATH_INFO': '/',
'wsgi.input': StringIO(form),
'CONTENT_LENGTH': len(form),
'SERVER_NAME': 'hi',
'SERVER_PORT': '80',
'wsgi.url_scheme': 'http',
})
handler.post()
TIP: 'msg=hello' above works fine in this case, but to pass multiple POST parameters you need to create a URL encoded string:
>>> form = {
... 'first_name': 'Per',
... 'last_name': 'Thulin',
... }
>>> from urllib import urlencode
>>> urlencode(form)
'first_name=Per&last_name=Thulin'
If you want to pass in multiple POST parameters with the same name, I think you need to do a little bit of the url encoding work yourself. For example:
>>> from urllib import urlencode
>>> form_inputs = [
... {'someparam': 'aaa'},
... {'someparam': 'bbb'},
... {'someparam': 'ccc'},
... ]
>>> '&'.join([urlencode(d) for d in form_inputs])
'someparam=aaa&someparam=bbb&someparam=ccc'
Then in the RequestHandler, you can extract the parameters with the Request.get_all method.
http://code.google.com/appengine/docs/python/tools/webapp/requestclass.html
http://code.google.com/appengine/docs/python/tools/webapp/requestclass.html#Request_get_all
For sure they are request objects that represents a http request. It should usually contain typical http request information like method , uri, message etc.
To find out what type self.request are, you could do introspection without the doc. if self.request is a instantiation of a class then you could obtain the class name through
print self.request.__class__
[Edit]:
The information is provided in the google app engine document
http://code.google.com/appengine/docs/python/tools/webapp/requestclass.html
Yes, that can be done. Here's the webapp response handler. The way to do it is to use the response.argument() which returns a list of the data arguments. Then iterate and get(each one). Just for clarity, I show how to handle named arguments that are known ahead of time and also multiple arbitrary arguments as you ask:
from google.appengine.ext import webapp
class postHandler(webapp.RequestHandler):
def post(self):
# Handle arguments one at a time
var1 = self.request.get('var1')
# or get them all and build a dict:
args = self.request.arguments()
arg_dict = {}
for arg in args:
arg_dict.update(arg: self.request.get(arg))
Here's some documentation: http://code.google.com/appengine/docs/python/tools/webapp/requestclass.html#Request_get_all
Related
I want to add type annotations to a view function that returns a call to redirect. What does redirect return, and how do I add an annotation for that to my view function?
I thought it might be str, or the redirect function, but I'm not sure.
def setalarm() -> redirect:
# Retrieves the information to create new alarms.
return redirect("/")
The straightforward answer is to annotate your view with whatever you're writing it to return. In your specific example, redirect returns an instance of werkzeug.wrappers.Response.
from werkzeug.wrappers import Response
def set_alarm() -> Response:
return redirect()
Rather than figuring out what any given function returns in order to annotate your view, it might seem easier to come up with a Union annotation that represents anything a Flask view is allowed to return. However, Flask doesn't provide typing information, and its dynamic nature makes representing the possibilities difficult.
By default, a Flask view can return:
A str or bytes.
A subclass of werkzeug.wrappers.BaseResponse.
A tuple in one of these forms, where data is any of the other types a Flask view can return:
(data,)
(data, status), where status can be either an int or a str or bytes.
(data, headers), where headers is either a dict, iterable of (key, value) tuples, or a werkzeug.datastructures.Headers object.
(data, status, headers)
A dict to be converted to JSON. The values should be types that app.json_encoder supports.
A WSGI callable.
Flask can support more or different return types by overriding the Flask.make_response method. The data it can serialize to JSON can be extended by overriding Flask.json_encoder. If you have customized Flask's behavior you'll need to customize the type information as well.
Here's a view_return_type that represents the possible return types from a Flask view, ignoring JSON typing. Once you define the type, you can annotate any view with it.
import typing as t
from werkzeug.datastructures import Headers
from werkzeug.wrappers import BaseResponse
_str_bytes = t.Union[str, bytes]
_data_type = t.Union[
_str_bytes,
BaseResponse,
t.Dict[str, t.Any],
t.Callable[
[t.Dict[str, t.Any], t.Callable[[str, t.List[t.Tuple[str, str]]], None]], t.Iterable[bytes]
],
]
_status_type = t.Union[int, _str_bytes]
_headers_type = t.Union[
Headers, t.Dict[_str_bytes, _str_bytes], t.Iterable[t.Tuple[_str_bytes, _str_bytes]],
]
view_return_type = t.Union[
_data_type,
t.Tuple[_data_type],
t.Tuple[_data_type, _status_type],
t.Tuple[_data_type, _headers_type],
t.Tuple[_data_type, _status_type, _headers_type],
]
#app.route("/users/<int:id>/")
def user_detail(id: int) -> view_return_type:
...
In Flask 2 you can use flask.typing.ResponseReturnValue.
from flask.typing import ResponseReturnValue
#app.get("/")
def index() -> ResponseReturnValue:
return "OK"
I have run into some trouble with the issue, that request.data sometimes is a dict (especially when testing) and sometimes a QueryDict instance (when using curl).
This is especially a problem because apparently there is a big difference when calling a view using curl like so:
curl -X POST --data "some_float=1.23456789012123123" "http://localhost:8000/myview"
Or using the django_webtest client like so:
class APIViewTest(WebTest):
def test_testsomething(self):
self.app.post(url=url, params=json.dumps({some_float=1.26356756467}))
And then casting that QueryDict to a dict like so
new_dict = dict(**request.data)
my_float = float(new_dict['some_float'])
Everything works fine in the tests, as there request.data is a dict, but in production the view crashes because new_dict['some_float'] is actually a list with one element, and not as expected a float.
I have considered fixing the issue like so:
if type(request.data) is dict:
new_dict = dict(**request.data)
else:
new_dict = dict(**request.data.dict())
which feels very wrong as the tests would only test line 2, and (some? all?) production code would run line 4.
So while I am wondering why QueryDict behaves in this way, I would rather know why and when response.data is a QueryDict in the first place. And how I can use django tests to simulate this behavior. Having different conditions for production and testing systems is always troublesome and sometimes unavoidable, but in this case I feel like it could be fixed. Or is this a specific issue related to django_webtest?
Your test isn't a reflection of your actual curl call.
In your test, you post JSON, which is then available as a dict from request.data. But your curl call posts standard form data, which is available as a QueryDict. This behaviour is managed by the parsers attribute of your view or the DEFAULT_PARSER_CLASSES settings - and further note that this is functionality specifically provided by django-rest-framework, not Django itself.
Really you should test the same thing as you are doing; either send JSON from curl or get your test to post form-data.
When your request content_type is "application/x-www-form-urlencoded", request.Data become QueryDict.
see FormParser class.
https://github.com/encode/django-rest-framework/blob/master/rest_framework/parsers.py
And
QueryDict has get lists method. but it can't fetch dict value.
convert name str to array.
<input name="items[name]" value="Example">
<input name="items[count]" value="5">
https://pypi.org/project/html-json-forms/
And define custom form paser.
class CustomFormParser(FormParser):
"""
Parser for form data.
"""
media_type = 'application/x-www-form-urlencoded'
def parse(self, stream, media_type=None, parser_context=None):
"""
Parses the incoming bytestream as a URL encoded form,
and returns the resulting QueryDict.
"""
parser_context = parser_context or {}
encoding = parser_context.get('encoding', settings.DEFAULT_CHARSET)
data = QueryDict(stream.read(), encoding=encoding)
return parse_json_form(data.dict()) # return dict
And overwite DEFAULT_PARSER_CLASSES.
https://www.django-rest-framework.org/api-guide/settings/#default_parser_classes
I have Pyramid/cornice resource, that requires a ?query=keyword in end of the url. But I don't know how to add this in a pyramid's dummyRequest object. Code works perfectly on browser and I will get correct response when using this url to get stuff: *url*/foo?query=keyword.
My class/resource is defined like this:
#resource(path='/bar/search/foo')
class SearchFooResource(object):
def __init__(self, request):
self.request = request
#view(renderer='json')
def get(self):
#get query string, it's a tuple
req = self.request.GET.items()
#do stuff with req
Now req should contain the all the query string 'stuffs' in a list that contains them as a tuple's, for example: [('query', 'bar'),('query', 'asd')]. But how do I make unittest to this resource? I can't seem to add anything to self.request.GET.items() method. When running unittest req is empty, and I will get this error: AttributeError: 'list' object has no attribute 'items'.
My current unittest:
def test_passing_GetFooBaarResource(self):
request = testing.DummyRequest()
request.GET = [('query', 'keyword')]
info = SearchFooResource.get(SearchFooResource(request))
self.assertEqual(info['foo'], 'baar')
In addition to what #matino has suggested, you can just use a plain dictionary (instead of a list of tuples you tried).
def test_passing_GetFooBaarResource(self):
request = testing.DummyRequest()
request.GET = {'query': 'keyword'}
info = SearchShowResource.get(SearchShowResource(request))
self.assertEqual(info['foo'], 'baar')
This will work in uncomplicated cases where you don't have multiple parameters with the same name (/someurl?name=foo&name=baz&name=bar).
If you need to test those more complicated queries you can replace your DummyRequest's GET attribute with a WebOb MultiDict
from webob.multidict import MultiDict
def test_passing_GetFooBaarResource(self):
request = testing.DummyRequest()
request.GET = MultiDict([('query', 'foo'), ('query', 'bar'), ('query', 'baz')])
info = SearchShowResource.get(SearchShowResource(request))
self.assertEqual(info['foo'], 'baar')
Then, normally, in your actual view method, if you need to handle multiple parameters with the same name you use request.GET.getall('query') which should return ['foo', 'bar', 'baz'].
In simpler cases you can just use request.GET['query'] or request.GET.get('query', 'default'). I mean, your use of request.GET.items() is a bit unusual...
According to the docs I think you need to pass it as params argument (not tested):
request = testing.DummyRequest(params={'query': 'keyword'})
Ok so I have my apps, that takes requests from root / Almost everything is using traversal.
But i'd like to make on top of that site a rest api.
So I'm off with two choices. I either separate the that in two different apps and put that rest application to : rest.site.com, Or I can move it to site.com/rest/*traversal
If I'm doing "/rest/*traversal", I guess I'll have to add a route called rest_traversal where the traversal path will be *traversal with the route /rest/*traversal. I did that once for an admin page.
I was wondering if there was a cleanest way to do that. I tried to use virtual_root, but as I understand virtual_root is actually getting added to the path for traversal.
like having virtual_root = /cms and requesting /fun will create the following path /cms/fun
I on the other hand wish to have /cms/fun turned into /fun
I know this has been answered already, but in case someone arrives here looking for another possible way to make "subapps" and using them in pyramid, I wanted to point out that some interesting things can be done with pyramid.wsgi
"""
example of wsgiapp decorator usage
http://docs.pylonsproject.org/projects/pyramid/en/1.3-branch/api/wsgi.html
"""
from pyramid.wsgi import wsgiapp2, wsgiapp
from pyramid.config import Configurator
from webob import Request, Response
import pprint
# define some apps
def wsgi_echo(environ, start_response):
"""pretty print out the environ"""
response = Response(body=pprint.pformat({k: v for k, v in environ.items()
if k not in ["wsgi.errors",
"wsgi.input",
"SCRIPT_NAME"]}))
return response(environ, start_response)
print Request.blank("/someurl").send(wsgi_echo).body
# convert wsgi app to a pyramid view callable
pyramid_echo = wsgiapp(wsgi_echo)
pyramid_echo_2 = wsgiapp2(wsgi_echo)
# wire up a pyramid application
config = Configurator()
config.add_view(pyramid_echo, name="foo") # /foo
config.add_view(pyramid_echo, name="bar") # /bar
config.add_view(pyramid_echo_2, name="foo_2") # /foo
config.add_view(pyramid_echo_2, name="bar_2") # /bar
pyramid_app = config.make_wsgi_app()
#call some urls
foo_body = Request.blank("/foo").send(pyramid_app).body
bar_body = Request.blank("/bar").send(pyramid_app).body
foo_body_2 = Request.blank("/foo_2").send(pyramid_app).body
bar_body_2 = Request.blank("/bar_2").send(pyramid_app).body
# both should be different because we arrived at 2 different urls
assert foo_body != bar_body, "bodies should not be equal"
# should be equal because wsgiapp2 fixes stuff before calling
# application in fact there's an additional SCRIPT_NAME in the
# environment that we are filtering out
assert foo_body_2 == bar_body_2, "bodies should be equal"
# so how to pass the path along? like /foo/fuuuu should come back
# /fuuuu does it
foo_body = Request.blank("/foo_2/fuuuu").send(pyramid_app).body
assert "'/fuuuu'," in foo_body, "path didn't get passed along"
# tldr: a wsgi app that is decorated with wsgiapp2 will recieve data
# as if it was mounted at "/", any url generation it has to do should
# take into account the SCRIPT_NAME variable that may arrive in the
# environ when it is called
If you're using traversal already, why not just use it to return your "rest API root" object when Pyramid traverses to /rest/? From there, everything will work naturally.
class ApplicationRoot(object):
def __getitem__(self, name):
if name == "rest":
return RestAPIRoot(parent=self, name=name)
...
If your "application tree" and "API tree" have the same children and you want to have different views registered for them depending on which branch of the tree the child is located in, you can use containment view predicate to register your API views, so they will only match when the child is inside the "API branch":
containment
This value should be a reference to a Python class or interface that a
parent object in the context resource’s lineage must provide in order
for this view to be found and called. The resources in your resource
tree must be “location-aware” to use this feature.
If containment is not supplied, the interfaces and classes in the
lineage are not considered when deciding whether or not to invoke the
view callable.
Another approach would be not to build a separate "API tree" but to use your "main" application's "URI-space" as RESTful API. The only problem with this is that GET and possibly POST request methods are already "taken" on your resources and mapped to your "normal" views which return HTML or consume HTTP form POSTs. There are numerous ways to work around this:
register the API views with a separate name, so, say GET /users/123 would return HTML and GET /users/123/json would return a JSON object. Similarly, POST /users/123 would expect HTTP form to be posted and POST /users/123/json would expect JSON. A nice thing about this approach is that you can easily add, say, an XML serializer at GET /users/123/xml.
use custom view predicates so GET /users/123 and GET /users/123?format=json are routed to different views. Actually, there's a built-in request_param predicate for that since Pyramid 1.2
use xhr predicate to differentiate requests based on HTTP_X_REQUESTED_WITH header or accept predicate to differentiate on HTTP_ACCEPT header
if i have this URL in from python code on appengine
http://localhost:8080/blog/view/2f1cab5844fb432b8426ae666c4ac493
how can i get the value of the key : 2f1cab5844fb432b8426ae666c4ac493
#Herms answer will work, but you may prefer this instead:
In the code that creates your webapp instance, capture the key part of the URL with a regex, like:
def main():
application = webapp.WSGIApplication( [
(r'/blog/view/(\w+)', MyBlogViewHandler),
## others listed here...
])
...then code your handler class like this - the key you captured will be passed to your get() method as an argument:
class MyBlogViewHandler(webapp.RequestHandler):
def get(self, key):
# do something useful with the 'key' argument
You can access the requested URL via self.request, assuming you're extending the standard webapp.RequestHandler class. That will give you access to the path and query, and you should be able to extract the values you want from the path.
Here's some documentation on the Request object:
http://code.google.com/appengine/docs/python/tools/webapp/requestclass.html