I'm using bottle to receive POST webhook from bitbucket. The body of the POST contains info about what changed in the repo, etc. I am able to do this fine with #post('/myroute'), however I'm having trouble getting to the actual POST body data text.
here is an image that shows what I'm doing end to end
http://i.imgur.com/rWc7Hry.png
When printed to consolerequest.body yields:
StringIO.StringIO instance at 0x7fa13341c4d0
and request.body.dir() yields:
AttributeError: StringIO instance has no attribute 'dir'
I'm wondering how do I get to the actual text of the request body (or inspect the object somehow to find the same)?
the POST request body will look something like this:
http://pastebin.com/SWjLrHig
I've also tried request.json (no luck)
any advice?
EDIT:
i ended up using this:
from bottle import get, post, request, run
import urllib
import json
#post('/bitbucket')
def postToJSON():
body = request.body.read()
body = body.replace("+","").replace("payload=","")
parsedBody = urllib.unquote(body).decode('utf8')
print parsedBody
jsonObj = json.loads(parsedBody)
print jsonObj
interesting now, parsedBody looks good:
{"repository":{"website":null,"fork":false,"name":"test","scm":"git","owner":"
testName","absolute_url":"/testNameTest/test/","slug":"test","is_private":true},"trunc
ated":false,"commits":[{"node":"04554d6980dd","files":[{"type":"modified","file"
:"stacker.py"}],"raw_author":"TestName<testName#testName.info>","utctimestamp":"
2015-05-2815:30:03+00:00","author":"testName","timestamp":"2015-05-2817:30:03","
raw_node":"04554d6980dd3c5fe4c3712d95b49fcf9b8da4f4","parents":["7f98b4e7532e"],
"branch":"master","message":"foo\n","revision":null,"size":-1}],"canon_url":"htt
ps://bitbucket.org","user":"testName"}
but jsonObj is not so good:
{u'commits': [{u'node': u'7f98b4e7532e', u'files': [{u'type': u'modified', u'fil
e': u'stacker.py'}], u'branch': u'master', u'utctimestamp': u'2015-05-2815:24:50
+00:00', u'author': u'TestName', u'timestamp': u'2015-05-2817:24:50', u'raw_node
': u'7f98b4e7532e02d53d83a29ec2073c5a5eac58c8', u'parents': [u'019e77d2e0d3'], u
'raw_author': u'TestNamer<TestName#TestName.info>', u'message': u'foo\n', u'size'
: -1, u'revision': None}], u'user': u'TestName', u'canon_url': u'https://bitbuck
et.org', u'repository': {u'website': None, u'fork': False, u'name': u'test', u's
cm': u'git', u'absolute_url': u'/ericTest/test/', u'owner': u'TestName', u'slug'
: u'test', u'is_private': True}, u'truncated': False}
however, when I do something like
print jsonObj['repository']['name']
it works as expected (just prints the name 'test')
As the bottle documentation states, the request data is "a file like object". http://bottlepy.org/docs/dev/tutorial.html#the-raw-request-body
So you access the raw body using read().
Also, dir is not a method of objects, it's a freestanding function which you call passing an object.
dir(request.body)
And googling for StringIO should have brought you here: https://docs.python.org/2/library/stringio.html
Related
Currently I'm having a problem setting up a simple REST service with flask and a chatterbot. You can see the full code here.
The goal is, that the service returns a json with a response from the chatbot to a given request.
The problem is, that when I want to save the response from the chatbot in a dict:
dialog = {
"id": 1,
"usersay": request,
# chatterbot function to get a response from the bot
"botsay": chatbot.get_response(request)
}
It will be saved as a chatterbot special Statement Object like which will then look like this:
"botsay": <Statement text:bot response>
When I try to jsonify a dict with this object I get the following error:
TypeError: Can't convert 'Statement' object to str implicitly
I searched online to find a solution but havent found anything helpful. In addition, I'm not experienced with python.
What is absolutely unexplainable for me is, when I use
>>> request = "Hi"
>>> print(chatbot.get_response(request))
I will get the correct output
> Hello
I just want to save the plain response in the dict so can I return it as a json to the client.
Could anyone explain the problem?
Thanks in advance!
Problem solved by simple accessing the "text" attribute of the Statement Object with the . notation (see heere).
>>> response = chatterbot.get_response("Hi")
>>> dialog = { ..., "botsay" = response.text, ... }
>>> print dialog
{ ..., "botsay": "Hello", ...}
I am trying to create a corpus of text documents via the New york Times API (articles concerning terrorist attacks) on Python.
I am aware that the NYP API do not provide the full body text, but provides the URL from which I can scrape the article. So the idea is to extract the "web_url" parameters from the API and consequently scrape the full body article.
I am trying to use the NYT API library on Python with these lines:
from nytimesarticle import articleAPI
api = articleAPI("*Your Key*")
articles = api.search( q = 'terrorist attack')
print(articles['response'],['docs'],['web_url'])
But I cannot extract the "web_url" or the articles. All I get is this output:
{'meta': {'time': 19, 'offset': 10, 'hits': 0}, 'docs': []} ['docs'] ['web_url']
There seems to be an issue with the nytimesarticle module itself. For example, see the following:
>>> articles = api.search(q="trump+women+accuse", begin_date=20161001)
>>> print(articles)
{'response': {'docs': [], 'meta': {'offset': 0, 'hits': 0, 'time': 21}}, 'status': 'OK', 'copyright': 'Copyright (c) 2013 The New York Times Company. All Rights Reserved.'}
But if I use requests (as is used in the module) to access the API directly, I get the results I'm looking for:
>>> import requests
>>> r = requests.get("http://api.nytimes.com/svc/search/v2/articlesearch.json?q=trump+women+accuse&begin_date=20161001&api-key=XXXXX")
>>> data = r.json()
>>> len(data["response"]["docs"])
10
meaning that 10 articles were returned (the full value of data is 16kb, so I won't include it all here). Contrast that to the response from api.search(), where articles["response"]["docs"] is an empty list.
nytimesarticle.py is only 115 lines long, so it's pretty straightforward to debug. Printing the value of the URL sent to the API reveals this:
>>> articles = api.search(q="trump+women+accuse", begin_date=20161001)
https://api.nytimes.com/svc/search/v2/articlesearch.json?q=b'trump+women+accuse'&begin_date=20161001&api-key=XXXXX
# ^^ THIS
The offending code encodes every string parameter to UTF-8, which makes it a bytes object. This is not necessary, and wrecks the constructed URL as shown above. Fortunately, there is a pull request that fixes this:
>>> articles = api.search(q="trump+women+accuse", begin_date=20161001)
http://api.nytimes.com/svc/search/v2/articlesearch.json?begin_date=20161001&q=trump+women+accuse&api-key=XXXXX
>>> len(articles["response"]["docs"])
10
This also allows for other string parameters such as sort="newest" to be used, as the bytes formatting was causing an error previously.
The comma in the print statement separates what is printed.
You'll want something like this
articles['response']['docs']['web_url']
But 'docs': [] is both an array and empty, so above line won't work, so you could try
articles = articles['response']['docs']
for article in articles:
print(article['web_url'])
Hi I have recently started programming in Python (I am newbie to python programming). I have a small collection of data in my MongoDB.I have written a simple get method to find all the data from my collection. But I have an error returning the fetched value.
Here is my code:
import bson
from bson import json_util
from bson.json_util import dumps
class TypeList(APIHandler):
#gen.coroutine
def get(self):
doc = yield db.vtype.find_one()
print(doc)
a = self.write(json_util.dumps(doc))
return a
def options(self):
pass
It gives me the fetched data.
But when I replace these lines
a = self.write....
return a
with return bson.json_util.dumps({ 'success': True, 'mycollectionKey': doc })
it gives me a type error.
TypeError: Expected None, got {'success': True, 'mycollectionKey': {'type': 1, 'item': 'cookie'}}
Can anyone explain me why I get this error and is there anyway to solve the problem.
Thanks in advance.
RequestHandler.get() is not supposed to return anything. This error is simply warning you that you returned a value that is being ignored. Tornado handlers produce output by calling self.write(), not by returning a value.
When I access the web service using soapUI I get the correctly formatted text.
But when I use the python code, I get a dictionary with all rows in a single allBusType key.
from pysimplesoap.client import SoapClient
url = 'http://180.92.171.93:8080/UPSRTCServices/UPSRTCService?wsdl'
namespace = 'http://service.upsrtc.trimax.com/'
client = SoapClient(wsdl=url, namespace=namespace, trace=True)
print client.GetBusTypes()
The above code returns the following:
{'return': {'allBusType': [{'busName': u'AC SLEEPER'}, {'busType': u'ACS'}, {'ischildconcession': u'N'}, {'isseatlayout': u'N'}, {'isseatnumber': u'N'}, {'busName': u'AC-JANRATH'}, {'busType': u'JNR'}, {'ischildconcession': u'N'}, {'isseatlayout': u'Y'}, {'isseatnumber': u'Y'},....
As per the following screen, soapUI is returning all the bus stops as separate tag. (And not all stops in a single tag as above)
<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/">
<S:Body>
<ns3:GetBusTypesResponse xmlns:ns2="com.trimax.upsrtc.xml.jaxb.model" xmlns:ns3="http://service.upsrtc.trimax.com/">
<return>
<allBusType>
<busName>AC SLEEPER</busName>
<busType>ACS</busType>
<ischildconcession>N</ischildconcession>
<isseatlayout>N</isseatlayout>
<isseatnumber>N</isseatnumber>
</allBusType>
<allBusType>
<busName>AC-JANRATH</busName>
<busType>JNR</busType>
<ischildconcession>N</ischildconcession>
<isseatlayout>Y</isseatlayout>
<isseatnumber>Y</isseatnumber>
</allBusType>
I will like to know if this is python issue or the server issue.
For each entry there is opening and closing tag called "allBusType" in the soapUI response that is missing in python response.
Python output is returning a single row for all entries.
SoapClient returns a SimpleXmlElement as stated in the first line of the SoapClient docs:
A simple, minimal and functional HTTP SOAP webservice consumer, using httplib2 for the connection ad SimpleXmlElement for the XML request/response manipulation.
Therefore to view it as xml you need to call the as_xml method on the returned SimpleXmlElement:
as_xml(pretty=False): Return the XML representation of the document
The following should work:
from pysimplesoap.client import SoapClient
url = 'http://180.92.171.93:8080/UPSRTCServices/UPSRTCService?wsdl'
namespace = 'http://service.upsrtc.trimax.com/'
client = SoapClient(wsdl=url, namespace=namespace, trace=True)
results = client.GetBusTypes()
print results.as_xml()
I wanted to port video to Tumblr using the API using the Tumblpy library.
My code is this:
import requests
r = requests.get(video-url)
f = {'data':r.content}
dat = urllib.urlencode(f)
t.post('post', blog_url='http://tumblrname.tumblr.com/',params={'type':'video',
'title':post.title, 'slug': post.slug,'date':post.date,'data':dat,'tags':post.tagscsv,
'caption': post.body_html}) #t is TumblPy instance
Well, I am not being successful in this. I do think I am missing out on how to encode the binary contents to make the post successful, though I am not sure.
Presumably it's going to be similar to how you post a photo, in which case the library wants a file(like) object. A requests response can act as a file-like object just fine:
import requests
r = requests.get(video_url)
t.post('post', blog_url='http://tumblrname.tumblr.com/',
params={'type': 'video', 'title': post.title, 'slug': post.slug,
'date': post.date, 'data': r.raw, 'tags': post.tagscsv,
'caption': post.body_html})
where r.raw gives you a file-like object that, when read, yields the video data read from video_url.