I'm writing an API wrapper to a couple of different web services.
I have a method that has an article url, and I want to extract text from it using alchemyapi.
def extractText(self):
#All Extract Text Methods ---------------------------------------------------------//
#Extract page text from a web URL (ignoring navigation links, ads, etc.).
if self.alchemyapi == True:
self.full_text = self.alchemyObj.URLGetText(self.article_link)
which goes to the following code in the python wrapper
def URLGetText(self, url, textParams=None):
self.CheckURL(url)
if textParams == None:
textParams = AlchemyAPI_TextParams()
textParams.setUrl(url)
return self.GetRequest("URLGetText", "url", textParams)
def GetRequest(self, apiCall, apiPrefix, paramObject):
endpoint = 'http://' + self._hostPrefix + '.alchemyapi.com/calls/' + apiPrefix + '/' + apiCall
endpoint += '?apikey=' + self._apiKey + paramObject.getParameterString()
handle = urllib.urlopen(endpoint)
result = handle.read()
handle.close()
xpathQuery = '/results/status'
nodes = etree.fromstring(result).xpath(xpathQuery)
if nodes[0].text != "OK":
raise 'Error making API call.'
return result
However I get this error ---
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "text_proc.py", line 97, in __init__
self.alchemyObj.loadAPIKey("api_key.txt");
File "text_proc.py", line 115, in extractText
if self.alchemyapi == True:
File "/Users/Diesel/Desktop/AlchemyAPI.py", line 502, in URLGetText
return self.GetRequest("URLGetText", "url", textParams)
File "/Users/Diesel/Desktop/AlchemyAPI.py", line 618, in GetRequest
raise 'Error making API call.'
I know I'm somehow passing the url string to the api wrapper in a faulty format, but I can't figure out how to fix it.
The information provided is not actually very helpful to diagnose or solve the problem. Have you considered taking a look at the response from the server? You might inspect a complete traffic log using Fiddler.
Additionally, the SDK provided by Alchemy doesn't seem to be of - cough, cough - the greatest quality. Since it really consists only of around 600 lines of source code, I'd consider writing a shorter, more robust / pythonic / whatever SDK.
I might also add that right now, even the on-site demo at the Alchemy web site is failing, so maybe your problem is related to that. I really suggest taking a look at the traffic.
You should raise Exception or a subclass thereof, instead of a string.
You're getting the error because your function GetRequest() raising a string as an exception:
if nodes[0].text != "OK":
raise 'Error making API call.'
If that's not what you want, you have two options:
You can have the function return the string or None, or
You can pass the error message to a real subclass of Exception (as suggested by knutin)
In either case, if you are assigning that return value to a variable, you can handle it accordingly. Here is an example:
Option 1
Let's assume you decide to have GetRequest() return None:
def URLGetText(self, url, textParams=None):
self.CheckURL(url)
if textParams == None:
textParams = AlchemyAPI_TextParams()
textParams.setUrl(url)
# Capture the value of GetRequest() before returning it
retval = self.GetRequest("URLGetText", "url", textParams)
if retval is None:
print 'Error making API call.' # print the error but still return
return retval
def GetRequest(self, apiCall, apiPrefix, paramObject):
# ...
if nodes[0].text != "OK":
return None
return result
This option is a little ambiguous. How do you know that it was really an error, or the return value truly was None?
Option 2
This is probably the better way to do it:
First create an subclass of Exception:
class GetRequestError(Exception):
"""Error returned from GetRequest()"""
pass
Then raise it in GetRequest()`:
def URLGetText(self, url, textParams=None):
self.CheckURL(url)
if textParams == None:
textParams = AlchemyAPI_TextParams()
textParams.setUrl(url)
# Attempt to get a legit return value & handle errors
try:
retval = self.GetRequest(apiCall, apiPrefix, paramObject)
except GetRequestError as err:
print err # prints 'Error making API call.'
# handle the error here
retval = None
return retval
def GetRequest(self, apiCall, apiPrefix, paramObject):
# ...
if nodes[0].text != "OK":
raise GetRequestError('Error making API call.')
return result
This way you're raising a legitimate error when GetRequest() doesn't return the desired result, and then you can handle the error using a try..except block and optionally print the error, stop the program there, or keep going (which is what I think you want to do based on your question).
This is Shaun from AlchemyAPI. We just posted a new version of the python SDK that raises exceptions properly. You can get it here http://www.alchemyapi.com/tools/.
If you have any other feedback about the SDK, please message me. Thanks for using our NLP service.
Related
I am working on a small texting application using Twilio API. Recently, I had some issues with the API so I added an Exception in my get_current_credits() function.
I am quite new to Python and programming in general and I would like to know what would be the cleanest way to do that.
If the Exception is throw, I only return a String. If not, I am returning a tuple. What would be the cleanest way to see what was the return from my inject_credits() function, I am thinking about type(res) but does seems a quick and dirty solution?
def get_current_credits():
try:
balance_data = twilio_client.api.v2010.balance.fetch()
balance = float(balance_data.balance)
currency = balance_data.currency
return balance, currency
except Exception:
return "503 Service Unavailable"
def inject_credit():
res = get_current_credits()
# if Exception:
# return the Exception message as a string
# else, do the following:
(credit, currency) = res
if currency != "EUR":
credit = currency_converter.convert(credit, currency, 'EUR')
return dict(credit=(round(credit,2)))
You could move the Exception outside, into the body of inject_credit. Thus, you don't have to do any if statements inside inject_credit, you can just catch the Exception there itself.
Checking the type isn't a bad idea, but it is not very clear what is being done if someone is only reading inject_credit.
You can try this:
if type(get_current_credits()) == str:
# Do something if it is a string
else:
# Do something if it is a tuple
However, the best way would be to add the try statement outside of the function so that you can catch the exception there instead.
try:
get_current_credits()
except Exception as e:
print(e)
# Do whatever
I have a Click application, that I´m working on and I want to customize (thus override) the default behavior of Click when a command/subcommand has been invoked that is not known.
Currently there is only this error message: Error: No such command xx and no code after that gets executed anymore.
My initial setup is like this:
#click.group(cls=CustomHelpOrder)
#click.option('-v', '--verbose',is_flag=True)
def myapp_cli(verbose):
if verbose:
do_verbose()
else:
do_not_verbose()
#myapp_cli.command(help_priority=1, short_help='Foo my project')
#click.option('--bar')
def baz(bar: str) -> None:
"""
Do something
"""
do_something(bar)
So what I want to be able to do is:
Check if this (special) exception above is raised. (Maybe if the user types bat instead of baz)
If yes, I want to save this unknown command (like bat (instead of just printing the default message and exit) and use it for further processing (like passing it to various functions (as a string maybe))
I tried to
try:
myapp_cli()
except Click.Exception:
handleExc() but I was not able to figure out how to do this since the application just exits if the command is unknown (I think before anything can catch it; a really naive approach).
I´m wondering if I have to overwrite the default behavior somehow, but since I´m quite new to Click I don´t know how.
I´d be thankful for any help.
You could implement your own custom group. From Click's manual:
class AliasedGroup(click.Group):
def get_command(self, ctx, cmd_name):
rv = click.Group.get_command(self, ctx, cmd_name)
if rv is not None:
return rv
matches = [x for x in self.list_commands(ctx)
if x.startswith(cmd_name)]
if not matches:
return None
elif len(matches) == 1:
return click.Group.get_command(self, ctx, matches[0])
ctx.fail('Too many matches: %s' % ', '.join(sorted(matches)))
In the above snippet, you can edit the following part to handle your Command not found case however you will.
if not matches:
# your custom logic for `Command not found` goes here
Note:- I have written my code after referring to few examples in stack overflow but still could not get the required output
I have a python script in which loop iterates with an Instagram API. I give the user_id as an input to the API which gets the no of posts, no of followers and no of following. Each time it gets a response, I load it into a JSON schema and append to lists data1, data2 and data3.
The issue is:= Some accounts are private accounts and the API call is not allowed to it. When I run the script in IDLE Python shell, its gives the error
Traceback (most recent call last):
File "<pyshell#144>", line 18, in <module>
beta=json.load(url)
File "C:\Users\rnair\AppData\Local\Programs\Python\Python35\lib\site- packages\simplejson-3.8.2-py3.5-win-amd64.egg\simplejson\__init__.py", line 455, in load
return loads(fp.read(),
File "C:\Users\rnair\AppData\Local\Programs\Python\Python35\lib\tempfile.py", line 483, in func_wrapper
return func(*args, **kwargs)
**ValueError: read of closed file**
But the JSON contains this:-
{
"meta": {
"error_type": "APINotAllowedError",
"code": 400,
"error_message": "you cannot view this resource"
}
}
My code is:-
for r in range(307,601):
var=r,sheet.cell(row=r,column=2).value
xy=var[1]
ij=str(xy)
if xy=="Account Deleted":
data1.append('null')
data2.append('null')
data3.append('null')
continue
myopener=Myopen()
try:
url=myopener.open('https://api.instagram.com/v1/users/'+ij+'/?access_token=641567093.1fb234f.a0ffbe574e844e1c818145097050cf33')
except urllib.error.HTTPError as e: // I want the change here
data1.append('Private Account')
data2.append('Private Account')
data3.append('Private Account')
continue
beta=json.load(url)
item=beta['data']['counts']
data1.append(item['media'])
data2.append(item['followed_by'])
data3.append(item['follows'])
I am using Python version 3.5.2. The main question is If the loop runs and a particular call is blocked and getting this error, how to avoid it and keep running the next iterations? Also, if the account is private, I want to append "Private account" to the lists.
Looks like the code that is actually fetching the URL is within your custom type - "Myopen" (which is not shown). It also looks like its not throwing the HTTPError you are expecting since your "json.load" line is still being executed (and leading to the ValueError that is being thrown).
If you want your error handling block to fire, you would need to check the response status code to see if its != 200 within Myopen and throw the HTTPError you are expecting instead of whatever its doing now.
I'm not personally familiar with FancyURLOpener, but it looks like it supports a getcode method. Maybe try something like this instead of expecting an HTTPError:
url = myopener.open('yoururl')
if url.getcode() == 400:
data1.append('Private Account')
data2.append('Private Account')
data3.append('Private Account')
continue
I'm trying to create a simple module for phenny, a simple IRC bot framework in Python. The module is supposed to go to http://www.isup.me/websitetheuserrequested to check is a website was up or down. I assumed I could use regex for the module seeing as other built-in modules use it too, so I tried creating this simple script although I don't think I did it right.
import re, urllib
import web
isupuri = 'http://www.isup.me/%s'
check = re.compile(r'(?ims)<span class="body">.*?</span>')
def isup(phenny, input):
global isupuri
global cleanup
bytes = web.get(isupuri)
quote = check.findall(bytes)
result = re.sub(r'<[^>]*?>', '', str(quote[0]))
phenny.say(result)
isup.commands = ['isup']
isup.priority = 'low'
isup.example = '.isup google.com'
It imports the required web packages (I think), and defines the string and the text to look for within the page. I really don't know what I did in those four lines, I kinda just ripped the code off another phenny module.
Here is an example of a quotes module that grabs a random quote from some webpage, I kinda tried to use that as a base: http://pastebin.com/vs5ypHZy
Does anyone know what I am doing wrong? If something needs clarified I can tell you, I don't think I explained this enough.
Here is the error I get:
Traceback (most recent call last):
File "C:\phenny\bot.py", line 189, in call
try: func(phenny, input)
File "C:\phenny\modules\isup.py", line 18, in isup
result = re.sub(r'<[^>]*?>', '', str(quote[0]))
IndexError: list index out of range
try this (from http://docs.python.org/release/2.6.7/library/httplib.html#examples):
import httplib
conn = httplib.HTTPConnection("www.python.org")
conn.request("HEAD","/index.html")
res = conn.getresponse()
if res.status >= 200 and res.status < 300:
print "up"
else:
print "down"
You will also need to add code to follow redirects before checking the response status.
edit
Alternative that does not need to handle redirects but uses exceptions for logic:
import urllib2
request = urllib2.Request('http://google.com')
request.get_method = lambda : 'HEAD'
try:
response = urllib2.urlopen(request)
print "up"
print response.code
except urllib2.URLError, e:
# failure
print "down"
print e
You should do your own tests and choose the best one.
The error means your regexp wasn't found anywhere on the page (the list quote has no element 0).
I am writing a google app engine app and I have this key value error upon requests coming in
from the backtrace I just access and cause the key error
self.request.headers
entire code snippet is here, I just forward the headers unmodified
response = fetch( "%s%s?%s" % (
self.getApiServer() ,
self.request.path.replace("/twitter/", ""),
self.request.query_string
),
self.request.body,
method,
self.request.headers,
)
and get method handling the request calling proxy()
# handle http get
def get(self, *args):
parameters = self.convertParameters(self.request.query_string)
# self.prepareHeader("GET", parameters)
self.request.query_string = "&".join("%s=%s" % (quote(key) , quote(value)) for key, value in parameters.items())
self.proxy(GET, *args)
def convertParameters(self, source):
parameters = {}
for pairs in source.split("&"):
item = pairs.split("=")
if len(item) == 2:
parameters[item[0]] = unquote(item[1])
return parameters
the error back trace:
'CONTENT_TYPE'
Traceback (most recent call last):
File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 513, in __call__
handler.post(*groups)
File "/base/data/home/apps/waytosing/1.342850593213842824/com/blogspot/zizon/twitter/RestApiProxy.py", line 67, in post
self.proxy(POST, *args)
File "/base/data/home/apps/waytosing/1.342850593213842824/com/blogspot/zizon/twitter/RestApiProxy.py", line 47, in proxy
self.request.headers,
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/urlfetch.py", line 240, in fetch
allow_truncated, follow_redirects)
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/urlfetch.py", line 280, in make_fetch_call
for key, value in headers.iteritems():
File "/base/python_runtime/python_dist/lib/python2.5/UserDict.py", line 106, in iteritems
yield (k, self[k])
File "/base/python_runtime/python_lib/versions/1/webob/datastruct.py", line 40, in __getitem__
return self.environ[self._trans_name(item)]
KeyError: 'CONTENT_TYPE'
Any idea why it happens or is this a known bug?
This looks weird. The docs mention that response "Headers objects do not raise an error when you try to get or delete a key that isn't in the wrapped header list. Getting a nonexistent header just returns None". It's not clear from the request documentation if request.headers are also objects of this class, but even they were regular dictionaries, iteritems seems to be misbehaving. So this might be a bug.
It might be worth inspecting self.request.headers, before calling fetch, and see 1) its actual type, 2) its keys, and 3) if trying to get self.request.headers['CONTENT_TYPE'] raises an error then.
But, if you simply want to solve your problem and move forward, you can try to bypass it like:
if 'CONTENT_TYPE' not in self.request.headers:
self.request.headers['CONTENT_TYPE'] = None
(I'm suggesting setting it to None, because that's what a response Header object should return on non-existing keys)
Here's my observation about this problem:
When the content-type is application/x-www-form-urlencoded and POST data is empty (e.g. jquery.ajax GET, twitter's favorite and retweet API...), the content-type is dropped by Google appengine.
You can add:
self.request.headers.update({'content-type':'application/x-www-form-urlencoded'})
before urlfetch.
Edit: indeed, looking at the error more carefully, it doesn't seem to be related to convertParameters, as the OP points out in the comments. I'm retiring this answer.
I'm not entirely sure what you mean by "just forward the headers unmodified", but have you taken a look at self.request.query_string before and after you call convertParameters? More to the point, you're leaving out any (valid) GET parameters of the form "key=" (that is, keys with empty values).
Maybe your original query_string had a value like "CONTENT_TYPE=", and your convertParameters is stripping it out.
Known issue http://code.google.com/p/googleappengine/issues/detail?id=3427 and potential workarounds here http://code.google.com/p/googleappengine/issues/detail?id=2040