Trying to catch this error:
ERROR: (gcloud.compute.instances.add-labels) HTTPError 404: The resource 'projects/matei-testing-4010-5cbdeeff/zones/us-east1-b/instances/all' was not found
Tried different versions of code and none worked for me.
My current code does not seem to catch an error:
from googleapiclient import discovery, errors
try:
print("Applying labels")
gcloud_value = (f'gcloud compute instances add-labels all --labels="key=value" --zone=us-east1-b')
process = subprocess.run([gcloud_value], shell=True)
except errors.HttpError:
print("Command did not succeed because of the following error: {}".format(errors.HttpError))
How do I catch the error to use it later?
Thank you
Try this:-
import subprocess
gcloud_value = 'gcloud compute instances add-labels all --labels="key=value" --zone=us-east1-b'
process = subprocess.run(gcloud_value, shell=True, capture_output=True)
print(process.stdout.decode('utf-8'))
print(process.stderr.decode('utf-8'))
print(process.returncode)
One would expect gcloud to emit errors to stderr. Therefore by examining process.stderr you should be able to figure out what (if anything) has gone wrong. Also, if process.returncode is non-zero you should be able to deduce that it didn't work but that depends entirely on how the underlying application (gcloud in this case) is written. There's plenty of stuff out there that returns zero even when there was a failure!
I strongly encourage you to consider using Google's client libraries to interact with its services' rather than subprocess'ing.
As you're experiencing calling out to a shell not only limits error handling but it often requires string "munging" which is also imprecise and ad hoc auth.
Google generally provides machine-generated "perfect" SDK implementations of all its services. For Cloud (except Compute!?), there are also Cloud Client libraries.
I encourage you to explore using the Python SDK for Compute Engine:
https://cloud.google.com/compute/docs/tutorials/python-guide
Related
Background: I'm trying to write a Python script that creates a task in ToDoist using their REST API Python SDK, based on the charge percentage of my dog's Fi Collar (obtained via Pytryfi). Basically, if the Fi collar's battery falls below a certain threshold, create a new task.
Problem: A 401 error is returned when trying to use the todoist_api_python SDK, copied exactly from Todoist's website.
Following these instructions, I was able to install todoist-api-python, but when I run this code (using a real API key):
pip install todoist-api-python
from todoist_api_python.api import TodoistAPI
api = TodoistAPI("XXXXXXX")
try:
projects = api.get_projects()
print(projects)
except Exception as error:
print(error)
I receive this error:
410 Client Error: Gone for url: https://api.todoist.com/rest/v1/projects
I do know there has been a recent change from v1->v2 of this API, and indeed when I put the URL above from the error message in to a browser with ../v2/projects, I see a list of my tasks.
I don't know how to make the todoist-api-python SDK point to the new URL. I'd really appreciate any help you can offer! Thanks!
Be sure you're using the version 2 of todoist-api-python, as this is the one that uses the latest version of the API: https://pypi.org/project/todoist-api-python/
You're probably using an old version that relies on REST API v1.
I am developing a django app which communicates with several Amazon Web Services.
So far I am having trouble dealing with and catching exceptions thrown by the boto3 client. What I am doing seems unnecessarily tedious:
Example:
client = boto3.client('sns')
client.create_platform_endpoint(PlatformApplicationArn=SNS_APP_ARN, Token=token)
this might throw an botocore.errorfactory.InvalidParameterException if e.g. the token is bad.
client.get_endpoint_attributes(EndpointArn=endpoint_arn)
might throw an botocore.errorfactory.NotFoundException.
First, I can't find these Errors anywhere in code, so they are probably generated somewhere. Bottom line: I can't import it and catch it as usual.
Second, I found one way to catch the error here using:
try:
# boto3 stuff
except botocore.exceptions.ClientError as e:
if e.response['Error']['Code'] == 'NotFound':
# handle exception
else:
raise e
But I have to remove the Exception part of the error name. Seems very random and I have no clue whether I would remove the Error in botocore.exceptions.ParamValidationError if I wanted to catch that one. So it's hard to generalize.
Another way to catch the error is using the boto3 client object I got:
try:
# boto3 stuff
except client.exceptions.NotFoundException as e:
# handle exception
This seems the cleanest way so far. But I don't always have the boto3 client object at hand where I want to catch the error. Also I am still only trying things out, so it's mostly guess work.
Does anybody know how boto3 errors are supposed to be handled?
Or can point me towards some coherent documentation which mentions the errors above? Thanks
You've summarized the situation well. The old boto had a simple hardcoded approach to supporting AWS APIs. boto3, in what appears to be an attempt to reduce the overhead of keeping Python client synced with evolving features on the various apis, has been more squishy around exceptions, so the ClientError approach you outlined above used to be the canonical way.
In 2017 they introduced the second mechanism you highlight: 'modeled' exceptions available on the client.
I am not familiar with SNS but in my experience with other AWS products, the ClientError naming matches up with the HTTP apis, which tend to be well documented. So I would start here: https://docs.aws.amazon.com/sns/latest/api/Welcome.html
It looks like the new-style modeled exceptions are generated from service definition files that live in botocore module. I can't find any documentation about it, but go browse around the AWS service models in https://github.com/boto/botocore/tree/master/botocore/data.
Also, it's good to know that if you are not (in contrast to OP's code) dealing directly with the low-level client, but instead are using a high-level AWS ServiceResource object, a low-level client is still easily available at my_service_resource.meta.client so you can handle exceptions like this:
try:
my_service_resource.do_stuff()
except my_service_resource.meta.client.exceptions.NotFoundException as e:
# handle exception
Use Boto3 exceptions: https://www.programcreek.com/python/example/97944/boto3.exceptions
client = boto3.client('logs')
try:
client.create_log_group(logGroupName=LOG_GROUP)
except client.exceptions.ResourceAlreadyExistsException:
pass
How about using pywin32 module ?
Or is there any way to achieve this through subprocess module ?
I can't imagine how this is possible.
Windows services do not expose generic messaging API; each service (should it choose to) exposes its own specific API via its own choice of IPC channel (eg. WCF).
Regardless though, nothing would allow you to do this synchronously; any kind of IPC will be an async call to the service endpoint.
You kind of need to be more specific in your question.
The available generic APIs for interacting with a windows service are basically limited to; stop, start, install, uninstall. Have a look here: https://msdn.microsoft.com/en-us/library/windows/desktop/ms685942(v=vs.85).aspx
(If you are writing a new windows service, in python, ZeroMQ would be a very reasonable choice to interact with it from a command line python script; there are any number of alternative IPC channels for python which would be equally good)
--
To just start a service, try:
import win32service
import win32serviceutil
import time
win32serviceutil.StartService(serviceName)
status = win32serviceutil.QueryServiceStatus(serviceName)
while status == win32service.SERVICE_START_PENDING:
time.sleep(1)
status = win32serviceutil.QueryServiceStatus(serviceName)
Nb. You'll get an access denied error unless you spawn the python instance as administrator.
I simply want to receive notifications from dropbox that a change has been made. I am currently following this tutorial:
https://www.dropbox.com/developers/reference/webhooks#tutorial
The GET method is done, verification is good.
However, when trying to mimic their implementation of POST, I am struggling because of a few things:
I have no idea what redis_url means in the def_process function of the tutorial.
I can't actually verify if anything is really being sent from dropbox.
Also any advice on how I can debug? I can't print anything from my program since it has to be ran on a site rather than an IDE.
Redis is a key-value store; it's just a way to cache your data throughout your application.
For example, access token that is received after oauth callback is stored:
redis_client.hset('tokens', uid, access_token)
only to be used later in process_user:
token = redis_client.hget('tokens', uid)
(code from https://github.com/dropbox/mdwebhook/blob/master/app.py as suggested by their documentation: https://www.dropbox.com/developers/reference/webhooks#webhooks)
The same goes for per-user delta cursors that are also stored.
However there are plenty of resources how to install Redis, for example:
https://www.digitalocean.com/community/tutorials/how-to-install-and-use-redis
In this case your redis_url would be something like:
"redis://localhost:6379/"
There are also hosted solutions, e.g. http://redistogo.com/
Possible workaround would be to use database for such purpose.
As for debugging, you could use logging facility for Python, it's thread safe and capable of writing output to file stream, it should provide you with plenty information if properly used.
More info here:
https://docs.python.org/2/howto/logging.html
pg module in python for interacting with postgres is not giving any error message for DML queries.
Is there any alternative to pg module which gives meaningful error messages.
>>>import pg
>>>
>>>
>>>conn = pg.connect(dbname="db", user="postgres", host="localhost")
>>>print conn.query("delete from item where item_id=0")
>>>None
>>>print conn.query("delete from item where item_id=0")
>>>None #This should have been an error message
Why are you expecting an error message? I delete does not raise an error in the server if no records were found. So why do you expect a generally applicable database driver to raise an error?
I can't think of any database driver that would issue an error in that case because there may be perfectly legitimate reasons for it. In database terms, for example, an error usually means you are going to roll back a transaction.
If you are going to wrap in your own API, my recommendation is that you either decide how you want your application to address this or at most you raise warnings instead.
Do you know the psycopg2 module? It seems a very nice module to interact with PostgreSQL via Python. There is a tutorial available, besides the modules' documentation. In the examples given in the docs, commands that fail indeed print out error messages.
I personally don't have experience in this module. But it looks very nice!