We Keep Coding

How to handle Spyne XSD exceptions gracefully

Whenever my Spyne application receives a request, XSD validation is performed. This is good, but whenever there is an XSD violation a fault is raised and my app returns a Client.SchemaValidationError like so:
<soap11env:Fault>
<faultcode>soap11env:Client.SchemaValidationError</faultcode>
<faultstring>:25:0:ERROR:SCHEMASV:SCHEMAV_CVC_DATATYPE_VALID_1_2_1: Element '{http://services.sp.pas.ng.org}DateTimeStamp': '2018-07-25T13:01' is not a valid value of the atomic type 'xs:dateTime'.</faultstring>
<faultactor></faultactor>
</soap11env:Fault>
I would like to know how to handle the schema validation error gracefully and return the details in the Details field of my service's out_message, rather than just raising a standard Client.SchemaValidationError. I want to store the details of the error as a variable and pass it to my OperationOne function.
Here is my code, I have changed var names for sensitivity.
TNS = "http://services.so.example.org"
class InMessageType(ComplexModel):
__namespace__ = TNS
class Attributes(ComplexModel.Attributes):
declare_order = 'declared'
field_one = Unicode(values=["ONE", "TWO"],
min_occurs=1)
field_two = Unicode(20, min_occurs=1)
field_three = Unicode(20, min_occurs=0)
Confirmation = Unicode(values=["ACCEPTED", "REJECTED"], min_occurs=1)
FileReason = Unicode(200, min_occurs=0)
DateTimeStamp = DateTime(min_occurs=1)
class OperationOneResponse(ComplexModel):
__namespace__ = TNS
class Attributes(ComplexModel.Attributes):
declare_order = 'declared'
ResponseMessage = Unicode(values=["SUCCESS", "FAILURE"], min_occurs=1)
Details = Unicode(min_len=0, max_len=2000)
class ServiceOne(ServiceBase):
#rpc(InMessageType,
_returns=OperationOneResponse,
_out_message_name='OperationOneResponse',
_in_message_name='InMessageType',
_body_style='bare',
)
def OperationOne(ctx, message):
# DO STUFF HERE
# e.g. return {'ResponseMessage': Failure, 'Details': XSDValidationError}
application = Application([ServiceOne],
TNS,
in_protocol=Soap11(validator='lxml'),
out_protocol=Soap11(),
name='ServiceOne',)
wsgi_application = WsgiApplication(application)
if __name__ == '__main__':
pass
I have considered the following approach but I can't quite seem to make it work yet:
create subclass MyApplication with call_wrapper() function overridden.
Instantiate the application with in_protocol=Soap11(validator=None)
Inside the call wrapper set the protocol to Soap11(validator='lxml') and (somehow) call something which will validate the message. Wrap this in a try/except block and in case of error, catch the error and handle it in whatever way necessary.
I just haven't figured out what I can call inside my overridden call_wrapper() function which will actually perform the validation. I have tried protocol.decompose_incoming_envelope() and other such things but no luck yet.

Overriding the call_wrapper would not work as the validation error is raised before it's called.
You should instead use the event subsystem. More specifically, you must register an application-level handler for the method_exception_object event.
Here's an example:
def _on_exception_object(ctx):
if isinstance(ctx.out_error, ValidationError):
ctx.out_error = NicerValidationError(...)
app = Application(...)
app.event_manager.add_listener('method_exception_object', _on_exception_object)
See this test for more info: https://github.com/arskom/spyne/blob/4a74cfdbc7db7552bc89c0e5d5c19ed5d0755bc7/spyne/test/test_service.py#L69
As per your clarification, if you don't want to reply with a nicer error but a regular response, I'm afraid Spyne is not designed to satisfy that use-case. "Converting" an errored-out request processing state to a regular one would needlessly complicate the already heavy request handling logic.
What you can do instead is to HACK the heck out of the response document.
One way to do it is to implement an additional method_exception_document event handler where the <Fault> tag and its contents are either edited to your taste or even swapped out.
Off the top of my head:
class ValidationErrorReport(ComplexModel):
_type_info = [
('foo', Unicode),
('bar', Integer32),
]
def _on_exception_document(ctx):
fault_elt, = ctx.out_document.xpath("//soap11:Fault", namespaces={'soap11': NS_SOAP11_ENV})
explanation_elt = get_object_as_xml(ValidationErrorReport(...))
fault_parent = fault_elt.parent()
fault_parent.remove(fault_elt)
fault_parent.add(explanation_elt)
The above needs to be double-checked with the relevant Spyne and lxml APIs (maybe you can use find() instead of xpath()), but you get the idea.
Hope that helps!

How to convert suds object to xml string

This is a duplicate to this question:
How to convert suds object to xml
But the question has not been answered: "totxt" is not an attribute on the Client class.
Unfortunately I lack of reputation to add comments. So I ask again:
Is there a way to convert a suds object to its xml?
I ask this because I already have a system that consumes wsdl files and sends data to a webservice. But now the customers want to alternatively store the XML as files (to import them later manually). So all I need are 2 methods for writing data: One writes to a webservice (implemented and tested), the other (not implemented yet) writes to files.
If only I could make something like this:
xml_as_string = My_suds_object.to_xml()
The following code is just an example and does not run. And it's not elegant. Doesn't matter. I hope you get the idea what I want to achieve:
I have the function "write_customer_obj_webservice" that works. Now I want to write the function "write_customer_obj_xml_file".
import suds
def get_customer_obj():
wsdl_url = r'file:C:/somepathhere/Customer.wsdl'
service_url = r'http://someiphere/Customer'
c = suds.client.Client(wsdl_url, location=service_url)
customer = c.factory.create("ns0:CustomerType")
return customer
def write_customer_obj_webservice(customer):
wsdl_url = r'file:C:/somepathhere/Customer.wsdl'
service_url = r'http://someiphere/Customer'
c = suds.client.Client(wsdl_url, location=service_url)
response = c.service.save(someparameters, None, None, customer)
return response
def write_customer_obj_xml_file(customer):
output_filename = r'C\temp\testxml'
# The following line is the problem. "to_xml" does not exist and I can't find a way to do it.
xml = customer.to_xml()
fo = open(output_filename, 'a')
try:
fo.write(xml)
except:
raise
else:
response = 'All ok'
finally:
fo.close()
return response
# Get the customer object always from the wsdl.
customer = get_customer_obj()
# Since customer is an object, setting it's attributes is very easy. There are very complex objects in this system.
customer.name = "Doe J."
customer.age = 42
# Write the new customer to a webservice or store it in a file for later proccessing
if later_processing:
response = write_customer_obj_xml_file(customer)
else:
response = write_customer_obj_webservice(customer)

I found a way that works for me. The trick is to create the Client with the option "nosend=True".
In the documentation it says:
nosend - Create the soap envelope but don't send. When specified, method invocation returns a RequestContext instead of sending it.
The RequestContext object has the attribute envelope. This is the XML as string.
Some pseudo code to illustrate:
c = suds.client.Client(url, nosend=True)
customer = c.factory.create("ns0:CustomerType")
customer.name = "Doe J."
customer.age = 42
response = c.service.save(someparameters, None, None, customer)
print response.envelope # This prints the XML string that would have been sent.

You have some issues in write_customer_obj_xml_file function:
Fix bad path:
output_filename = r'C:\temp\test.xml'
The following line is the problem. "to_xml" does not exist and I can't find a way to do it.
What's the type of customer? type(customer)?
xml = customer.to_xml() # to be continued...
Why mode='a'? ('a' => append, 'w' => create + write)
Use a with statement (file context manager).
with open(output_filename, 'w') as fo:
fo.write(xml)
Don't need to return a response string: use an exception manager. The exception to catch can be EnvironmentError.
Analyse
The following call:
customer = c.factory.create("ns0:CustomerType")
Construct a CustomerType on the fly, and return a CustomerType instance customer.
I think you can introspect your customer object, try the following:
vars(customer) # display the object attributes
help(customer) # display an extensive help about your instance
Another way is to try the WSDL URLs by hands, and see the XML results.
You may obtain the full description of your CustomerType object.
And then?
Then, with the attributes list, you can create your own XML. Use an XML template and fill it with the object attributes.
You may also found the magic function (to_xml) which do the job for you. But, not sure the XML format matches your need.

client = Client(url)
client.factory.create('somename')
# The last XML request by client
client.last_sent()
# The last XML response from Web Service
client.last_received()

Getting Objects with urllib2

I have two GAE apps working in conjunction. One holds an object in a database, the other gets that object from the first app. Below I have the bit of code where the first app is asked for and gives the Critter Object. I am trying to access the first app's object via urllib2, is this really possible? I know it can be used for json but can it be used for objects?
Just for some context I am developing this as a project for a class. The students will learn how to host a GAE app by creating their critters. Then they will give me the url for their critters and my app will use the urls to collect all of their critters then put them into my app's world.
I've only recently heard about pickle, have not looked into yet, might that be a better alternative?
critter.py:
class Access(webapp2.RequestHandler):
def get(self):
creature = CritStore.all().order('-date').get()
if creature:
stats = loads(creature.stats)
return SampleCritter(stats)
else:
return SampleCritter()
map.py:
class Out(webapp2.RequestHandler):
def post(self):
url = self.request.POST['url']#from a simple html textbox
critter = urllib2.urlopen(url)
...work with critter as if it were the critter object...

Yes you can use pickle.
Here is some sample code to transfer an entity, including the key :
entity_dict = entity.to_dict() # First create a dict of the NDB entity
entity_dict['entity_ndb_key_safe'] = entity.key.urlsafe() # add the key of the entity to the dict
pickled_data = pickle.dumps(entity_dict, 1) # serialize the object
encoded_data = base64.b64encode(pickled_data) # encode it for safe transfer
As an alternative for urllib2 you can use the GAE urlfetch.fetch()
In the requesting app you can :
entity_dict = pickle.loads(base64.b64decode(encoded_data))

Unable to view item in browser based on its key in Python GAE

I'm using python GAE with webapp.
I have a form for a user to create a object in the database, something like:
class SpamRecord(db.Model):
author = db.ReferenceProperty(Author, required=True)
text = db.StringProperty()
After it's created, the user is redirected to a page whose URL contains that object's key... using code such as:
spam = SpamRecord(author=author, text=text)
spam.put()
new_spam_key = spam.key()
self.redirect("/view_spam/%s" % new_spam_key)
And this mostly works, with me being able to view items at:
sitename.com/view_spam/ag1waWNreXByZXNlbnRzchQLEgxBbm5vdW5jZW1lbnQYy8oJDA
sitename.com/view_spam/ag1waWNreXByZXNlbnRzchQLEgxBbm5vdW5jZW1lbnQY_boJDA
However, there's an occasional key that won't work. Here are 2 recent examples of pages that won't load and return HTTP 404 not found errors:
sitename.com/view_spam/ag1waWNreXByZXNlbnRzchQLEgxBbm5vdW5jZW1lbnQY-5MJDA
sitename.com/view_spam/ag1waWNreXByZXNlbnRzchQLEgxBbm5vdW5jZW1lbnQY-boJDA
My html-mappings.py contains the following mapping:
(r"/view_spam/(\w+)", ViewSpamPage)
And the ViewSpamPage looks something like:
class ViewSpamPage(webapp.RequestHandler):
def get(self, spam_id):
self.response.out.write("Got here")
Can anyone offer any insight as to why this is occurring and how it may be prevented?
Thanks very much!

In regular expressions, \w doesn't match hyphens. (It will match underscores.) For that second pair of keys, this'll result in only passing part of the key to your handler.
In your URL pattern, try r"/view_spam/(.*)" instead.

how can i get value fron the URL in python appengine?

if i have this URL in from python code on appengine
http://localhost:8080/blog/view/2f1cab5844fb432b8426ae666c4ac493
how can i get the value of the key : 2f1cab5844fb432b8426ae666c4ac493

#Herms answer will work, but you may prefer this instead:
In the code that creates your webapp instance, capture the key part of the URL with a regex, like:
def main():
application = webapp.WSGIApplication( [
(r'/blog/view/(\w+)', MyBlogViewHandler),
## others listed here...
])
...then code your handler class like this - the key you captured will be passed to your get() method as an argument:
class MyBlogViewHandler(webapp.RequestHandler):
def get(self, key):
# do something useful with the 'key' argument

You can access the requested URL via self.request, assuming you're extending the standard webapp.RequestHandler class. That will give you access to the path and query, and you should be able to extract the values you want from the path.
Here's some documentation on the Request object:
http://code.google.com/appengine/docs/python/tools/webapp/requestclass.html