We Keep Coding

How to convert suds object to xml string

This is a duplicate to this question:
How to convert suds object to xml
But the question has not been answered: "totxt" is not an attribute on the Client class.
Unfortunately I lack of reputation to add comments. So I ask again:
Is there a way to convert a suds object to its xml?
I ask this because I already have a system that consumes wsdl files and sends data to a webservice. But now the customers want to alternatively store the XML as files (to import them later manually). So all I need are 2 methods for writing data: One writes to a webservice (implemented and tested), the other (not implemented yet) writes to files.
If only I could make something like this:
xml_as_string = My_suds_object.to_xml()
The following code is just an example and does not run. And it's not elegant. Doesn't matter. I hope you get the idea what I want to achieve:
I have the function "write_customer_obj_webservice" that works. Now I want to write the function "write_customer_obj_xml_file".
import suds
def get_customer_obj():
wsdl_url = r'file:C:/somepathhere/Customer.wsdl'
service_url = r'http://someiphere/Customer'
c = suds.client.Client(wsdl_url, location=service_url)
customer = c.factory.create("ns0:CustomerType")
return customer
def write_customer_obj_webservice(customer):
wsdl_url = r'file:C:/somepathhere/Customer.wsdl'
service_url = r'http://someiphere/Customer'
c = suds.client.Client(wsdl_url, location=service_url)
response = c.service.save(someparameters, None, None, customer)
return response
def write_customer_obj_xml_file(customer):
output_filename = r'C\temp\testxml'
# The following line is the problem. "to_xml" does not exist and I can't find a way to do it.
xml = customer.to_xml()
fo = open(output_filename, 'a')
try:
fo.write(xml)
except:
raise
else:
response = 'All ok'
finally:
fo.close()
return response
# Get the customer object always from the wsdl.
customer = get_customer_obj()
# Since customer is an object, setting it's attributes is very easy. There are very complex objects in this system.
customer.name = "Doe J."
customer.age = 42
# Write the new customer to a webservice or store it in a file for later proccessing
if later_processing:
response = write_customer_obj_xml_file(customer)
else:
response = write_customer_obj_webservice(customer)

I found a way that works for me. The trick is to create the Client with the option "nosend=True".
In the documentation it says:
nosend - Create the soap envelope but don't send. When specified, method invocation returns a RequestContext instead of sending it.
The RequestContext object has the attribute envelope. This is the XML as string.
Some pseudo code to illustrate:
c = suds.client.Client(url, nosend=True)
customer = c.factory.create("ns0:CustomerType")
customer.name = "Doe J."
customer.age = 42
response = c.service.save(someparameters, None, None, customer)
print response.envelope # This prints the XML string that would have been sent.

You have some issues in write_customer_obj_xml_file function:
Fix bad path:
output_filename = r'C:\temp\test.xml'
The following line is the problem. "to_xml" does not exist and I can't find a way to do it.
What's the type of customer? type(customer)?
xml = customer.to_xml() # to be continued...
Why mode='a'? ('a' => append, 'w' => create + write)
Use a with statement (file context manager).
with open(output_filename, 'w') as fo:
fo.write(xml)
Don't need to return a response string: use an exception manager. The exception to catch can be EnvironmentError.
Analyse
The following call:
customer = c.factory.create("ns0:CustomerType")
Construct a CustomerType on the fly, and return a CustomerType instance customer.
I think you can introspect your customer object, try the following:
vars(customer) # display the object attributes
help(customer) # display an extensive help about your instance
Another way is to try the WSDL URLs by hands, and see the XML results.
You may obtain the full description of your CustomerType object.
And then?
Then, with the attributes list, you can create your own XML. Use an XML template and fill it with the object attributes.
You may also found the magic function (to_xml) which do the job for you. But, not sure the XML format matches your need.

client = Client(url)
client.factory.create('somename')
# The last XML request by client
client.last_sent()
# The last XML response from Web Service
client.last_received()

Python, CouchDb: how to Update already existing document by ID

I am trying to update an already existing document by ID. My intention is to find the doc by its id, then change its "firstName" with new value coming in "json", then update it into the CouchDB database.
Here is my code:
def updateDoc(self, id, json):
doc = self.db.get(id)
doc["firstName"] = json["firstName"]
doc_id, doc_rev = self.db.save(doc)
print doc_id, doc_rev
print "Saved"
//"json" is retrieved from PUT request (request.json)
at self.db.save(doc) I'm getting exception as "too many values to unpack".
I am using Bottle framework, Python 2.7 and Couch Query.
How do I update the document by id? what is the right way to do it?

In couchdb-python the db.save(doc) method returns tuple of _id and _rev. You're using couch-query - a bit different project that also has a db.save(doc) method, but it returns a different result. So your code should look like this:
def updateDoc(self, id, json):
doc = self.db.get(id)
doc["firstName"] = json["firstName"]
doc = self.db.save(doc)
print doc['_id'], doc['_rev']
print "Saved"

Updating DataStore JSON values using endpoints (Python)

I am trying to use endpoints to update some JSON values in my datastore. I have the following Datastore in GAE...
class UsersList(ndb.Model):
UserID = ndb.StringProperty(required=True)
ArticlesRead = ndb.JsonProperty()
ArticlesPush = ndb.JsonProperty()
In general what I am trying to do with the API is have the method take in a UserID and a list of articles read (with an article being represented by a dictionary holding an ID and a boolean field saying whether or not the user liked the article). My messages (centered on this logic) are the following...
class UserID(messages.Message):
id = messages.StringField(1, required=True)
class Articles(messages.Message):
id = messages.StringField(1, required=True)
userLiked = messages.BooleanField(2, required=True)
class UserIDAndArticles(messages.Message):
id = messages.StringField(1, required=True)
items = messages.MessageField(Articles, 2, repeated=True)
class ArticleList(messages.Message):
items = messages.MessageField(Articles, 1, repeated=True)
And my API/Endpoint method that is trying to do this update is the following...
#endpoints.method(UserIDAndArticles, ArticleList,
name='user.update',
path='update',
http_method='GET')
def get_update(self, request):
userID = request.id
articleList = request.items
queryResult = UsersList.query(UsersList.UserID == userID)
currentList = []
#This query always returns only one result back, and this for loop is the only way
# I could figure out how to access the query results.
for thing in queryResult:
currentList = json.loads(thing.ArticlesRead)
for item in articleList:
currentList.append(item)
for blah in queryResult:
blah.ArticlesRead = json.dumps(currentList)
blah.put()
for thisThing in queryResult:
pushList = json.loads(thisThing.ArticlesPush)
return ArticleList(items = pushList)
I am having two problems with this code. The first is that I can't seem to figure out (using the localhost Google APIs Explorer) how to send a list of articles to the endpoints method using my UserIDAndArticles class. Is it possible to have a messages.MessageField() as an input to an endpoint method?
The other problem is that I am getting an error on the 'blah.ArticlesRead = json.dumps(currentList)' line. When I try to run this method with some random inputs, I get the following error...
TypeError: <Articles
id: u'hi'
userLiked: False> is not JSON serializable
I know that I have to make my own JSON encoder to get around this, but I'm not sure what the format of the incoming request.items is like and how I should encode it.
I am new to GAE and endpoints (as well as this kind of server side programming in general), so please bear with me. And thanks so much in advance for the help.

A couple things:
http_method should definitely be POST, or better yet PATCH because you're not overwriting all existing values but only modifying a list, i.e. patching.
you don't need json.loads and json.dumps, NDB does it automatically for you.
you're mixing Endpoints messages and NDB model properties.
Here's the method body I came up with:
# get UsersList entity and raise an exception if none found.
uid = request.id
userlist = UsersList.query(UsersList.UserID == uid).get()
if userlist is None:
raise endpoints.NotFoundException('List for user ID %s not found' % uid)
# update user's read articles list, which is actually a dict.
for item in request.items:
userslist.ArticlesRead[item.id] = item.userLiked
userslist.put()
# assuming userlist.ArticlesPush is actually a list of article IDs.
pushItems = [Article(id=id) for id in userlist.ArticlesPush]
return ArticleList(items=pushItems)
Also, you should probably wrap this method in a transaction.

Obtaining tags from AWS instances with boto

I'm trying to obtain tags from instances in my AWS account using Python's boto library.
While this snippet works correctly bringing all tags:
tags = e.get_all_tags()
for tag in tags:
print tag.name, tag.value
(e is an EC2 connection)
When I request tags from individual instances,
print vm.__dict__['tags']
or
print vm.tags
I'm getting an empty list (vm is actually an instance class).
The following code:
vm.__dict__['tags']['Name']
of course results in:
KeyError: 'Name'
My code was working until yesterday and suddenly I'm not able to get the tags from an instance.
Does anybody know whether there is a problem with AWS API?

You have to be sure that the 'Name' tag exists before accessing it. Try this:
import boto.ec2
conn=boto.ec2.connect_to_region("eu-west-1")
reservations = conn.get_all_instances()
for res in reservations:
for inst in res.instances:
if 'Name' in inst.tags:
print "%s (%s) [%s]" % (inst.tags['Name'], inst.id, inst.state)
else:
print "%s [%s]" % (inst.id, inst.state)
will print:
i-4e444444 [stopped]
Amazon Linux (i-4e333333) [running]

Try something like this:
import boto.ec2
conn = boto.ec2.connect_to_region('us-west-2')
# Find a specific instance, returns a list of Reservation objects
reservations = conn.get_all_instances(instance_ids=['i-xxxxxxxx'])
# Find the Instance object inside the reservation
instance = reservations[0].instances[0]
print(instance.tags)
You should see all tags associated with instance i-xxxxxxxx printed out.

For boto3 you will need to do this.
import boto3
ec2 = boto3.resource('ec2')
vpc = ec2.Vpc('<your vpc id goes here>')
instance_iterator = vpc.instances.all()
for instance in instance_iterator:
for tag in instance.tags:
print('Found instance id: ' + instance.id + '\ntag: ' + tag)

It turned out to be an error in my code. I did not consider the case of having one instance without the tag 'Name'.
There was one instance without the tag "Name" and my code was trying to get this tag from every instance.
When I ran this piece of code in an instance without the tag 'Name',
vm.__dict__['tags']['Name']
I got: KeyError: 'Name'. vm is a AWS instance.
With the instances that actually had this tag set, I didn't have any problem.
Thank you for your help and sorry for asking when it was only my own mistake.

SUDS - programmatic access to methods and types

I'm investigating SUDS as a SOAP client for python. I want to inspect the methods available from a specified service, and the types required by a specified method.
The aim is to generate a user interface, allowing users to select a method, then fill in values in a dynamically generated form.
I can get some information on a particular method, but am unsure how to parse it:
client = Client(url)
method = client.sd.service.methods['MyMethod']
I am unable to programmaticaly figure out what object type I need to create to be able to call the service
obj = client.factory.create('?')
res = client.service.MyMethod(obj, soapheaders=authen)
Does anyone have some sample code?

Okay, so SUDS does quite a bit of magic.
A suds.client.Client, is built from a WSDL file:
client = suds.client.Client("http://mssoapinterop.org/asmx/simple.asmx?WSDL")
It downloads the WSDL and creates a definition in client.wsdl. When you call a method using SUDS via client.service.<method> it's actually doing a whole lot of recursive resolve magic behind the scenes against that interpreted WSDL. To discover the parameters and types for methods you'll need to introspect this object.
For example:
for method in client.wsdl.services[0].ports[0].methods.values():
print '%s(%s)' % (method.name, ', '.join('%s: %s' % (part.type, part.name) for part in method.soap.input.body.parts))
This should print something like:
echoInteger((u'int', http://www.w3.org/2001/XMLSchema): inputInteger)
echoFloatArray((u'ArrayOfFloat', http://soapinterop.org/): inputFloatArray)
echoVoid()
echoDecimal((u'decimal', http://www.w3.org/2001/XMLSchema): inputDecimal)
echoStructArray((u'ArrayOfSOAPStruct', http://soapinterop.org/xsd): inputStructArray)
echoIntegerArray((u'ArrayOfInt', http://soapinterop.org/): inputIntegerArray)
echoBase64((u'base64Binary', http://www.w3.org/2001/XMLSchema): inputBase64)
echoHexBinary((u'hexBinary', http://www.w3.org/2001/XMLSchema): inputHexBinary)
echoBoolean((u'boolean', http://www.w3.org/2001/XMLSchema): inputBoolean)
echoStringArray((u'ArrayOfString', http://soapinterop.org/): inputStringArray)
echoStruct((u'SOAPStruct', http://soapinterop.org/xsd): inputStruct)
echoDate((u'dateTime', http://www.w3.org/2001/XMLSchema): inputDate)
echoFloat((u'float', http://www.w3.org/2001/XMLSchema): inputFloat)
echoString((u'string', http://www.w3.org/2001/XMLSchema): inputString)
So the first element of the part's type tuple is probably what you're after:
>>> client.factory.create(u'ArrayOfInt')
(ArrayOfInt){
_arrayType = ""
_offset = ""
_id = ""
_href = ""
_arrayType = ""
}
Update:
For the Weather service it appears that the "parameters" are a part with an element not a type:
>>> client = suds.client.Client('http://www.webservicex.net/WeatherForecast.asmx?WSDL')
>>> client.wsdl.services[0].ports[0].methods.values()[0].soap.input.body.parts[0].element
(u'GetWeatherByZipCode', http://www.webservicex.net)
>>> client.factory.create(u'GetWeatherByZipCode')
(GetWeatherByZipCode){
ZipCode = None
}
But this is magic'd into the parameters of the method call (a la client.service.GetWeatherByZipCode("12345"). IIRC this is SOAP RPC binding style? I think there's enough information here to get you started. Hint: the Python command line interface is your friend!

According to suds documentation, you can inspect service object with __str()__. So the following gets a list of methods and complex types:
from suds.client import Client;
url = 'http://www.webservicex.net/WeatherForecast.asmx?WSDL'
client = Client(url)
temp = str(client);
The code above produces following result (contents of temp):
Suds ( https://fedorahosted.org/suds/ ) version: 0.3.4 (beta) build: R418-20081208
Service ( WeatherForecast ) tns="http://www.webservicex.net"
Prefixes (1)
ns0 = "http://www.webservicex.net"
Ports (2):
(WeatherForecastSoap)
Methods (2):
GetWeatherByPlaceName(xs:string PlaceName, )
GetWeatherByZipCode(xs:string ZipCode, )
Types (3):
ArrayOfWeatherData
WeatherData
WeatherForecasts
(WeatherForecastSoap12)
Methods (2):
GetWeatherByPlaceName(xs:string PlaceName, )
GetWeatherByZipCode(xs:string ZipCode, )
Types (3):
ArrayOfWeatherData
WeatherData
WeatherForecasts
This would be much easier to parse. Also every method is listed with their parameters along with their types. You could, probably, even use just regular expression to extract information you need.

Here's a quick script I wrote based on the above information to list the input methods suds reports as available on a WSDL. Pass in the WSDL URL. Works for the project I'm currently on, I can't guarantee it for yours.
import suds
def list_all(url):
client = suds.client.Client(url)
for service in client.wsdl.services:
for port in service.ports:
methods = port.methods.values()
for method in methods:
print(method.name)
for part in method.soap.input.body.parts:
part_type = part.type
if(not part_type):
part_type = part.element[0]
print(' ' + str(part.name) + ': ' + str(part_type))
o = client.factory.create(part_type)
print(' ' + str(o))

You can access suds's ServiceDefinition object. Here's a quick sample:
from suds.client import Client
c = Client('http://some/wsdl/link')
types = c.sd[0].types
Now, if you want to know the prefixed name of a type, it's also quite easy:
c.sd[0].xlate(c.sd[0].types[0][0])
This double bracket notation is because the types are a list (hence a first [0]) and then in each item on this list there may be two items. However, suds's internal implementation of __unicode__ does exactly that (i.e. takes only the first item on the list):
s.append('Types (%d):' % len(self.types))
for t in self.types:
s.append(indent(4))
s.append(self.xlate(t[0]))
Happy coding ;)

Once you created WSDL method object you can get information about it from it's __metadata__, including list of it's arguments' names.
Given the argument's name, you can access it's actual instance in the method created. That instance also contains it's information in __metadata__, there you can get it's type name
# creating method object
method = client.factory.create('YourMethod')
# getting list of arguments' names
arg_names = method.__metadata__.ordering
# getting types of those arguments
types = [method.__getitem__(arg).__metadata__.sxtype.name for arg in arg_names]
Disclaimer: this only works with complex WSDL types. Simple types, like strings and numbers, are defaulted to None

from suds.client import Client
url = 'http://localhost:1234/sami/2009/08/reporting?wsdl'
client = Client(url)
functions = [m for m in client.wsdl.services[0].ports[0].methods]
count = 0
for function_name in functions:
print (function_name)
count+=1
print ("\nNumber of services exposed : " ,count)

i needed an example of using suds with objects.
beside the answers found here, i found a very good article
that answered my question even further.
here is a short summary:
first, print the client to see an overview of it's content.
from suds.client import Client client =
Client("https://wsvc.cdiscount.com/MarketplaceAPIService.svc?wsdl")
print client
second, create an instance of a type (using it's name including it's prefix ns*.), and print it, to see it's member data.
HeaderMessage = client.factory.create('ns0:HeaderMessage')
print HeaderMessage
to fill your object's data members, either assign them a scalar value for scalar members, or a dict, to object members.
HeaderMessage.Context = {
"CatalogID": "XXXXX"
"CustomerID": 'XXXXX'
"SiteID": 123
}
members whose type name start with ArrayOf expect a list of objects of the type mentioned in the rest of the type name.
ArrayOfDomainRights = client.factory.create('ns0:ArrayOfDomainRights')
ArrayOfDomainRights.DomainRights = [XXXXXXXXXXXXX, XXXXXXXXXXXX]

i needed an example of using suds with objects.
beside the answers found here, i found a very good article
that answered my question even further.
here is a short summary:
first, print the client to see an overview of it's content.
from suds.client import Client client =
Client("https://wsvc.cdiscount.com/MarketplaceAPIService.svc?wsdl")
print client
second, create an instance of a type (using it's name including it's prefix ns*.), and print it, to see it's member data.
HeaderMessage = client.factory.create('ns0:HeaderMessage')
print HeaderMessage
to fill your object's data members, either assign them a scalar value for scalar members, or a dict, to object members.
HeaderMessage.Context = {
"CatalogID": "XXXXX"
"CustomerID": 'XXXXX'
"SiteID": 123
}
members whose type name start with ArrayOf expect a list of objects of the type mentioned in the rest of the type name.
ArrayOfDomainRights = client.factory.create('ns0:ArrayOfDomainRights')
ArrayOfDomainRights.DomainRights = [XXXXXXXXXXXXX, XXXXXXXXXXXX]