I am trying to communicate to a webservice using Suds, reading from the service works fine, however writing throws an error.
suds.WebFault: Server raised fault: 'The formatter threw an exception
while trying to deserialize the message: There was an error while
trying to deserialize parameter http://tempuri.org/:tagValues. The
InnerException message was 'Element Value from namespace
http://schemas.datacontract.org/2004/07/NOV.Api.Messages cannot have
child contents to be deserialized as an object. Please use XmlNode[]
to deserialize this pattern of XML.'. Please see InnerException for
more details.'
The XML produces does not seem to add the neccessary xsi:type="xsd:int"
Produced:
<ns1:TagValue>
<ns1:Quality>
<ns1:Id>1</ns1:Id>
<ns1:QualityData>Quality</ns1:QualityData>
</ns1:Quality>
<ns1:TagID>
<ns1:Id>0</ns1:Id>
<ns1:TagID>BitDepth</ns1:TagID>
</ns1:TagID>
<ns1:Value>23</ns1:Value>
</ns1:TagValue>
Expected:
<ns1:TagValue>
<ns1:Quality>
<ns1:Id>1</ns1:Id>
<ns1:QualityData>Quality</ns1:QualityData>
</ns1:Quality>
<ns1:TagID>
<ns1:Id>0</ns1:Id>
<ns1:TagID>BitDepth</ns1:TagID>
</ns1:TagID>
<ns1:Value xsi:type="xsd:int">23</ns1:Value>
</ns1:TagValue>
After searching around i figured to try the ImportDoctor to see if i could get in the xsi:type
I added
schema_url = 'http://schemas.xmlsoap.org/soap/encoding/'
schema_import = Import(schema_url)
schema_doctor = ImportDoctor(schema_import)
and doctor=schema_doctor in the Client ctor
This now gave me an additional prefix and a much extended list of Types
Prefixes (4)
ns0 = "http://schemas.datacontract.org/2004/07/NOV.Api.Messages"
ns1 = "http://schemas.microsoft.com/2003/10/Serialization/"
ns2 = "http://schemas.xmlsoap.org/soap/encoding/"
ns3 = "http://tempuri.org/"
I now have a ns2:int
I used the factory to create an object of type ns2:int, setting its value to 23
When sending this, i get the following XML:
<ns1:TagValue>
<ns1:Quality>
<ns1:Id>1</ns1:Id>
<ns1:QualityData>Quality</ns1:QualityData>
</ns1:Quality>
<ns1:TagID>
<ns1:Id>0</ns1:Id>
<ns1:TagID>BitDepth</ns1:TagID>
</ns1:TagID>
<ns1:Value xsi:type="ns2:int">23</ns1:Value>
</ns1:TagValue>
I now get the following exception when trying to send it:
suds.WebFault: Server raised fault: 'The formatter threw an exception
while trying to deserialize the message: There was an error while
trying to deserialize parameter http://tempuri.org/:tagValues. The
InnerException message was 'Error in line 1 position 651. Element
'http://schemas.datacontract.org/2004/07/NOV.Api.Messages:Value'
contains data from a type that maps to the name 'http://schemas.xm
lsoap.org/soap/encoding/:int'. The deserializer has no knowledge of
any type that maps to this name. Consider using a DataContractResolver
or add the type corresponding to 'int' to the list of known types -
for example, by using the KnownTypeAttribute attribute or by adding it
to the list of known types passed to DataContractSerializer.'. Please
see InnerException for more details.'
Seems slightly closer, but seems like there is some mess with namespaces?
Full XML produced:
<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:ns3="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns0="http://tempuri.org/" xmlns:ns1="http://schemas.datacontract.org/2004/07/NOV.Api.Messages" xmlns:ns2="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header/>
<ns3:Body>
<ns0:WriteRealtimeValues>
<ns0:tagValues>
<ns1:TagValue>
<ns1:Quality>
<ns1:Id>1</ns1:Id>
<ns1:QualityData>Quality</ns1:QualityData>
</ns1:Quality>
<ns1:TagID>
<ns1:Id>0</ns1:Id>
<ns1:TagID>BitDepth</ns1:TagID>
</ns1:TagID>
<ns1:Value xsi:type="ns2:int">23</ns1:Value>
</ns1:TagValue>
</ns0:tagValues>
</ns0:WriteRealtimeValues>
</ns3:Body>
</SOAP-ENV:Envelope>
As reference, I create the client using the following code
credentials = dict(username='%s' % (username), password='%s' % password)
url= "http://%s:%s/TagValueWriteService?wsdl" % (ip,port)
self.transport = HttpAuthenticated(**credentials)
suds.client.Client.__init__(self,url, transport=self.transport, cache=None,doctor=schema_doctor)
There seem to be several similar issues here on stackoverflow, most of them mentioning the ImportDoctor in a similar manner as i tried. I am lacking some of the fundamental understanding of SOAP i suspect...
I managed to solve it, using the answer from Adding xsi:type and envelope namespace when using SUDS ( https://stackoverflow.com/a/10977734/696768 )
I am not sure this is the only possible solution, and to me it seems more of a hack than anything else, however it will work fine for my current scenario.
The solution i used, is making a plugin for the client, looking for the particular element that i need to be xsi:type="xsd:int", then adding these attributes to those elements.
The code i ended up using for reference (from the aforementioned stackoverflow question with minor adjustments):
from suds.plugin import MessagePlugin
from suds.sax.attribute import Attribute
class SoapFixer(MessagePlugin):
def marshalled(self, context):
# Alter the envelope so that the xsd namespace is allowed
context.envelope.nsprefixes['xsd'] = 'http://www.w3.org/2001/XMLSchema'
# Go through every node in the document and apply the fix function to patch up incompatible XML.
context.envelope.walk(self.fix_any_type_string)
def fix_any_type_string(self, element):
"""Used as a filter function with walk in order to fix errors.
If the element has a certain name, give it a xsi:type=xsd:int. Note that the nsprefix xsd must also
be added in to make this work."""
# Fix elements which have these names
fix_names = ['Value', 'anotherelementname']
if element.name in fix_names:
element.attributes.append(Attribute('xsi:type', 'xsd:int'))
plugin=SoapFixer()
Then I added plugins=[plugin] to the client ctor.
Example:
client = suds.client.Client("http://127.0.0.1:8099/TagValueWriteService?wsdl",plugins=[plugin])
This is not an 'answer' because this question is client-side. But I'm putting this here for the search engines for now.
The problem is the request message is a complex type.
My solution was on the server side. My service now accepts untyped elements in the request.
The server-side parsing of the request body must know about the Request schema. Once that happens, the server can typecheck and parse the request without the elements being typed by the client.
Specifically, My error came from a service implemented with Python ZSI module, and Zope.
Any cannot parse untyped element
Here, I got the hint on complex request objects:
http://pypi.python.org/pypi/z3c.soap/ (see ValidateEmailRequest)
Here, I got a crash course in ZSI:
Are there any working examples of Zolera SOAP Infrastructure (ZSI)?
And decent ZSI docs here: http://pywebsvcs.sourceforge.net/zsi.html#SECTION0071100000000000000000
To make ZSI happy, you just have to create a class that represents the Request message, and add a typecode to it. This is why you see a lot of services to "operation foo" and "fooRequest" and "fooResponse", so they can type the request and response objects as xml complex types.
for the example above, I would import something like this into the namespace where the soap request body is being parsed. You can get much more complicated, but this is really all that's necessary:
import ZSI
class WriteRealTimeValuesRequest(object):
tagValues = array() #of TagValue
WriteRealTimeValuesRequest.typecode = ZSI.TC.Struct(WriteRealTimeValuesRequest,
(ZSI.TC.Array("TagValue",
TagValue.typecode,
"tagValues"
),
),
"WriteRealTimeValuesRequest")
"Whats a tag value?"
class TagValue(object):
Quality = Quality
TagId = TagId
Value = Value
TagValue.typecode = ZSI.TC.Struct(TagValue,
(Quality.typecode,
TagId.typecode,
Value.typecode),
"TagValue")
What's a Quality?
class Quality(object):
Id = 0
QualityData = "I'm a string"
Quality.typecode = ZSI.TC.Struct(Quality,
(ZSI.TC.Integer("Id"), #this is the secret sauce
ZSI.TC.String("QualityData") #and here
),
"Quality")
And so on, until you've drilled down all the way to all the primitive types.
Related
I am using boto3 to tag EC2, and the script was working fine but out of blue somehow its no more tagging it. The peculiar thing is that its not returning any error. I mess the ARN up with dummy digits or no ARN at all just few digits, Still no error. Any Ideas what caused this?
# client = session.resource('ec2')
# This is the client that is passed into the function
def tag_ec2(client, cache, logger):
try:
resource_id = cache['ARN'].split(':')[-1]
response = client.Tag(
resource_id,
cache['key'],
cache['value']
)
print(response)
except Exception as e:
logger.error(f"Exception occurred while tagging {cache['resource']}: {e}.")
return str(e)
return None
This is the function that I use. No matter what ARN I give a real one or a fake one, It never throws any error or tag the instance with the real one either.
I've tried it on other EC2 resources like subnet and it works fine with them.
The documentation says, client.Tag represents a tag, you can think of it as accessing one of the tags in the instance as an object. It is not actually meant to 'create' tags. In that sense, it does make sense that it never returns an error b/c it may actually be your intention that you will first create the object and then do something with it (see resource's available actions here: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ec2.html#networkinterface )
You can use instance resource's create_tags method as follows (see ref here):
# client = session.resource('ec2')
instance = client.Instance(resource_id)
response= instance.create_tags(Tags=[{'Key': cache['key'], 'Value': cache['Value']}])
and check results with
instance.tags
There are other ways you can use service resource. Actually, it is probably confusing for people to see you are referring to ec2 service resource as client, so let's use correct terminology as follows:
resource = session.resource('ec2')
tags = [{'Key': cache['key'], 'Value': cache['Value']}]
response = resource.create_tags(Resources=[resource_id], Tags=tags)
Note that when you directly use Service Resource, you need to specify both Tags and resource id(s) as explained here.
First of all, the documentation and examples are so bad, please explain for simple people, like me. Rant over.
I am trying to access my organizations ldap server. Ive tried python-ldap, but i ran into issues, where it decodes and encodes responses super weird. I am testing ldap3 lib atm. But i cannot use the same stuff using ldap3.
def ldap3_connection_initalizer(username, password):
server = ldap3.Server('ldap.blaah.blaaah')
connection = ldap3.Connection(server, user=f'uid={username},ou=people,ou=users,dc=blaah,dc=blaah', password=f'{password}')
with ldap3.Connection(server, user=f'uid={username},ou=people,ou=users,dc=blaah,dc=blaaah', password=f'{password}', auto_bind = True) as c:
print(c)
base_dn='ou=people,ou=users,dc=blaaah,dc=blaaah'
status, result, response, _ = c.search(search_base=base_dn, search_filter='(objectClass=group)', search_scope='SUBTREE', attributes = ['member'])
print(status)
print(result)
print(response)
Running this I get : ldap3.core.exceptions.LDAPObjectClassError: invalid class in objectClass attribute: group
and before that raise LDAPObjectClassError('invalid class in objectClass attribute: ' + str(value))
Could anyone explain why is this not working, but examples in the internet use similar response and it works?
EDIT: Update, tried to change group to person and got this error TypeError: cannot unpack non-iterable bool object
The 1st error says that group is not a valid objectClass (actually it exists but is specific to some Active Directory groups - AD only), so you need to fix the filter. The most generic objectClass for structuring a group entry (ie. having member attribute) is groupOfNames (cf. LDAP Group).
Then you changed the filter with (objectClass=person) which is valid (but for user entries), so you run into the 2nd error which is related to how is unpacked the returned value(s) from c.search(). By default ldap3' search() function returns only the status (other values are returned only when the "strategy" is defined to be thread-safe, which is not obvious at all).
When running a search the response is always added to the connection object via the response keyword, so in order to get the actual ldap entries whatever the thread mode, we can just iterate the connection response.
To sum it up, you can do :
status = c.search(search_base=base_dn, search_filter='(objectClass=groupOfNames)', search_scope='SUBTREE', attributes = ['member'])
print(status)
for entry in c.response:
print(entry['dn'])
print(entry['attributes'])
I am implementing a cloud bursting system with Softlayer instances and Slurm. But I got a problem with Python Softlayer API.
When I try to get a list of some specific instances with the API call SoftLayer.VSManager.list_instances() I use the parameter 'tags', since I tagged the instances to classify them. But it does not work as expected.
It is supposed to find instances whose 'tagReferences' field matches with the value of the parameter 'tags' you passed in the API call.
However, I get a list with all the nodes whose 'tagReferences' field is not empty. Whatever is the value I pass as 'tags' parameter.
I have the following nodes:
hostname: 'node000' tags: 'slurm, node'
hostname: 'node005' tags: 'test'
I run this script:
import os
import SoftLayer
os.environ["SL_USERNAME"] = "***"
os.environ["SL_API_KEY"] = "******"
client = SoftLayer.Client()
mgr = SoftLayer.VSManager(client)
for vsi in mgr.list_instances(tags = 'slurm'):
print vsi['hostname']
This is the output I get:
node000
node005
I tried passing different values as 'tags' parameter (see below), but I always get the same result shown above, even with the last one.
Set of values passed as 'tags' parameter:
slurm, node
slurm
node
test
random
Did I miss anything?
I wrote a ticket to Softlayer support team but they believe my script should work and they assured me that the tags feature does work. Even they told me explicitly to come here to ask because they have no idea of what is happening.
According the documentation of the method that you are using, you need to send a list of tags, so change the string by a list like this:
client = SoftLayer.Client()
mgr = SoftLayer.VSManager(client)
for vsi in mgr.list_instances(tags = ['mytag']):
print (vsi['hostname'])
Regards
I'm trying to make a Python client that can consume a .NET (soap) web service. I've been looking at the suds library to do so, as it appears to be frequently recommended. This is what I have so far:
from suds.client import Client
from suds.transport.https import WindowsHttpAuthenticated
ntlm_transport = WindowsHttpAuthenticated(username='myUserName', password='myPassword')
client = Client('http://server:port/path?wsdl', transport=ntlm_transport)
some_complex_type = client.factory.create('SomeComplexType')
# HOW DO I SET PROPERTIES OF some_complex_type IF THE WSDL DOESN'T DEFINE WHAT IT LOOKS LIKE?
return_value = client.service.MethodThatUsesSomeComplexType(some_complex_type)
From the suds documentation for Complex Arguments, it appears that typically if you print client.factory.create('SomeComplexType') it will output what properties that complex type has (according to the wsdl). In my case, however, if I do: print some_complex_type, I get "<empty>", which, I guess, means that the wsdl is missing the definition for SomeComplexType (aside from stating that it exists).
Does the maker of the web service that I'm consuming just have things set up incorrectly? Or is there a special/different way that Microsoft defines types in wsdl, and I just need to configure my suds Client differently?
I've found that when using the suds library, as in the example above, if a complex type definition is missing from the wsdl, you can send a python dictionary, with the values that the remote service ultimately wants, in place of the complex type.
For example, using the example above, instead of doing the following:
some_complex_type = client.factory.create('SomeComplexType')
some_complex_type.foo = 'Hello' # borks is the definition of SomeComplexType is missing from the wsdl
some_complex_type.bar = 'World'
return_value = client.service.MethodThatUsesSomeComplexType(some_complex_type)
you can just do the following:
some_complex_type = {
'foo': 'Hello'
'bar': 'World'
}
return_value = client.service.MethodThatUsesSomeComplexType(some_complex_type)
The some_complex_type dictionary can also contain sub-dictioaries and sub-lists of other dictionaries, etc. The form of the dictionary tree should match the form of the XML that the web service ultimately wants to receive.
my question is quite hard to describe, so I will focus on explaining the situation. So let's say I have 2 different entities, which may run on different machines. Let's call the first one Manager and the second one Generator. The manager is the only one which can be called via the user.
The manager has a method called getVM(scenario_Id), which takes the ID of a scenario as a parameter, and retrieve a BLOB from the database corresponding to the ID given as a parameter. This BLOB is actually a XML structure that I need to send to the Generator. Both have a Flask running.
On another machine, I have my generator with a generateVM() method, which will create a VM according to the XML structure it recieves. We will not talk about how the VM is created from the XML.
Currently I made this :
Manager
# This method will be called by the user
#app.route("/getVM/<int:scId>", methods=['GET'])
def getVM(scId):
xmlContent = db.getXML(scId) # So here is what I want to send
generatorAddr = sgAdd + "/generateVM" # sgAdd is declared in the Initialize() [IP of the Generator]
# Here how should I put my data ?
# How can I transmit xmlContent ?
testReturn = urlopen(generatorAddr).read()
return json.dumps(testReturn)
Generator
# This method will be called by the user
#app.route("/generateVM", methods=['POST'])
def generateVM():
# Retrieve the XML content...
return "Whatever"
So as you can see, I am stuck on how to transmit the data itself (the XML structure), and then how to treat it... So if you have any strategy, hint, tip, clue on how I should proceed, please feel free to answer. Maybe there are some things I do not really understand about Flask, so feel free to correct everything wrong I said.
Best regards and thank you
PS : Lines with routes are commented because they mess up the syntax coloration
unless i'm missing something couldn't you just transmit it in the body of a post request? Isn't that how your generateVM method is setup?
#app.route("/getVM/<int:scId>", methods=['GET'])
def getVM(scId):
xmlContent = db.getXML(scId)
generatorAddr = sgAdd + "/generateVM"
xml_str = some_method_to_generate_xml()
data_str = urllib.urlencode({'xml': xml_str})
urllib.urlopen(generatorAddr, data=data_str).read()
return json.dumps(testReturn)
http://docs.python.org/2/library/urllib.html#urllib.urlopen