Please advise library for working with soap in python.
Now, I'm trying to use "suds" and I can't undestand how get http headers from server reply
Code example:
from suds.client import Client
url = "http://10.1.0.36/money_trans/api3.wsdl"
client = Client(url)
login_res = client.service.Login("login", "password")
variable "login_res" contain xml answer and doesnt contain http headers. But I need to get session id from them.
I think you actually want to look in the Cookie Jar for that.
client = Client(url)
login_res = client.service.Login("login", "password")
for c in client.options.transport.cookiejar:
if "sess" in str(c).lower():
print "Session cookie:", c
I'm not sure. I'm still a SUDS noob, myself. But this is what my gut tells me.
The response from Ishpeck is on the right path. I just wanted to add a few things about the Suds internals.
The suds client is a big fat abstraction layer on top of a urllib2 HTTP opener. The HTTP client, cookiejar, headers, request and responses are all stored in the transport object. The problem is that none of this activity is cached or stored inside of the transport other than, maybe, the cookies within the cookiejar, and even tracking these can sometimes be problematic.
If you want to see what's going on when debugging, my suggestion would be to add this to your code:
import logging
logging.basicConfig(level=logging.INFO)
logging.getLogger('suds.client').setLevel(logging.DEBUG)
logging.getLogger('suds.transport').setLevel(logging.DEBUG)
Suds makes use of the native logging module and so by turning on debug logging, you get to see all of the activity being performed underneath including headers, variables, payload, URLs, etc. This has saved me tons of times.
Outside of that, if you really need to definitively track state on your headers, you're going to need to create a custom subclass of a suds.transport.http.HttpTransport object and overload some of the default behavior and then pass that to the Client constructor.
Here is a super-over-simplified example:
from suds.transport.http import HttpTransport, Reply, TransportError
from suds.client import Client
class MyTransport(HttpTransport):
# custom stuff done here
mytransport_instance = MyTransport()
myclient = Client(url, transport=mytransport_instance)
I think Suds library has a poor documentation so, I recommend you to use Zeep. It's a SOAP requests library in Python. Its documentation isn't perfect, but it's very much clear than Suds Doc.
Related
So, I am trying to use the pyodata library in Python to access and download data from Odata.
I tried accessing the Northwind data and it worked. So, i guess the codes i used is ok.
import requests
import pyodata
url_t = 'http://services.odata.org/V2/Northwind/Northwind.svc'
# connection set up
northwind = pyodata.Client(url_t, requests.Session())
# This prints out a single value from the table Customers
for customer in northwind.entity_sets.Customers.get_entities().execute():
print(customer.CustomerID,",", customer.CompanyName)
break
# This will print out - ALFKI , Alfreds Futterkiste
I also tried connecting to Odata in excel to see if the codes above return the correct data, and it did.
Click to see the screenshot in excel for Odata connection
Now, using the same code to connect to the data source where I want to pull the data did not work:
#using this link to connect to Odata worked.
url_1 = 'https://batch.decisionkey.npd.com/odata/dkusers'
session = requests.Session()
session.auth = (user_name, psw)
theservice = pyodata.Client(url_1, session)
The above codes return this error message(is it something about security?):
Click to see error message
Connecting to the data in excel looks like this:
Click the view image
I am thinking about it might be security issue that is blocking me from accessing the data, or it could be something else. Please let me know if anything need to be clarify. Thanks.
First time asking question, so please let me know if anything I did not do right here. ^_^
You got HTTP 404 - Not Found.
The service "https://batch.decisionkey.npd.com/odata/dkusers" is not accessible from outside world for me to try it, so there is something more from networking point of view that happens in the second picture in the Excel import.
You can forget the pyodata at the moment, for your problem it is just wrapper around HTTP networking layer, the Requests library. You need to find a way initialize the Requests session in a way, that will return HTTP 200 OK instead.
Northwind example service is just plain and simple, so no problem during initialization of pyodata.Client
Refer to Requests library documentation- https://docs.python-requests.org/en/latest/user/advanced/
//sample script
url_1 = 'https://batch.decisionkey.npd.com/odata/dkusers'
session = requests.Session()
session.auth = (user_name, psw)
//??? SSL certificate needs to be provided perhaps?
//?? or maybe you are behind some proxy that Excel uses but python not.. try ping in CMD
response = session.get(url_1)
print(response.text)
Usable can be pyodata documentation about initialization, however you will not find there the reason why you get HTTP 404 - https://pyodata.readthedocs.io/en/latest/usage/initialization.html
I'm building a new retry feature in my Orchestrate script and I want to know how many times, and, if possible, what error my request method got when trying to connect to a specific URL.
For now, I need this for logging purposes, because I'm working on a messaging system and I may need this 'retry' information to understand when and why I'm facing any kind of problem in HTTP requests, once I work in a micro-service environment.
So far, I debugged and certify that retries are working as expected (I have a mocked flask server for all micro services that we use), but I couldn't find a way to got the 'retries history' data.
In other words, for example, I want to see if a specific micro-service may respond only after the third request, and those kind of thing.
Below is the code that I'm using now:
from requests import exceptions, Session
from urllib3.util.retry import Retry
from requests.adapters import HTTPAdapter
def open_request_session():
# Default retries configs
toggle = True #loaded from config file
session = Session()
if toggle:
# loaded from config file as
parameters = {'total': 5,
'backoff_factor': 0.0,
'http_error_to_retry': [400]}well
retries = Retry(total=parameters['total'],
backoff_factor=parameters['backoff_factor'],
status_forcelist=parameters['http_error_to_retry'],
# Do not force an exception when False
raise_on_status=False)
session.mount('http://', HTTPAdapter(max_retries=retries))
session.mount('https://', HTTPAdapter(max_retries=retries))
return session
# Request
with open_request_session() as request:
my_response = request.get(url, timeout=10)
I see in the urllib3 documentation that Retry has a history attribute, but when I try to consult the attribute it is empty.
I don't know if I'm doing wrong or forgetting something, once Software Development is not my best skill.
So, I have two questions:
Does anyone know a way to got this history information?
How can I do create tests to verify if the retries behavior is working as expected? (So far I only test in debug mode)
I'm using Python 3.6.8.
I know that I can create a while statement to 'control' this, but I'm trying to avoid complexity. And this is why I'm here, I'm looking for an alternative based on Python and community best practices.
A bit late, but I just figured this out myself so thought I'd share what I found.
Short answer:
response.raw.retries.history
will get you what you are looking for.
Long answer:
You cannot get the history off the original Retry instance created. Under the covers, urllib3 creates a new Retry instance for every attempt.
urllib3 does store the last Retry instance on the response when one is returned. However, the response from the requests library is a wrapper around the urllib3 response. Luckily, requests stores the original urllib3 response on the raw field.
I need to somehow extract plain HTTP request message from a Request object in Scrapy (so that I could, for example, copy/paste this request and run from Burp).
So given a scrapy.http.Request object, I would like to get the corresponding request message, such as e.g.
POST /test/demo_form.php HTTP/1.1
Host: w3schools.com
name1=value1&name2=value2
Clearly I have all the information I need in the Request object, however trying to reconstruct the message manually is error-prone as I could miss some edge cases. My understanding is that Scrapy first converts this Request into Twisted object, which then writes headers and body into a TCP transport. So maybe there's away to do something similar, but write to a string instead?
UPDATE
I could use the following code to get HTTP 1.0 request message, which is based on http.py. Is there a way to do something similar with HTTP 1.1 requests / http11.py, which is what's actually being sent? I would obviously like to avoid duplicating code from Scrapy/Twisted frameworks as much as possible.
factory = webclient.ScrapyHTTPClientFactory(request)
transport = StringTransport()
protocol = webclient.ScrapyHTTPPageGetter()
protocol.factory = factory protocol.makeConnection(transport)
request_message = transport.value()
print(request_message.decode("utf-8"))
As scrapy is open source and also has plenty of extension points, this should be doable.
The requests are finally assembled and sent out in scrapy/core/downloader/handlers/http11.py in ScrapyAgent.download_request ( https://github.com/scrapy/scrapy/blob/master/scrapy/core/downloader/handlers/http11.py#L270 )
If you place your hook there you can dump the request type, request headers, and request body.
To place your code there you can either try monkey patching ScrapyAgent.download_request or to subclass ScrapyAgent to do the request logging, then subclass HTTP11DownloadHandler to use your Scrapy Agent and then set HTTP11DownloadHandler as new DOWNLOAD_HANDLER for http / https requests in your project's settings.py (for details see: https://doc.scrapy.org/en/latest/topics/settings.html#download-handlers)
In my opinion this is the closest you can get to logging the requests going out without using a packet sniffer or a logging proxy (which might be a bit overkill for your scenario).
In the Flask documentation on testing (http://flask.pocoo.org/docs/testing/), it has a line of code
rv = self.app.get('/')
And below it, it mentions "By using self.app.get we can send an HTTP GET request to the application with the given path."
Where can the documentation be found for these direct access methods (I'm assuming that there's one for all of the restful methods)? Specifically, I'm wondering what sort of arguments they can take (for example, passing in data, headers, etc). Looking around on flask's documentation for a Flask object, it doesn't seem to list these methods, even though it uses them in the above example.
Alternatively, a knowledgeable individual could answer what I am trying to figure out: I'm trying to simulate sending a POST request to my server, as I would with the following line, if I were doing it over HTTP:
res = requests.post("http://localhost:%d/generate" % port,
data=json.dumps(payload),
headers={"content-type": "application/json"})
The above works when running a Flask app on the proper port. But I tried replacing it with the following:
res = self.app.post("/generate",
data=json.dumps(payload),
headers={"content-type": "application/json"})
And instead, the object I get in response is a 400 BAD REQUEST.
This is documented in the Werkzeug project, from which Flask gets the test client: Werkzeug's test client.
The test client does not issue HTTP requests, it dispatches requests internally, so there is no need to specify a port.
The documentation isn't very clear about support for JSON in the body, but it seems if you pass a string and set the content type you should be fine, so I'm not exactly sure why you get back a code 400. I would check if your /generate view function is invoked at all. A debugger should be useful to figure out where is the 400 coming from.
I want to convert a perl SOAP client into a python SOAP client.
The perl client is initialized like
$url = 'https://host:port/cgi-devel/Service.cgi';
$uri = 'https://host/Service';
my $soap = SOAP::Lite
-> uri($uri)
-> proxy($url);
I tried to replicate this in python 2.4.2 with suds 0.3.6 doing
from suds.client import Client
url="https://host:port/cgi-devel/Service.cgi"
client=Client(url)
However when running this python script I get the error
suds.transport.TransportError: HTTP Error 411: Length Required
Is it because of https or what might be the problem?
Any help would be greatly appreciated!
urllib2 module doesn't add Content-Length (required for POST method) header automatically when Request object is constructed manually as suds does. You have to patch suds, probably suds.transport.HttpTransport.open() method or suds.transport.Request class.
I had the same error, then switched to using a local WSDL file, this worked:
import suds
wsdl = 'file:///tmp/my.wsdl'
client = suds.client.Client(wsdl, username='lbuser', password='lbpass', location='https://path.to.our.loadbalancer:9090/soap')
You should ask this in the suds's mailing list. This library is under development, is open source, and the authors are very keen to get feedback from the users.
Your code looks fine, this could be an error of the wsdl itself or of the suds library, therefore I encourage you to ask the author directly (after having checked with other wsdls that your installation is correct).