A little context: I am opening this question arose here, after solving an authentication problem. I prefer to open a new one to avoid polluting the previous with comments not related to the original issue, and to give it the proper visibility.
I am working on a SOAP client running in the same intranet as the server, without internet access.
from requests.auth import HTTPBasicAuth
from zeep import Client
from zeep.transports import Transport
wsdl = 'http://mysite.dom/services/MyWebServices?WSDL'
client = Client(wsdl, transport=HTTPBasicAuth('user','pass'), cache=None)
The problem: WSDL contains an import to an external resource located outside the intranet ('import namespace="schemas.xmlsoap.org/soap/encoding/"') and therefore Zeep Client instantiation fails with:
Exception: HTTPConnectionPool(host='schemas.xmlsoap.org', port=80): Max retries exceeded with url: /soap/encoding/ (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f3dab9d30b8>: Failed to establish a new connection: [Errno 110] Connection timed out',))
Question: is it possible (and does it make sense) to create the Zeep Client without accessing the external resource?
As an additional detail, another client written in Java, based on XML rpc ServiceFactory seems to be more resilient to this kind of problem, the service is created (and works) even if no internet connection is available.
Is it really needed to import the namespace from xmlsoap.org?
Edit, after answer from #mvt:
So, I went for the proposed solution, which allows me at the same time to control the access to external resources (read: forbid access to servers different from the one hosting the endpoint).
class MyTransport(zeep.Transport):
def load(self, url):
if not url:
raise ValueError("No url given to load")
parsed_url = urlparse(url)
if parsed_url.scheme in ('http', 'https'):
if parsed_url.netloc == "myserver.ext":
response = self.session.get(url, timeout=self.load_timeout)
response.raise_for_status()
return response.content
elif url == "http://schemas.xmlsoap.org/soap/encoding/":
url = "/some/path/myfile.xsd"
else:
raise
elif parsed_url.scheme == 'file':
if url.startswith('file://'):
url = url[7:]
with open(os.path.expanduser(url), 'rb') as fh:
return fh.read()
I would suggest doing your custom overriding of the URL and calling load() from the super class. This way if the super class code changes (which it has), you would not need to refactor your CustomTransport class.
from zeep.transports import Transport
class CustomTransport(Transport):
def load(self, url):
# Custom URL overriding to local file storage
if url and url == "http://schemas.xmlsoap.org/soap/encoding/":
url = "/path/to/schemas.xmlsoap.org.xsd"
# Call zeep.transports.Transport's load()
return super(CustomTransport, self).load(url)
The way to use the Transports in zeep is described here, but here is a quick example of using the CustomTransport:
from requests import Session
from requests.auth import HTTPBasicAuth
from zeep import Client
session = Session()
client = Client('http://example.com/production.svc?wsdl', transport=CustomTransport(session=session))
client.service.foo()
You could create your own subclass of the tranport class and add additional logic to the load() method so that specific url's are redirected / loaded from the filesystem.
The code is pretty easy i think: https://github.com/mvantellingen/python-zeep/blob/master/src/zeep/transports.py :-)
Related
I am trying to access a server over my internal network under https://prodserver.de/info.
I have the code structure as below:
import requests
from requests.auth import *
username = 'User'
password = 'Hello#123'
resp = requests.get('https://prodserver.de/info/', auth=HTTPBasicAuth(username,password))
print(resp.status_code)
While trying to access this server via browser, it works perfectly fine.
What am I doing wrong?
By default, requests library verifies the SSL certificate for HTTPS requests. If the certificate is not verified, it will raise a SSLError. You check this by disabling the certificate verification by passing verify=False as an argument to the get method, if this is the issue.
import requests
from requests.auth import *
username = 'User'
password = 'Hello#123'
resp = requests.get('https://prodserver.de/info/', auth=HTTPBasicAuth(username,password), verify=False)
print(resp.status_code)
try using requests' generic auth, like this:
resp = requests.get('https://prodserver.de/info/', auth=(username,password)
What am I doing wrong?
I can not be sure without investigating your server, but I suggest checking if assumption (you have made) that server is using Basic authorization, there exist various Authentication schemes, it is also possible that your server use cookie-based solution, rather than headers-based one.
While trying to access this server via browser, it works perfectly
fine.
You might then use developer tools to see what is actually send inside and with request which does result in success.
I am trying to use Zeep to implement a SOAP client, as it seems the only maintained library at the moment:
ZSI looked very good but its latest version on pypi dates 2006
suds seemed to be a popular alternative, but the master is unmaintained since 2011 and there are a lot of forks out there but none seems "official" and "recent" enough to be used in a large project.
So, trying to use Zeep, I am stuck with the authentication required by the server to access the WSDL.
Such operation was quite easy with ZSI:
from ZSI.client import Binding
from ZSI.auth import AUTH
b = Binding(url='http://mysite.dom/services/MyWebServices?WSDL')
b.SetAuth(AUTH.httpbasic, 'userid', 'password')
and I can find something similar in __main__.py of Zeep:
from six.moves.urllib.parse import urlparse
from zeep.cache import InMemoryCache, SqliteCache
from zeep.client import Client
from zeep.transports import Transport
cache = SqliteCache() if args.cache else InMemoryCache()
transport_kwargs = {'cache': cache}
result = urlparse(args.wsdl_file)
if result.username or result.password:
transport_kwargs['http_auth'] = (result.username, result.password)
transport = Transport(**transport_kwargs)
client = Client(args.wsdl_file, transport=transport)
but that does not work in my case, I get an error:
Exception: HTTPConnectionPool(host='schemas.xmlsoap.org', port=80): Max retries exceeded with url: /soap/encoding/ (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f3dab9d30b8>: Failed to establish a new connection: [Errno 110] Connection timed out',))
Probably with the newer Version of zeep the older solution does not work anymore. Here is the new way:
from requests.auth import HTTPBasicAuth # or HTTPDigestAuth, or OAuth1, etc.
from requests import Session
from zeep import Client
from zeep.transports import Transport
session = Session()
session.auth = HTTPBasicAuth(user, password)
client = Client('http://my-endpoint.com/production.svc?wsdl',
transport=Transport(session=session))
For Basic Access Authentication you can use the HTTPBasicAuth class from the requests module, as explained on Zeep documentation http://docs.python-zeep.org/en/master/transport.html:
from requests.auth import HTTPBasicAuth # or HTTPDigestAuth, or OAuth1, etc.
from zeep import Client
from zeep.transports import Transport
client = Client('http://my-endpoint.com/production.svc?wsdl',
transport=Transport(http_auth=HTTPBasicAuth(user, password)))
In my case the API I was working with required WS-Security (WSSE) rather than HTTP.
from zeep import Client
from zeep.wsse.username import UsernameToken
client = Client(<wsdl_url>, wsse=UsernameToken(<username>, <password>)
I am running a webserver based on Flask, which serves a resource being versioned (e.g. installation file of some versioned program). I want to serve my HTTP client with new resource only in case, it already does not have the current version available. If there is new version, I want the client to download the resource and install it.
my Flask server looks like this
import json
import redis
import math
import requests
from flask import Flask,render_template,request
app=Flask(__name__)
#app.route('/version', methods=['GET','POST'])
def getversion():
r_server=redis.Redis("127.0.0.1")
if request.method == 'POST':
jsonobj_recieve=request.data
data=json.loads(jsonobj)
currentversion=r_server.hget('version')
if data == currentversion:
#code to return a 'ok'
else:
#code to return 'not ok' also should send the updated file to the client
else:
return r_server.hget('version')
if __name__ == '__main__':
app.run(
debug=True,
host="127.0.0.1",
port=80
)
my client is very basic:
import sys
import json
import requests
url="http://127.0.0.1/version"
jsonobj=json.dumps(str(sys.argv[1]))
print jsonobj
r=requests.post(url,data=jsonobj)
I will likely have to recode the entire client, this is not a problem but I really have no idea where to start....
Requirements Review
have web app, serving a versioned resource. It can be e.g. file with an applications.
have client, which allows fetching the resource only in case, the version of resource on the server and what client has locally already available differ
the client is aware of version string of the resource
allow client to learn new version string if new version is available
HTTP like design of your solution
If you want to allow downloading an application only in case, the client does not have it already, following design could be used:
use etag header. This usually contains some string describing unique status of resource you want to get from that url. In your case it could be current version number of your application.
in your request, use header "if-none-match", providing version number of your application present at client. This will result in HTTP Status code 306 - Not Modified in case, your client and server share the same version of resource. In case it differs, you would simply provide the content of the resource and use it. Your resource shall also denote in etag current version of the resource and your client shall take note of it, or find new version name from other sources (like from the downloaded file).
This design follows HTTP principles.
Flask serving resource with declaring version in etag
This is focusing on showing the principle, you shall elaborate on providing real content of the resource.
from flask import Flask, Response, request
import werkzeug.exceptions
app = Flask(__name__)
class NotModified(werkzeug.exceptions.HTTPException):
code = 304
def get_response(self, environment):
return Response(status=304)
#app.route('/download/app')
def downloadapp():
currver = "1.0"
if request.if_none_match and currver in request.if_none_match:
raise NotModified
def generate():
yield "app_file_part 1"
yield "app_file_part 2"
yield "app_file_part 3"
return Response(generate(), headers={"etag": currver})
if __name__ == '__main__':
app.run(debug=True)
Client getting resource only, if it is new
import requests
ver = "1.0"
url = "http://localhost:5000/download/app"
req = requests.get(url, headers={"If-None-Match": ver})
if req.status_code == 200:
print "new content of resource", req.content
new_ver = req.headers["etag"]
else:
print "resource did not change since last time"
Alternative solution of web part using web server (e.g. NGINX)
Assuming the resource is static file, which updates only sometime, you shall be able configuring your web server, e.g. NGINX, to serve that resource and declaring in your configuration explicit value for etag header to the version string.
Note, that as it was not requested, this alternative solution is not elaborated here (and was not tested).
Client implementation would not be modified by that (here it pays back the design is following HTTP concepts).
There are multiple ways of achieving this but as this is a Flask app, here's one using HTTP.
If the version is OK, just return a relevant status code, like a 200 OK. You can add a JSON response in the body if that's necessary. If you return a string with flask, the status code will be 200 OK and you can inspect that in your client.
If the version differs, return the URL where the file is located. The client will have to
download the file. That's pretty simple using requests. Here's a typical example for downloading file by streaming requests:
def get(url, chunk_size=1024):
""" Download a file in chunks of n bytes """
fn = url.split("/")[-1] # if you're url is complicated, use urlparse.
stream = requests.get(url, stream=True)
with open(fn, "wb") as local:
for chunk in stream.iter_content(chunk_size=chunk_size):
if chunk:
f.write(chunk)
return fn
This is very simplified. If your file is not static and cannot live on the server (like software update patches probably shouldn't) then you'll have to figure out a way to get the file from a database or generate it on the fly.
I am using the suds package to make API request from one website. I wrote a function which opens up the client to the website and make request.
I am wondering should I or how can I terminate the connection in the end of the function?
I am wondering will the client be something like the MySQLDb.connect that actually opens up many many separate API connections that never close every time I call this function.
from suds.client import Client
import sys, re
def querysearch(reqPartNumber, reqMfg, lock):
try:
client = Client('http://app....')
userInfo = {'id':.., 'password':...}
apiResponse = client.service.getParts(...)
...
print apiResponse
except:
...
SOAP is still an HTTP request, which is stateless. Each request will start a whole new connection, re-auth, etc. Browsers kind of short circuit that with cookies, but SOAP doesn't. So, you don't need to close the connection, it's already closed by the time suds returns your data back to you.
Additionally, looking at the latest source, Client() doesn't define a close or __exit__ method, so there's nothing you really have to do here.
Is there any example code of a cpython (not IronPython) client which can call Windows Communication Foundation (WCF) service?
I used suds.
from suds.client import Client
print "Connecting to Service..."
wsdl = "http://serviceurl.com/service.svc?WSDL"
client = Client(wsdl)
result = client.service.Method(variable1, variable2)
print result
That should get you started. I'm able to connect to exposed services from WCF and a RESTful layer. There needs to be some data massaging to help do what you need, especially if you need to bind to several namespaces.
TL;DR: For wsHttpBinding (SOAP 1.2) use zeep
In case someone is having trouble using suds (or suds-jurko for that matter) with WCF and wsHttpBinding (which is SOAP 1.2):
suds is pretty much dead (can't even pip install it on python 3)
suds-jurko seems kind-of dead. The 0.6 release has a very annoying infinite recursion bug (at least on the WSDL exposed by our service) which is fixed in the tip but that's not released and it's been 1.5years (at time of this writing in Feb'17) since the last commit.
It works on python 3 but doesn't support SOAP 1.2. Sovetnikov's answer is an attempt to get it working with 1.2 but I haven't managed to make it work for me.
zeep seems to be the current way to go and worked out of the box (I'm not affiliated with zeep, it just works for me and I spent several hours banging my head against a brick wall trying to make suds work). For zeep to work, the WCF service host configuration must include <security mode="None"/> under the wsHttpBinding node Actually zeep seems to support username and signature (x509) based WS-SE but I haven't tried that so can't speak to any problems around it.
WCF needs to expose functionality through a communication protocol. I think the most commonly used protocol is probably SOAP over HTTP. Let's assume that's
what you're using then.
Take a look at this chapter in Dive Into Python. It will show you how to
make SOAP calls.
I know of no unified way of calling a WCF service in Python, regardless of communication
protocol.
Just to help someone to access WCF SOAP 1.2 service with WS-Addressing using suds.
Main problem is to inject action name in every message.
This example for python 3 and suds port https://bitbucket.org/jurko/suds.
Example uses custom authentification based on HTTP headers, i leave it as is.
TODO: Automatically get api_direct_url from WSDL (at now it is hard coded).
from suds.plugin import MessagePlugin
from suds.sax.text import Text
from suds.wsse import Security, UsernameToken
from suds.sax.element import Element
from suds.sax.attribute import Attribute
from suds.xsd.sxbasic import Import
api_username = 'some'
api_password = 'none'
class api(object):
api_direct_url = 'some/mex'
api_url = 'some.svc?singleWsdl|Wsdl'
NS_WSA = ('wsa', 'http://www.w3.org/2005/08/addressing')
_client_instance = None
#property
def client(self):
if self._client_instance:
return self._client_instance
from suds.bindings import binding
binding.envns = ('SOAP-ENV', 'http://www.w3.org/2003/05/soap-envelope')
api_inst = self
class _WSAPlugin(MessagePlugin):
def marshalled(self, context):
api_inst._marshalled_message(context)
self._client_instance = Client(self.api_url,
plugins=[_WSAPlugin()],
headers={'Content-Type': 'application/soap+xml',
'login':api_username,
'password': api_password}
)
headers = []
headers.append(Element('To', ns=self.NS_WSA).setText(self.api_direct_url))
headers.append(Element('Action', ns=self.NS_WSA).setText('Blank'))
self._client_instance.set_options(soapheaders=headers)
cache = self._client_instance.options.cache
cache.setduration(days=10)
return self._client_instance
def _marshalled_message(self, context):
def _children(r):
if hasattr(r, 'children'):
for c in r.children:
yield from _children(c)
yield c
for el in _children(context.envelope):
if el.name == 'Action':
el.text = Text(self._current_action)
return
_current_action = None
def _invoke(self, method, *args):
try:
self._current_action = method.method.soap.action.strip('"')
return method(*args)
finally:
self._current_action = None
def GetRequestTypes(self):
return self._invoke(self.client.service.GetRequestTypes)[0]
def GetTemplateByRequestType(self, request_type_id):
js = self._invoke(self.client.service.GetTemplateByRequestType, request_type_id)
return json.loads(js)
def GetRequestStatus(self, request_guid):
return self._invoke(self.client.service.GetRequestStatus, request_guid)
def SendRequest(self, request_type_id, request_json):
r = json.dumps(request_json, ensure_ascii=False)
return self._invoke(self.client.service.SendRequest, request_type_id, r)
I do not know of any direct examples, but if the WCF service is REST enabled you could access it through POX (Plain Old XML) via the REST methods/etc (if the service has any). If you are in control of the service you could expose endpoints via REST as well.
if you need binary serialized communication over tcp then consider implementing solution like Thrift.
Even if there is not a specific example of calling WCF from Python, you should be able to make a fully SOAP compliant service with WCF. Then all you have to do is find some examples of how to call a normal SOAP service from Python.
The simplest thing will be to use the BasicHttpBinding in WCF and then you can support your own sessions by passing a session token with each request and response.