I'm using Pysimplesoap for sending a data to the web-service. I want to know the status of the request code. I'm able to print a traceback using trace=True. Over there it does prints the status code and other response variables but how do I get the to store all the traceback into a variable and then check the status code for the same?
Here's my code :_
client = SoapClient(
location = url,
action = 'http://tempuri.org/IService_1_0/',
namespace = "http://tempuri.org/",
soap_ns='soap', ns = False,trace = True
)
data = {'AWB_Number' : '406438762211111', 'Weight':'0.023' ,'Length':'16.4','Height':'4.5','Width':'9.9'}
response= client.UpdateShipment(
ShipmentNumber = data['AWB_Number'],
Weight = Decimal(data['Weight']),
Length = Decimal(data['Length']),
Height = Decimal(data['Height']),
Width = Decimal(data['Width']) ,
InboundLane = "2",
SequenceNumber = "1",
)
I do get a traceback :-
Content-length: 526
Content-type: text/xml; charset="UTF-8"
<?xml version="1.0" encoding="UTF-8"?><soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soap:Header/>
<soap:Body>
<UpdateShipment xmlns="http://tempuri.org/">
<SequenceNumber>1</SequenceNumber><Weight>0.023</Weight><Height>4.5</Height><Width>9.9</Width><Length>16.4</Length><ShipmentNumber>406438762211111</ShipmentNumber><InboundLane>2</InboundLane></UpdateShipment>
</soap:Body>
</soap:Envelope>
status: 200
content-length: 293
x-powered-by: ASP.NET
server: Microsoft-IIS/7.5
date: Sat, 23 Aug 2014 07:27:38 GMT
content-type: text/xml; charset=utf-8
<s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/"><s:Body xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><UpdateShipmentResponse xmlns="http://tempuri.org/"><UpdateShipmentResult/></UpdateShipmentResponse></s:Body></s:Envelope>
===============================================================================
There's a status code mention there as 200, but I dont get to store this traceback into a variable to get to know the status code of it. A human need to intervene to have a look at the status code. How do my program gets to know the status code?
The SoapClient instance retains a .response attribute, containing information about the response. What that information is depends on the transport picked.
If you have just PySimpleSoap installed, the urllib2 library is picked and the status code is not part of the client.response attribute; the information is not retained from the actual response from urllib2, only the HTTP headers are preserved.
The pycurl transport gives you even less info; client.response is always an empty dictionary then.
Only if you also installed httplib2 will you get anything useful out of this; client.response is then set to a dictionary that includes a status code:
>>> import pysimplesoap
>>> client = pysimplesoap.client.SoapClient(wsdl='http://www.webservicex.net/stockquote.asmx?WSDL')
>>> response = client.GetQuote('GOOG')
>>> client.response
{'status': '200', 'content-length': '991', 'x-aspnet-version': '4.0.30319', 'vary': 'Accept-Encoding', 'server': 'Microsoft-IIS/7.0', '-content-encoding': 'gzip', 'cache-control': 'private, max-age=0', 'date': 'Sat, 23 Aug 2014 08:05:19 GMT', 'x-powered-by': 'ASP.NET', 'content-type': 'text/xml; charset=utf-8'}
>>> client.response['status']
'200'
Note that the value is a string, not an integer.
The httplib2 transport is picked by default if available.
As for the trace option; that sets up a logging module log, calls logging.basicConfig() with a log level and gets on with it. You could add a custom handler to the pysimplesoap.client.log object, but I wouldn't bother, really. If you see the status logged to the trace, then in all likelihood you are already using httplib2 and can access the status code more directly. For urllib2, for example, no status is logged either. That's because it is the client.response.items() values that are being logged here.
From the pysimplesoap source code, at line 261 of client, the send method is written which is used to send the http request from client's end
Taking cue from it,
def send(self, method, xml):
...
response, content = self.http.request(
location, 'POST', body=xml, headers=headers)
self.response = response
self.content = content
...
you should be able to access client.response to get the raw response for the request. From there, try doing a client.response.status, or do a dir(client.response) to find out any supportive method to get status code.
EDIT
As mentioned here in transport module, you can specifiy a library (like urllib2 or httplib2). to make it the default one to be picked up.
Related
When I send a message to a queue I want the response returned into an object which I can then include in my log or not. However for some reason when I execute the following code:
from azure.storage.queue import QueueClient, TextBase64EncodePolicy
# ... some code running ##########################
queue = QueueClient.from_connection_string(conn_str=conn_queue, queue_name="items",
message_encode_policy=TextBase64EncodePolicy())
# ... some message generated #####################
response=queue.send_message(json.dumps(item_dict))
a complete message is printed to my log. It looks for example like this:
Request URL: 'https://{some_storage_account}.queue.core.windows.net/items/messages'
Request method: 'POST'
Request headers: 'Accept': 'application/xml'
'Content-Type': 'application/xml; charset=utf-8'
'x-ms-version': 'REDACTED'
'Content-Length': '1295'
'x-ms-date': 'REDACTED'
'x-ms-client-request-id': '3452464c-06b2-11eb-9f96-00155d6ebdc5'
'User-Agent': 'azsdk-python-storage-queue/12.1.3 Python/3.8.5 (Linux-4.19.104-microsoft-standard-x86_64-with-glibc2.2.5)'
'Authorization': 'REDACTED'
A body is sent with the request
Response status: 201
Response headers:
'Transfer-Encoding': 'chunked'
'Content-Type': 'application/xml'
'Server': 'Windows-Azure-Queue/1.0 Microsoft-HTTPAPI/2.0'
'x-ms-request-id': '1720a5da-c003-002b-18be-9ab4d4000000'
'x-ms-version': 'REDACTED'
'Date': 'Mon, 05 Oct 2020 02:26:41 GMT'
How can I prevent this gulp of information to be printed to my log?
I have spent at least half an hour on the docs from microsoft. But I can't find where I can turn this behaviour off.
logger = logging.getLogger("azure.core.pipeline.policies.http_logging_policy")
logger.setLevel(logging.WARNING)
This Worked For Me - Add this to your code
I tried with the logging-levels - it does not do the trick for me. However when I define a new logger for this connection then it works:
queue_logger = logging.getLogger("logger_name")
# queue_logger.disabled = True
queue_logger.setLevel(logging.DEBUG)
queue = QueueClient.from_connection_string(conn_str=conn_queue, queue_name="debug",message_encode_policy=TextBase64EncodePolicy(), logger=queue_logger)
Ultimately I feel like I should use the REST API instead of the Python Azure SDK for this. REST API allows me to log-output based on the response status. For some weird reason the SDK does not offer me this possibility.
The answer should be here, in the README of azure-storage-python in GitHub:
Here is how we use the logging levels, it is recommended to use INFO:
DEBUG: log strings to sign
INFO: log outgoing requests and responses, as well as retry attempts # <--
WARNING: not used
ERROR: log calls that still failed after all the retries
This should do it, as recommended by Kalies LAMIRI (I didn't try myself):
logger = logging.getLogger("azure.storage")
logger.setLevel(logging.ERROR)
I have an URL pointing to a binary file which I need to download after checking its size, because the download should only be (re-)executed if the local file size differs from the remote file size.
This is how it works with wget (anonymized host names and IPs):
$ wget <URL>
--2020-02-17 11:09:18-- <URL>
Resolving <URL> (<host>)... <IP>
Connecting to <host> (<host>)|<ip>|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 31581872 (30M) [application/x-gzip]
Saving to: ‘[...]’
This also works fine with the --continue flag in order to resume a download, including skipping if the file was completely downloaded earlier.
I can do the same with curl, the content-length is also present:
$ curl -I <url>
HTTP/2 200
date: Mon, 17 Feb 2020 13:11:55 GMT
server: Apache/2.4.25 (Debian)
strict-transport-security: max-age=15768000
last-modified: Fri, 14 Feb 2020 15:42:29 GMT
etag: "[...]"
accept-ranges: bytes
content-length: 31581872
vary: Accept-Encoding
content-type: application/x-gzip
In Python, I try to implement the same logic by checking the Content-length header using the requests library:
with requests.get(url, stream=True) as response:
total_size = int(response.headers.get("Content-length"))
if not response.ok:
logger.error(
f"Error {response.status_code} when downloading file from {url}"
)
elif os.path.exists(file) and os.stat(file).st_size == total_size:
logger.info(f"File '{file}' already exists, skipping download.")
else:
[...] # download file
It turns out that the Content-length header is never present, i.e. gets a None value here. I know that this should be worked around by passing a default value to the get() call, but for the purpose of debugging, this example consequently triggers an exception:
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'
I can confirm manually that the Content-length header is not there:
requests.get(url, stream=True).headers
{'Date': '[...]', 'Server': '[...]', 'Strict-Transport-Security': '[...]', 'Upgrade': '[...]', 'Connection': 'Upgrade, Keep-Alive', 'Last-Modified': '[...]', 'ETag': ''[...]'', 'Accept-Ranges': 'bytes', 'Vary': 'Accept-Encoding', 'Content-Encoding': 'gzip', 'Keep-Alive': 'timeout=15, max=100', 'Transfer-Encoding': 'chunked', 'Content-Type': 'application/x-gzip'}
This logic works fine though for other URLs, i.e. I do get the Content-length header.
When using requests.head(url) (omitting the stream=True), I get the same headers except for Transfer-Encoding.
I understand that a server does not have to send a Content-length header.
However, wget and curl clearly do get that header. What do they do differently from my Python implementation?
Not really an answer to the question about the missing Content-length header, but a solution to the underlying problem:
Instead of checking the local file size vs the content length of the remote, I have ended up checking the Last-modified header and compare that to the mtime of the local file. This is also safer in (the unlikely) case that the remote file is updated, but still has the exact same size.
Hi I am using zeep to consume a soap based web service,
and I keep on getting HTTP status 415 error. I dig down a bit and use
Pycharm Debuggger and found that the reason was:
'Cannot process the message because the content type \'text/xml;
charset=utf-8 XaSOfalw: rtt; ___utmvmBfuwVEwB=yEnqIuCmRhw\' was
not the expected type \'text/xml; charset=utf-8\'.'
What is wrong with the content type? and how do I change it in Zeep?
I just created a simple test code which looks like this:
from zeep import Client
pretend_wsdl = 'https://pretendwsdl'
client = Client(wsdl=pretend_wsdl)
res = client.service.NameOfService()
print(res)
and get this error:
zeep.exceptions.TransportError: Server returned HTTP status 415 (no
content available)
I have solved the problem by using plugins in zeep client.
My code looks like this:
from zeep import Client
from zeep import Plugin
class MyLoggingPlugin(Plugin):
def ingress(self, envelope, http_headers, operation):
return envelope, http_headers
def egress(self, envelope, http_headers, operation, binding_options):
http_headers['Content-Type'] = 'text/xml; charset=utf-8;'
return envelope, http_headers
pretend_wsdl = 'https://pretendwsdl.com'
client = Client(wsdl=pretend_wsdl, plugins=[MyLoggingPlugin()])
res = client.service.NameOfService()
print(res)
I find it weird because the default content type of zeep is text/xml; charset=utf-8;
and the wsdl I'm using doesn't think that the content type from zeep is text/xml; charset=utf-8;
So I used zeep plugins to explicitly set the content type to text/xml; charset=utf-8; and it surprisingly works.
Problem
When I switched Macbooks, all of the sudden I am getting an HTTP 411: Length Required (I wasn't getting this using a different Mac) trying to use a POST request with httplib. I cannot seem to find a work around for this.
Code Portion 1: from a supporting class; retrieves data and other things,
class Data(object):
def __init__(self, value):
self.company_id = None
self.host = settings.CONSUMER_URL
self.body = None
self.headers = {"clienttype": "Cloud-web", "Content-Type": "application/json", "ErrorLogging": value}
def login(self):
'''Login and store auth token'''
path = "/Security/Login"
body = self.get_login_info()
status_code, resp = self.submit_request("POST", path, json.dumps(body))
self.info = json.loads(resp)
company_id = self.get_company_id(self.info)
self.set_token(self.info["token"])
return company_id
def submit_request(self, method, path, body=None, header=None):
'''Submit requests for API tests'''
conn = httplib.HTTPSConnection(self.host)
conn.set_debuglevel(1)
conn.request(method, path, body, self.headers)
resp = conn.getresponse()
return resp.status, resp.read()
Code Portion 2: my unittests,
# logging in
cls.api = data.Data(False) # initializing the data class from Code Portion 1
cls.company_id = cls.api.login()
...
# POST Client/Register
def test_client_null_body(self):
'''Null body object - 501'''
status, resp = self.api.submit_request('POST', '/Client/register')
if status != 500:
log.log_warning('POST /Client/register: %s, %s' % (str(status), str(resp)))
self.assertEqual(status, 500)
Code Portion 3: example of the data I send from a settings file,
API_ACCOUNT = {
"userName": "account#account.com",
"password": "password",
"companyId": 107
}
From Logging
WARNING:root: POST /Client/register: 411, <!DOCTYPE HTML PUBLIC "-//W3C//DTD
HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Length Required</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Length Required</h2>
<hr><p>HTTP Error 411. The request must be chunked or have a content length.</p>
</BODY></HTML>
Additional Info: I was using a 2008 Macbook Pro without issue. Switched to a 2013 Macbook Pro and this keeps occurring.
I took a look at this post:
Python httplib and POST and it seems that at the time httplib did not automatically generate the content length.
Now https://docs.python.org/2/library/httplib.html:
If one is not provided in headers, a Content-Length header is added automatically for all methods if the length of the body can be determined, either from the length of the str representation, or from the reported size of the file on disk.
when using conn.set_debuglevel(1) we see that httplib is sending a header
reply: 'HTTP/1.1 411 Length Required\r\n'
header: Content-Type: text/html; charset=us-ascii
header: Server: Microsoft-HTTPAPI/2.0
header: Date: Thu, 26 May 2016 17:08:46 GMT
header: Connection: close
header: Content-Length: 344
Edit
Unittest Failure:
======================================================================
FAIL: test_client_null_body (__main__.NegApi)
Null body object - 501
----------------------------------------------------------------------
Traceback (most recent call last):
File "API_neg.py", line 52, in test_client_null_body
self.assertEqual(status, 500)
AssertionError: 411 != 500
.send: 'POST /Client/register HTTP/1.1\r\nHost: my.host\r\nAccept-Encoding: identity\r\nAuthorizationToken: uhkGGpJ4aQxm8BKOCH5dt3bMcwsHGCHs1p+OJvtf9mHKa/8pTEnKyYeJr+boBr8oUuvWvZLr1Fd+Og2xJP3xVw==\r\nErrorLogging: False\r\nContent-Type: application/json\r\nclienttype: Cloud-web\r\n\r\n'
reply: 'HTTP/1.1 411 Length Required\r\n'
header: Content-Type: text/html; charset=us-ascii
header: Server: Microsoft-HTTPAPI/2.0
header: Date: Thu, 26 May 2016 17:08:27 GMT
header: Connection: close
header: Content-Length: 344
Any ideas as to why this was working on a previous Mac and is currently not working here? It's the same code, same operating systems. Let me know if I can provide any more information.
Edit 2
The issue seemed to be with OSX 10.10.4, after upgrading to 10.10.5 all is well. I still would like to get some insight on why I was having this issue.
The only change from 10.10.4 to 10.10.5, that seems close, would have been the python update from 2.7.6 to 2.7.10 which includes this bug fix: http://bugs.python.org/issue22417
I'm sure I'm doing something simple wrong, but I can't for the life of me figure out how to set the "IsRead" property to true. It's the last step of my process that gets a filtered list of messagesa and stores and processes any attachments.
According to the docs "IsRead" is writable: http://msdn.microsoft.com/office%5Coffice365%5CAPi/complex-types-for-mail-contacts-calendar#ResourcesMessage
http://msdn.microsoft.com/office%5Coffice365%5CAPi/mail-rest-operations#MessageoperationsUpdatemessages
I'm using python 2.7 and the requests module:
# once file acquired mark the email as read
params = {'IsRead':'True'}
base_email_url = u'https://outlook.office365.com/api/v1.0/me/messages/{0}'.format( msgId )
response = requests.patch(base_email_url, params, auth=(email,pwd))
log.debug( response )
The response that comes back is this:
{"error":{"code":"ErrorInvalidRequest","message":"Cannot read the request body."}}
What's the problem with my request?
At first glance it looks OK. I wonder if the Content-Type header isn't being set to "application/json" or something along those lines. Try getting a network trace and verify that the request looks something like:
PATCH https://outlook.office365.com/api/v1.0/Me/Messages('msgid') HTTP/1.1
Accept: application/json;odata.metadata=full
Authorization: Bearer <token>
Content-Type: application/json;odata.metadata=full
Host: outlook.office365.com
Content-Length: 24
Expect: 100-continue
Connection: Keep-Alive
{
"IsRead": "true"
}
Well I have an answer for myself and it is indeed a simple matter.
It was a mistake to not fully read how PATCH is different from GET or POST.
In short it's important to make sure your headers are set for the right content-type.
Here is the working code:
# once file acquired mark the email as read
changes = {u'IsRead':u'True'}
headers = {'Content-Type': 'application/json'}
json_changes = json.dumps(changes)
base_email_url = u'https://outlook.office365.com/api/v1.0/me/messages/{0}'.format( msgId )
response = requests.patch(base_email_url, data=json_changes, auth=__AUTH, headers=headers)
log.debug( response )