I'm trying to use a put request to update Dynamics CRM. When I run the following python code I get a 204 response, but instead of updating a value it clears it. For example firstname gets updated to null instead of "WebApiUpdate".
import adal
import json
import urllib2
import requests
endpoint = "https://login.microsoftonline.com/00000000-0000-0000-0000-000000000002"
username = "example#example.com"
password = "password"
resource_uri = "https://orgname.crm.dynamics.com"
token_response = adal.acquire_token_with_username_password(endpoint,username,password,resource= resource_uri)
print token_response['accessToken']
payload = { "value": "WebApiUpdate"}
url = "http://xxxxx.crm.dynamics.com/api/data/v8.0/contacts(00000000-0000-0000-0000-000000000001)/firstname"
headers = {"Authorization":"Bearer "+token_response['accessToken']}
r = requests.put(url,data=payload,headers=headers)
print r.content
Solution:
set the url to be https:// instead of http://
set data = payload to json = payload in the request. ex:
r = requests.put(url,json=payload,headers=headers)
What was going wrong:
The initial call, which included the payload (regardless of if it was set to data or json), received a response of HTTP/1.0 301 Moved Permanently. In this response was a header with the new location. This location simply included https:// in the url. Then my code sent out a second put request that went to the correct https url, had the correct header parameters, but did not include the payload. This caused my put request to not include the value to update to, so Microsoft Dynamics CRM set the value to null.
Microsoft Dynamics CRM requires the put request payload to be formatted in json. After I updated the link to be https:// and the payload was included, it was formatted as x-www-form-urlencoded.That means the payload was appended to the request as value=WebApiUpdate. This caused a reply of HTTP/1.1 400 Bad Request. Changing the payload to be formatted as json means the payload will be appended to the request as {"value": "WebApiUpdate"}.
How I solved it:
I set my program to output the http put call it was making by adding the below code that I found from this stack overflow question. I then noticed that there were two non identical put calls and went from there.
import logging
try:
import http.client as http_client
except ImportError:
# Python 2
import httplib as http_client
http_client.HTTPConnection.debuglevel = 1
logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True
I read the documentation on the request method here which I found on this stack overflow question
Related
I need to print all requests from specific Postman collection. I have this code:
import requests
# Set up Postman API endpoint and authorization
postman_api_endpoint = "https://api.getpostman.com/collections"
postman_api_key = "PMAK-63b6bf724ebf902ad13d4bf2-e683c12d426716552861acda**********"
headers = {"X-Api-Key": postman_api_key}
# Get all requests from Postman collection
collection_id = "25184041-c1537769-f598-4c0e-b8ae-8cd185a79c03"
response = requests.get(f"{postman_api_endpoint}/{collection_id}/items", headers)
if response.status_code != 200:
print("Error retrieving collection:", response.text)
else:
# Print all requests
requests_data = response.json()["items"]
for request_data in requests_data:
request_method = request_data["request"]["method"]
request_url = request_data["request"]["url"]
request_headers = request_data["request"]["header"]
request_body = request_data["request"]["body"]["raw"] \
if request_data["request"]["body"]["mode"] == "raw" else ""
print(f"{request_method} {request_url}")
print("Headers:")
for header in request_headers:
print(f"{header['key']}: {header['value']}")
print("Body:")
print(request_body)
I received an error while I try to call response.text and have such massage:
Error retrieving collection: {"error":{"name":"notFound","message":"Requested resource not found"}}
Which means that I have 404 error. I have several assumptions what I did wrong:
I entered incorrect api key(But I checked several times and regenerated it twice)
I entered incorrect collection id, but in the screen below you can see where I took it and it is correct
And as I think the most likely variant I wrote incorrect request where I put my key and my collection id(I din't find any example how such requests should be like)
And of course I have requests in my collection, so error can not be because the collection is empty
Please give me some advice how I can fix this error. Thank you!
The answer is actually is really simple. I didn't know that I need to push button save request in Postman. I think if I create request in collection it will automatically save it. But I didn't, so I just saved all requests manually and finally receive correct response.
I have to send a POST request to the /batch endpoint of : 'https://www.google-analytics.com'.
As mentioned in the Documentation I have to send the request to /batch endpoint and specify each payload on its own line.
I was able to achieve this using POSTMAN as follows:
My query is to make a POST request using Python's requests library
I tried something like this :
import requests
text = '''v=1&cid=43223523&tid=UA-XXXXXX-1&t=event&ec=aggregated_stats&ea=daily_kpi&el=bookmarks&ev=13
v=1&cid=43223523&tid=UA-XXXXXX-1&t=event&ec=aggregated_stats&ea=daily_kpi&el=upvotes&ev=65
v=1&cid=43223523&tid=UA-XXXXXX-1&t=event&ec=aggregated_stats&ea=daily_kpi&el=questions&ev=15
v=1&cid=43223523&tid=UA-XXXXXX-1&t=event&ec=aggregated_stats&ea=daily_kpi&el=postviews&ev=95'''
response = requests.post('https://www.google-analytics.com/batch', data=text)
but it doesn't works.
UPDATE
I Tried this and it works !
import http.client
conn = http.client.HTTPSConnection("www.google-analytics.com")
payload = "v=1&cid=43223523&tid=UA-200248207-1&t=event&ec=aggregated_stats&ea=daily_kpi&el=bookmarks&ev=13\r\nv=1&cid=43223523&tid=UA-200248207-1&t=event&ec=aggregated_stats&ea=daily_kpi&el=upvotes&ev=63\r\nv=1&cid=43223523&tid=UA-200248207-1&t=event&ec=aggregated_stats&ea=daily_kpi&el=questions&ev=11\r\nv=1&cid=43223523&tid=UA-200248207-1&t=event&ec=aggregated_stats&ea=daily_kpi&el=postviews&ev=23"
headers = {
'Content-Type': 'text/plain'
}
conn.request("POST", "/batch", payload, headers)
res = conn.getresponse()
But the question remains open, what's the issue with requests here.
You don't need to double-escape the newline symbol.
Moreover, you don't need the newline symbol at all for the multi-line string.
And also the indentations you put in your multi-line string are counted:
test = '''abc
def
ghi'''
print(test)
Here's an SO answer that explains this with some additional ways to make long stings: https://stackoverflow.com/a/10660443/4570170
Now the request body.
The documentation says
payload_data – The BODY of the post request. The body must include exactly 1 URI encoded payload and must be no longer than 8192 bytes.
So try uri-encoding your payload:
text = '''v=1&cid=43223523&tid=UA-XXXXXX-1&t=event&ec=aggregated_stats&ea=daily_kpi&el=bookmarks&ev=13
v=1&cid=43223523&tid=UA-XXXXXX-1&t=event&ec=aggregated_stats&ea=daily_kpi&el=upvotes&ev=65
v=1&cid=43223523&tid=UA-XXXXXX-1&t=event&ec=aggregated_stats&ea=daily_kpi&el=questions&ev=15
v=1&cid=43223523&tid=UA-XXXXXX-1&t=event&ec=aggregated_stats&ea=daily_kpi&el=postviews&ev=95'''
text_final = requests.utils.quote(text)
response = requests.post('https://www.google-analytics.com/batch', data=text_final)
Finally , I figured out the solution myself.
Updating for others help.
The problem was I was working on AWS Cloud9 and as mentioned in the documentation
Some environments are not able to send hits to Google Analytics directly. Examples of this are older mobile phones that can't run JavaScript or corporate intranets behind a firewall.
So we just need to include the User Agent parameter
ua=Opera/9.80
in each of our payloads
It works !
I'm using Python 3.7 with urllib.
All work fine but it seems not to athomatically redirect when it gets an http redirect request (307).
This is the error i get:
ERROR 2020-06-15 10:25:06,968 HTTP Error 307: Temporary Redirect
I've to handle it with a try-except and manually send another request to the new Location: it works fine but i don't like it.
These is the piece of code i use to perform the request:
req = urllib.request.Request(url)
req.add_header('Authorization', auth)
req.add_header('Content-Type','application/json; charset=utf-8')
req.data=jdati
self.logger.debug(req.headers)
self.logger.info(req.data)
resp = urllib.request.urlopen(req)
url is an https resource and i set an header with some Authhorization info and content-type.
req.data is a JSON
From urllib documentation i've understood that the redirects are authomatically performed by the the library itself, but it doesn't work for me. It always raises an http 307 error and doesn't follow the redirect URL.
I've also tried to use an opener specifiyng the default redirect handler, but with the same result
opener = urllib.request.build_opener(urllib.request.HTTPRedirectHandler)
req = urllib.request.Request(url)
req.add_header('Authorization', auth)
req.add_header('Content-Type','application/json; charset=utf-8')
req.data=jdati
resp = opener.open(req)
What could be the problem?
The reason why the redirect isn't done automatically has been correctly identified by yours truly in the discussion in the comments section. Specifically, RFC 2616, Section 10.3.8 states that:
If the 307 status code is received in response to a request other
than GET or HEAD, the user agent MUST NOT automatically redirect the
request unless it can be confirmed by the user, since this might
change the conditions under which the request was issued.
Back to the question - given that data has been assigned, this automatically results in get_method returning POST (as per how this method was implemented), and since that the request method is POST, and the response code is 307, an HTTPError is raised instead as per the above specification. In the context of Python's urllib, this specific section of the urllib.request module raises the exception.
For an experiment, try the following code:
import urllib.request
import urllib.parse
url = 'http://httpbin.org/status/307'
req = urllib.request.Request(url)
req.data = b'hello' # comment out to not trigger manual redirect handling
try:
resp = urllib.request.urlopen(req)
except urllib.error.HTTPError as e:
if e.status != 307:
raise # not a status code that can be handled here
redirected_url = urllib.parse.urljoin(url, e.headers['Location'])
resp = urllib.request.urlopen(redirected_url)
print('Redirected -> %s' % redirected_url) # the original redirected url
print('Response URL -> %s ' % resp.url) # the final url
Running the code as is may produce the following
Redirected -> http://httpbin.org/redirect/1
Response URL -> http://httpbin.org/get
Note the subsequent redirect to get was done automatically, as the subsequent request was a GET request. Commenting out req.data assignment line will result in the lack of the "Redirected" output line.
Other notable things to note in the exception handling block, e.read() may be done to retrieve the response body produced by the server as part of the HTTP 307 response (since data was posted, there might be a short entity in the response that may be processed?), and that urljoin is needed as the Location header may be a relative URL (or simply has the host missing) to the subsequent resource.
Also, as a matter of interest (and for linkage purposes), this specific question has been asked multiple times before and I am rather surprised that they never got any answers, which follows:
How to handle 307 redirection using urllib2 from http to https
HTTP Error 307: Temporary Redirect in Python3 - INTRANET
HTTP Error 307 - Temporary redirect in python script
I am just trying to access an API through Postman and its wokring fine.The Postman header response is returning some details like below
Authentication-Token →/DwG7gAxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Connection →keep-alive
Content-Length →16
Content-Type →application/json;charset=UTF-8
Date →Tue, 25 Sep 2018 17:44:01 GMT
Server →Apache-Coyote/1.1
But when I am trying to do same in Python I am just receivig the response status.(200)
How can I get the above Authentication-Token etc details like Postman in python code.
import requests
import json
url = 'https://test-orchestrator.lmig.com/baocdp/rest/login/'
headers = {"Content-Type": "application/json"}
data1 = {"username":"abc", "password":"abc"}
print("Testing authentication for Remedy test environment...")
change_response=requests.post(url,data=json.dumps(data1),headers=headers)
print(change_response)
If you print change_response, it will most likely look like this <status [200]> or something to that effect. If you want to see the contents of the response, you can use the vars response.text, response.content, or response.headers (among others) or since this is a json response, you can use the method response.json() to convert the contents of the response into a dictionary full of native Python data types.
I would reccommend x = response.json(), as the contents of your response seem to contain an auth token that you will most likely need to communicate with this device further. You can then use auth+token = x[token_key] to isolate that token.
I am trying to login to a website in order to get some data. I have noticed that there is not form-data in the 'post' method but there is a 'request payload'. Furthermore, when I login in I cannot see anymore the login post method. Here is a screenshot of the network post login method:
When I login the next page showed is I use the following code in order to login:
import requests
urlData = 'https://b*********.dk/Account/Market'
urlLogin = 'https://b**********an.dk/
with requests.Session() as c:
urlLogin = 'https://b*************n.dk/Authorization/
c.get(urlLogin)
NetSession = c.cookies['ASP.NET_SessionId']
login_data = {
'ASP.NET_SessionId': NetSession,
'username':"A******",
'Password':"q******",
'remmemberMe': True
}
lol = c.post(urlLogin, data=login_data)
print(lol.text)
Running this code the following is outputed:
{"Processed":true,"Message":"The user name or password provided is incorrect.","NeedResetPassword":false}
When i input a wrong password the Processed value is false, while with correct credentials is true. But it deosnt login. Any idea why this could happen?
As you've already correctly noticed, the original credentials are not sent using form encoding (meaning &user=alice&password=secret), but are JSON encoded (so rather {"user":"alice", "password": "secret"}). You can also see this in the request's Content-Type header, which is application/json where (as opposed to application/x-www-form-urlencoded otherwhise).
For your custom request to work, you propably also need to send JSON-encoded data. This is documented in length in the official Documentation, so I'll just give the short version:
import json
# Build session and request body just like you already did in your question
# ...
headers = {"Content-Type": "application/json"}
lol = c.post(urlLogin, data=json.dumps(login_data), headers=headers)
print(lol.json())