I am working on a project where I have to get data from a website and then post it to a different website. At the moment I am still new, so I am using a thingspeak.com account to experiment on posting, and I am also following the same example used in the documentations of urllib, https://docs.python.org/2/library/urllib.html.
params = urllib.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0})
# I am using this to check => eggs=2&bacon=0&spam=1
print params
f = urllib.urlopen("https://api.thingspeak.com/update?key=8TU6AC31T81MV52N&field1=",params)
print f.read()
However, when I check my thingspeak website http://api.thingspeak.com/channels/42628/feed.json?key=8TU6AC31T81MV52N, I will get this for that specific entry
{"created_at":"2015-06-17T10:05:12Z","entry_id":19,"field1":""}
Supposedly, or at least how I understood it, params should be posted to the website but I don't understand why I get the empty string. Also, it doesn't work if I loop it, it only works for the first iteration posting the empty string, and the rest are I will get 0 read from f.read(). What's the problem?! :\
All help is much appreciated. Also please correct if by I was mistaken.
You are sending the query parameter named field1 in the URL, but it is set to any empty string. Hence field1 is set to "" by thingspeak.com. Those values you pass in params are sent in the body of the HTTP POST request, they are not somehow appended to the URL.
You can set the field in one of two ways:
a GET request using:
f = urllib.urlopen("https://api.thingspeak.com/update?key=8TU6AC31T81MV52N&field1=my+value")
which includes the API key and a value for field1.
a POST request:
params = urllib.urlencode({'key': '8TU6AC31T81MV52N', 'field1': 'my value'})
f = urllib.urlopen("https://api.thingspeak.com/update", data=params)
thinkspeak will accept query parameters in the URL with a POST request, e.g. you can pass field1 in the URL and the API key in the body:
params = urllib.urlencode({'key': '8TU6AC31T81MV52N'})
f = urllib.urlopen("https://api.thingspeak.com/update?field1=hello", data=params)
However, playing around with the API it looks like it will only accept the query parameter named field1. All other fields are ignored, even those documented such as field2, field3, etc.
Related
I am making my first API; any advice to improve my process is much appreciated.
I plan on passing JSON-like strings into the HTML request to this FastAPI microservice down there
#app.get("/create/{value}")
def createJSON(value:str):
person_json = value.strip()
fileName = person_json['Value']['0'] + person_json['Value']['1']
with open('%s.JSON','w') as writeFile:
writeFile.write(string)
return "Person has been created"
My HTTP request would look like this:
http://127.0.0.1:8000/create/{"Key":{"0":"name","1":"grad_year","2":"major","3":"quarter","4":"pronoun","5":"hobbies","6":"fun_fact","7":"food","8":"clubs","9":"res"},"Value":{"0":"adfasdfa","1":"adf'asd","2":"asd","3":"fads","4":"fa","5":"sdfa","6":"df","7":"asd","8":"fa","9":"df"}}
However, when doing this. The values passed are strings. Thus rendering the fileName portion of the code useless. How can I convert it to a Python dict? I have tried to use .strip(), but it did not help.
You're on the wrong track, Such a request should be essentially modeled as POST or a PUT request. That would allow you to send JSON in the body of the request and obtain it as a dict in python. You can see here
And even if you want to pass data in a GET request, there are query params
Coming back to the original doubt, you would have to use json.loads() to parse the json data and load it in a python dict then you can dump whatever file you like after that.
I'd recommend using the requests library
import requests
url = 'http://127.0.0.1:8000/create/'
params = dict(
name = 'Josh',
grad_year = '1987',
major = 'computer science',
quarter = '3'
)
resp = requests.get(url=url, params=params)
data = resp.json()
Then see here how to handle the JSON Response Content:
https://requests.readthedocs.io/en/master/user/quickstart/#json-response-content
The dict in the code I posted is different than the JSON you're trying to send through though. I assume you have a specific reason for having a "Key" array with the names than a "Value" array for the values of those specific names. But if not I'd recommend using a dictionary instead that way you can do things like:
fileName = person_json['name'] + person_json['grad-year']
I have an Http endpoint exposed as http://localhost:8080/test/api/v1/qc/{id} for delete, while making this API delete call I have to replace with the proper id
I tried below way using the requests module of python
param = {
"id" : 1
}
requests.delete(url = http://localhost:8080/test/api/v1/qc/{id}, params=param)
This API call is breaking with the error
ValueError: No JSON object could be decoded.
How can I do this?
Your code can't run as-is. You need to quote your url string:
url = "http://localhost:8080/test/api/v1/qc/{id}"
Reading the docs for requests, the params only sends the dictionary param as the query string, so it'll only tack on ?id=1 to the end of the URL.
What you want is the {id} to get the value from the dictionary. You can look at this answer for various ways: How do I format a string using a dictionary in python-3.x?
You want something like
requests.delete(url = "http://localhost:8080/test/api/v1/qc/{id}".format(**param))
I'm trying to write API client for Jira with Python requests lib according reference:
https://developer.atlassian.com/server/jira/platform/jira-rest-api-examples/
Request to be generated:
http://localhost:8080/rest/api/2/search?jql=assignee=charlie&startAt=2&maxResults=2
As I know, parameters to GET request should be passed as dictionary like:
params = {'assignee':'charlie', 'startAt':'2'}
But all main parameters are nested in jql parameter, so I assume there is should be a nested dict like:
params = {'jql': {'assignee': 'charlie'}}
But that's doesn't work - as a result I've got request to
/rest/api/2/search?jql=assignee
As expect /rest/api/2/search?jql=assignee=charlie
using
r = requests.get(url, params=params)
How to manage such request?
UPD:
To be more clear, I'd like to wrap request in a method with kwargs, like:
search_query(assignee='charlie', startAt=1, etc...)
And then generate a query using this params, but maybe there are any other ideas.
You are missing couple of key parameters, mainly if you are pushing data via requests, the data go into the data argument. Also the moment you push JSON data, you need to set the headers correctly as well. The last thing is authentication. Have you tried to post it in this manner?
import json
requests.post(url=url, headers={"Content-Type": "application/json"},
auth=('username', 'password'), # your username and password
data=json.dumps(params)
)
Also by the JIRA documentation you've provided (https://developer.atlassian.com/server/jira/platform/jira-rest-api-examples/) if you want to push query as data, the url you want is /rest/api/2/search.
Before downvoting/marking as duplicate, please note:
I have already tried out this, this, this, this,this, this - basically almost all the methods I could find pointed out by the Requests documentation but do not seem to find any solution.
Problem:
I want to make a POST request with a set of headers and form data.
There are no files to be uploaded. As per the request body in Postman, we set the parameters by selecting 'form-data' under the 'Body' section for the request.
Here is the code I have:
headers = {'authorization': token_string,
'content-type':'multipart/form-data; boundary=----WebKitFormBoundaryxxxxxXXXXX12345'} # I get 'unsupported application/x-www-form-url-encoded' error if I remove this line
body = {
'foo1':'bar1',
'foo2':'bar2',
#... and other form data, NO FILE UPLOADED
}
#I have also tried the below approach
payload = dict()
payload['foo1']='bar1'
payload['foo2']='bar2'
page = ''
page = requests.post(url, proxies=proxies, headers=headers,
json=body, files=json.dump(body)) # also tried data=body,data=payload,files={} when giving data values
Error
{"errorCode":404,"message":"Required String parameter 'foo1' is not
present"}
EDIT:
Adding a trace of the network console. I am defining it in the same way in the payload as mentioned on the request payload.
There isn't any gui at all? You could get the network data from chrome, although:
Try this:
headers = {'authorization': token_string}
Probably there is more authorization? Or smthng else?
You shouldn't add Content-Type as requests will handle it for you.
Important, you could see the content type as WebKitFormBoundary, so for the payload you must take, the data from the "name" variable.
Example:
(I know you won't upload any file, it just an example) -
So in this case, for my payload would look like this: payload = {'photo':'myphoto'} (yea there would be an open file etc etc, but I try to keep it simple)
So your payload would be this-> (So always use name from the WebKit)
payload = {'foo1':'foo1data',
'foo2':'foo2data'}
session.post(url,data = payload, proxies etc...)
Important! As I can see you use the method from requests library. Firstly you always should create a session like this
session = requests.session() -> it will handle cookies, headers, etc, and won't open a new session, or plain requests with every requests.get/post.
I've read many posts on the Internet, but none of them seems to work for me.
I have a web form which requires an username and a password in order to log in. Here's a snippet of code among those I tried so far
params = urllib.parse.urlencode({'login' : 'user', 'password' : 'pwd'})
f = urllib.request.urlopen("https://example.com/login.php?%s" % params)
print(f.read().decode('utf-8'))
However it gives me back the login page source instead of the "user panel" that appears after log in. If I write https:/example.com/login.php?login=user&password=pwd in the url bar of the browser, it logs me in. The goal is to get the page source that appears after logging in (the "user panel").
Can anyone help me please?
Using Python 3.5
The correct usage is
f = urllib.request.urlopen("https://example.com/login.php", data=params)
From urllib.request documentation:
data may be a string specifying additional data to send to the server, or None if no such data is needed.
Note that this will send a POST request instead of a GET. If you have to send a GET request, you can simply do the following:
f = urllib.request.urlopen("https://example.com/login.php?user={0}&password={1}".format('user', 'pwd'))
According to the documentation for urllib, you need to pass keyword argument data (or the second positional argument) to urlopen, instead of adding them to the url:
import urllib
params = urllib.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0})
f = urllib.urlopen("http://www.musi-cal.com/cgi-bin/query", params)
print f.read()