We Keep Coding

Why does requests.post raise 404 Not found code?

does anyone have any idea, why the output of this script, where i use requests.post to login is code 404, Not found, and the same script, where I use only requests.get has code 200 OK? What should I change?
import requests
URL = 'https://www.stratfor.com/login'
session = requests.Session()
page = session.post(URL)
print(page.status_code, page.reason)
Thank you.

it seem to be worked with get request and should returned 405 but it depends on the server

One good way to note the right page to log in is to log the network calls.
After looking at the calls, a request is sent to
URL = https://www.stratfor.com/api/v3/user/login
The API endpoint actually expects a payload like this:
payload = {username: "YOU_USER", password: "YOUR_PASS"}
Try something like this:
r = requests.post(URL,json=payload)
You might need to pass more headers, which you can poke the network call log for. Although, it seems like that user and password are passed as raw strings here? If so, that's definitely not safe.

How to POST - API in thingspeak.comusing urllib in Python

I am working on a project where I have to get data from a website and then post it to a different website. At the moment I am still new, so I am using a thingspeak.com account to experiment on posting, and I am also following the same example used in the documentations of urllib, https://docs.python.org/2/library/urllib.html.
params = urllib.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0})
# I am using this to check => eggs=2&bacon=0&spam=1
print params
f = urllib.urlopen("https://api.thingspeak.com/update?key=8TU6AC31T81MV52N&field1=",params)
print f.read()
However, when I check my thingspeak website http://api.thingspeak.com/channels/42628/feed.json?key=8TU6AC31T81MV52N, I will get this for that specific entry
{"created_at":"2015-06-17T10:05:12Z","entry_id":19,"field1":""}
Supposedly, or at least how I understood it, params should be posted to the website but I don't understand why I get the empty string. Also, it doesn't work if I loop it, it only works for the first iteration posting the empty string, and the rest are I will get 0 read from f.read(). What's the problem?! :\
All help is much appreciated. Also please correct if by I was mistaken.

You are sending the query parameter named field1 in the URL, but it is set to any empty string. Hence field1 is set to "" by thingspeak.com. Those values you pass in params are sent in the body of the HTTP POST request, they are not somehow appended to the URL.
You can set the field in one of two ways:
a GET request using:
f = urllib.urlopen("https://api.thingspeak.com/update?key=8TU6AC31T81MV52N&field1=my+value")
which includes the API key and a value for field1.
a POST request:
params = urllib.urlencode({'key': '8TU6AC31T81MV52N', 'field1': 'my value'})
f = urllib.urlopen("https://api.thingspeak.com/update", data=params)
thinkspeak will accept query parameters in the URL with a POST request, e.g. you can pass field1 in the URL and the API key in the body:
params = urllib.urlencode({'key': '8TU6AC31T81MV52N'})
f = urllib.urlopen("https://api.thingspeak.com/update?field1=hello", data=params)
However, playing around with the API it looks like it will only accept the query parameter named field1. All other fields are ignored, even those documented such as field2, field3, etc.

unable to run requests for flask on internal server

Running a server on
> http://127.0.0.1:5000/
and trying to do a post request (the actual code is a bit more complicated but this is the part I cannot get working). Basically, trying to get something like the following to work but this returns and error saying its not found.
r = requests.post('http://127.0.0.1:5000/api/user')
I also tried something like the following but it returns:
url = url_for('api.userlistapi')
payload = {'email':email, 'password':password,'profile_verified':False}
r = requests.post(url)
self.prepare_url(url, params)
File "....appp/flask/lib/python2.7/site-packages/requests/models.py", line 360, in prepare_url
"Perhaps you meant http://{0}?".format(url))
MissingSchema: Invalid URL u'/api/user': No schema supplied. Perhaps you meant http:///api/user?
Any help would be appreciated! It might just be something dumb with routing or that a post request should be done differntly? I am also using angular and those requests to the same domain work.

You were on the right tracks with the second solution you tested.
You just have to add _external=True to url_for arguments:
url = url_for('api.userlistapi', _external=True)
payload = {'email':email, 'password':password,'profile_verified':False}
r = requests.post(url)
This way, Flask is able to construct a full url with domain included. Otherwise, url_for only builds a relative url meant to be called from within your domain.
--
Also, as a side note, you can pass your parameters directly with requests the following way:
r = requests.post(url, params=payload)
But it depends on other factors in the rest of your code, based on what you want to do.

Unit testing Django JSON View

I'm trying to write some unit tests for some Django json_view views and I'm having trouble passing the json_string to the view. I posted a related question yesterday about passing a json string to a Django view from the JS, the issue was that in my JS I was just passing the json string where I needed to be passing the string as the attribute of an object, because I was failing to do this the string was being taken as the key for the resulting query dict. I'm having a similar problem again except that this time it is form a Django unit test to the Django View. Here is a simplified version of my code which produces the same result.
class MyTestCase(TestCase):
def setUp(self):
self.u = User.objects.create_user('test','test','test')
self.u.is_active = True
self.u.save()
self.client.login(username='test',password='test')
def test_create_object_from_form(self):
"""Test the creation of the Instance from the form data."""
import json
json_string json.dumps({'resource':{'type':'book','author':'John Doe'}})
print(json_string)
response = self.client.post(reverse('ajax_view'),
{'form':json_string},'json')
self.assetNotContains(response,'error')
and the view looks like this
#json_view
def ajax_view(request):
"""Process the incoming form data."""
if request.method == 'POST':
print(request.POST)
form_data = json.loads(request.POST['form'])
resource_data = form_data['resource']
form = MyUserForm(resource_data)
if form.is_valid():
...
Here is what the two print statements produce when the test is run. The json_string is
{"resource": {"type": "book", "author": "John Doe"}}
and the query dict looks like
<QueryDict: {u'{\'form\': \'{"resource": {"type": "book", "author": "John Doe"}}\'}': [u'']}>
I'm total newbie with JS and ajax, so don't worry about hurting my pride, the answer is probably so close it could jump up and bite me.

Final edit
I originally stated that header HTTP_X_REQUESTED_WITH='XMLHttpRequest' was necessary in the post call but this is currently false while in tests. This header is necessary for the csrf middleware but csrf is disabled in tests. However, I still believe it is a good practice to put in test even if middleware disables csrf since most javascript library already pass this header by default when doing ajax. Also, if another piece of code that is not disabled ever use the is_ajax method, you won't need to debug your unittest for hours to figure out that the header was missing.
The problem is with the content-type because when django gets a value in there that is different than text/html, it doesn't use the default post data handling which is to format your data like in a query: type=book&author=JohnDoe for example.
Then the fixed code is:
response = self.client.post(reverse('ajax_view'),
{'form':json_string},
HTTP_X_REQUESTED_WITH='XMLHttpRequest')
Here's how I'm using it myself:
post_data = {
"jsonrpc" : "2.0", "method": method, "params" : params, "id" : id }
return client.post('/api/json/',
json.dumps(post_data), "text/json",
HTTP_X_REQUESTED_WITH='XMLHttpRequest')
to do some json-rpc. Notice that since I pass a different content-type than the default value, my data is passed as is in the post request.

Thank you to #Eric_Fortin for turning me on to the header, it does not however resolve my issue with the malformed query dictionary using 'client.post'. Once I made the change from POST to GET with the XMLHttpRequest header my query dictionary straitened itself out. Here is the current solution:
response = self.client.get(reverse('ajax_view'),
{'form':json_string},'json',
HTTP_X_REQUESTED_WITH='XMLHttpRequest')
this is only a partial answer since this request is going to change data on the server and should be POST not a GET.
Edit:
Here is the final code in my test that works for passing a JSON string via POST to my view:
response = self.client.post(reverse('ajax_view'),
{'form':json.dumps(json_dict)})
Now printing from the view shows that the query dictionary is well formed.
<QueryDict: {u'form': [u'{"resource": {"status": "reviewed", "name": "Resource Test", "description": "Unit Test"}}']}>
I found the answer while tinkering with one of my co-workers, removing the content_type 'json' fixed the malformed query dictionary. The view that is being tested does not make use of or call the 'HttpRequest.is_ajax()', sending the header XMLHttpRequest' has no impact on my issue, though including the header would constitute good-form since this post is an ajax request.

Google App Engine self.redirect() POST method

In GAE (Python), using the webApp Framework, calling self.redirect('some_url') redirects the user to that URL via the GET method. Is it possible to do a (redirect) via the POST method with some parameters as well?
If possible, how?
Thanks!

This is not possible due to how most clients implement redirection [1]:
However, most existing user agent implementations treat 302 as if it
were a 303 response, performing a GET on the Location field-value regardless
of the original request method.
So you must use a workaround (like simply calling the method post() from the RequestHandler) or forget the idea.
[1] http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.3.2

You can pass parameters. Here is an example:
Let's say you have a main page and you want to POST to '/success'. Usually, you may use this way:
self.redirect('/sucess')
But if you want to pass some parameters from the main page to /success page, like username for example, you can modify the code to this:
self.redirect('/sucess?username=' + username)
In this way, you successfully passed the username value into the URL. In /success page, you can read and store the value by using this:
username = self.request.get('username')
At last, you can make you favorite information onto the /success page by using this simple code:
self.response.out.write('You\'ve succeeded, ' + username + '!')
But, it's of course not a safe way to pass password. I wish it helps.

Looks like there's a similar question asked here: Google App Engine self.redirect post
The answer to that one recommends using the urlfetch.fetch() to do the post.
import urllib
form_fields = {
"first_name": "Albert",
"last_name": "Johnson",
"email_address": "Albert.Johnson#example.com"
}
form_data = urllib.urlencode(form_fields)
headers = {'Content-Type': 'application/x-www-form-urlencoded'}
result = urlfetch.fetch(url=url,
payload=form_data,
method=urlfetch.POST,
headers=headers)