I have the following Serializer to handle a user:
class FriendSerializer(serializers.Serializer):
login = serializers.CharField(max_length=15, required=True)
password = serializers.CharField(max_length=15, required=True)
mail = serializers.CharField(max_length=50, required=True)
Currently, my view which processes the POST request to register a new user is, based on the Django REST tutorial:
#api_view(['POST'])
def register_new_user(request):
if request.method == 'POST':
print('POST request !')
stream = BytesIO(request.body)
data = JSONParser().parse(stream)
print(data)
serializer = FriendSerializer(data=data)
print(serializer.is_valid())
else:
print('Not a POST request!')
return HttpResponse('Nothing')
Thus, to simulate a client with a POST request, I use the following lines:
import requests
import json
json_data = json.dumps({'login': 'mylogin', 'password': 'mypassword', 'mail': 'mymail'})
r = requests.post('http://127.0.0.1:8000/register_new_user', json=json_data)
However, although the print(data) retrieves, as expected,
{"login": "mylogin", "mail": "mymail", "password": "mypassword"}
The serializer.is_valid() always returns False.
Am I missing any processing of my request?
EDIT:
I got the following info with serializer.errors:
{'non_field_errors': ['Invalid data. Expected a dictionary, but got str.']}
You dont have to convert a dictionary to string when using requests library's json option. This should work:
import requests
data = {'login': 'mylogin', 'password': 'mypassword', 'mail': 'mymail'}
r = requests.post('http://127.0.0.1:8000/register_new_user', json=data)
The solution is to use ast.literal_eval(request.data) to convert the string to a dictionary. I use request.data instead of manually parsing request.body.
However, v1k45's answer is the best solution, as I do not need to convert my dict to a string before sending my request.
Related
The situation:
I am using React in the front-end and a Flask api server. I am wanting to send the data from React to the api and once I have done this I would like to use WTForms to run validations on the data before handling it. The question may seem similar to CSRF Protection with Flask/WTForms and React , but this does not answer the question, please take a look through I have put a lot of effort in writing a good question.
What I have
Currently the data is being sent successfully as a json object, where the keys match the names within the wtform structure, the aim is to get wtforms to take that json data and insert it into the object and and handle from there as normal
The JSON object being sent
{'username': 'tster', 'fullname': 'Tester test', 'phone': '038287827216', 'email': 'test#example.com', 'password': 'Tester1010', 'repeatPassword': 'Tester1010'}
The entire concept is to sent this from the React component and add it to wtforms in order to validate accordingly and work with.
Python code:
I have tried multple different approaches, this being the latest:
#bp.route('/api/register_user', methods=['POST', 'GET'])
def register():
""" End-point to register users """
if request.method == 'GET':
return ('', {'csrf_token': generate_csrf()})
elif request.method == 'POST':
req = request.get_json(force=True)
if validate_csrf(request.headers['X-Csrftoken']):
print('validated')
return{"message": "posted"}
The concept was that if I couldn't get it to work with wtforms I could generate the csrf_token manually, validate it and once thats validated run manual validations against the data instead of relying on wtforms, however it doesn't seem to validate accordingly. This taken from https://flask-wtf.readthedocs.io/en/stable/api.html#flask_wtf.csrf.generate_csrf
Prior to this I tried:
#bp.route('/api/register_user', methods=['POST', 'GET'])
def register():
""" End-point to register users """
form = RegistrationForm()
print(request.headers)
if request.method == 'GET':
print(form.csrf_token._value())
return ('', {'csrf_token': form.csrf_token._value()})
elif request.method == 'POST':
req = request.get_json(force=True)
form = RegistrationForm.from_json(req)
print(form.username)
return{"message": "posted"}
else:
return { 'errors': form.errors }
The idea behind this was to send the csrf_token as with other cases, due to the fact that in the prior link I read:
Generate a CSRF token. The token is cached for a request, so multiple calls to this function will generate the same token.
time_limit -- Number of seconds that the token is valid. Default is WTF_CSRF_TIME_LIMIT or 3600 seconds (60 minutes).
After reading this I logically thought that this would make sense then that I could essentially get the csrf_token from a standard form object, as you would normally operate with them and then when the request method is POST I could rerender the form and insert the data from the JSON within the request, I got the idea from https://wtforms-json.readthedocs.io/en/latest/
This however is met with the same behavior of not being able to access the data in the form object receiving multiple errors from:
print(form.username.data)
AttributeError: 'tuple' object has no attribute 'data'
print(form.username)(<UnboundField(StringField, ('username',), {'validators': [<wtforms.validators.DataRequired object at 0x000002A3B600A6D0>]})>,)
Thus I couldn't even access the data I was inserting into the form object. (If it even inserted as I assummed it would.)
Prior to this:
I tried the standard way of operating with wtforms:
#bp.route('/api/register_user', methods=['POST', 'GET'])
def register():
""" End-point to register users """
form = RegistrationForm()
print(request.headers)
if request.method == 'GET':
print(form.csrf_token._value())
return ('', {'csrf_token': form.csrf_token._value()})
elif form.validate_on_submit():
req = request.get_json(force=True)
return{"message": "posted"}
else:
return { 'errors': form.errors }
Now it never got past validation, which is strange because the validators are simply DataRequired().
However it did get past the form.is_submitted() when tested as such, but I got the same errors when printing:
print(form.username.data)
AttributeError: 'tuple' object has no attribute 'data'
print(form.username)(<UnboundField(StringField, ('username',), {'validators': [<wtforms.validators.DataRequired object at 0x000002A3B600A6D0>]})>,)
Also with this the react code had to change slightly in order to work, or else it send back bad request the request had to be:
handleSubmit = (details) => {
const finalSend = {
'csrf_token': this.state.csrf_token,
'username': this.state.usernameInput,
'fullname': this.state.fullnameInput,
'phone': this.state.phoneInput,
'email': this.state.emailInput,
'password': this.state.passwordInput,
'repeatPassword': this.state.repeatPasswordInput
}
axios({
method: 'post',
url: '/api/register_user',
data: finalSend,
headers: {
'content-type': 'application/json'
}
})
.then(res => res.json()).catch(e => console.log(e));
}
The Question
So after all this what I am basically asking is:
How can I get wtforms to take the data sent as a JSON object?
Would it be better to operate separately from WTForms, if so how?
If I do the above how can I set up CSRF security, as I tried recently, and it isn't working as needed.
Extra Needed:
The JSON that is sent by the component is in the form of:
{'csrf_token': 'ImQ5MjhlY2VlYzM5Zjg0NmY4ZTg0NDk5ZjNlMjlkNzVlZGM4OGZhY2Ui.YBPbuw.D6BW8XpwEyXySCaBBeS0jIKYabU', 'username': 'charlie', 'fullname': 'charlie char', 'phone': '344444444', 'email': 'cindy#example.com', 'password': 'charlieandcindy', 'repeatPassword': 'charlieandcindy'}
The WTForm that I am using:
class RegistrationForm(FlaskForm):
username = StringField('username', validators=[DataRequired()]),
fullname = StringField('fullname', validators=[DataRequired()]),
email = BooleanField('email', validators=[DataRequired(), Email()]),
phone = StringField('phone', validators=[DataRequired()]),
password = StringField('password', validators=[DataRequired()]),
repeatPassword = StringField('repeatPassword', validators=[DataRequired(), EqualTo('password')])
def validate_username(self, username):
print("validating username")
I have also consulted the following websites looking for a solution:
Flask wtforms - 'UnboundField' object is not callable, dynamic field won't init properly
https://wtforms.readthedocs.io/en/2.3.x/forms/
https://wtforms-json.readthedocs.io/en/latest/
Flask-WTF set time limit on CSRF token
https://flask-wtf.readthedocs.io/en/stable/api.html#flask_wtf.csrf.generate_csrf
How do I get the information from a meta tag with JavaScript?
https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch#uploading_json_data
https://flask-wtf.readthedocs.io/en/stable/csrf.html#javascript-requests
CSRF Protection with Flask/WTForms and React
Once more I have tried my best finding the solution and put a lot of effort into writing a good question, if you need any more just ask.
EDIT
Working on this some more I have played around with wtforms_json some more and have got it to poulate slightly, however it only populates the repeatPassword field, which I am unsure why.
In order to do this I have had to turn off csrf validation and using the following code in the route:
#bp.route('/api/register_user', methods=['POST'])
def register():
""" End-point to register users """
json = request.get_json()
form = RegistrationForm.from_json(json)
print(request.get_json())
print(form.data)
if form.validate():
print("validated")
return{"message": "posted"}
else:
return { 'errors': form.errors }
The result when printing form.data is coming with the repeatPassword being set but nothing else and not sure why... source to this advancement is here WTForms-JSON not working with FormFields
I actually tried out your solution without realizing that you had used the wtforms_json extension. I wanted to see if I could find a solution without having to add an extra flask extension.
My version:
from werkzeug.datastructures import ImmutableMultiDict
#bp.route('/api/register_user', methods=['GET','POST'])
def register():
""" End-point to register users """
reg_json = request.get_json()
form_input = ImmutableMultiDict(reg_json)
form = RegistrationForm(form_input)
if form.validate():
user = Users(
username = form.username.data,
name = form.fullname.data,
phone = form.phone.data,
email = form.email.data,
password = guard.hash_password(form.password.data),
roles = 'User'
)
user.save()
return{"message": "inserted successfully"}
else:
return { 'errors': form.errors }
Breaking it down:
I found some examples of using flask-login where clients send form data as a form rather than JSON data. The code looked something like this:
form = RegistrationForm(request.form)
I continued to dig into the source code and realized that the data structure for the request.form input (into the RegistrationForm constructor) is ImmutableMultiDict.
https://pythonise.com/series/learning-flask/the-flask-request-object (Scroll down to query strings)
ImmutableMultiDict can be found within werkzeug extension that is conveniently installed as a result of installing flask. So basically, I created a ImmutableMultiDict object by passing data from the request:
reg_json = request.get_json()
...
form_input = ImmutableMultiDict(reg_json)
And used that as the input to RegistrationForm:
form = RegistrationForm(form_input)
Now you can receive JSON data from different clients such as a frontend developed with React via POST requests. You also don't have to worry about keeping up-to-date with outdated extensions.
Hope this helps!
Cheers.
I found the answer too this.
In order to do this I ended up using the wtforms_json from json methodas below:
#bp.route('/api/register_user', methods=['GET','POST'])
def register():
""" End-point to register users """
reg_json = request.get_json()
form = RegistrationForm.from_json(reg_json)
if form.validate():
user = Users(
username = form.username.data,
name = form.fullname.data,
phone = form.phone.data,
email = form.email.data,
password = guard.hash_password(form.password.data),
roles = 'User'
)
user.save()
return{"message": "inserted successfully"}
else:
return { 'errors': form.errors }
As well as had to adjust the forms:
class RegistrationForm(FlaskForm):
username = StringField('username', validators=[DataRequired()])
fullname = StringField('fullname', validators=[DataRequired()])
email = StringField('email', validators=[DataRequired()])
phone = StringField('phone', validators=[DataRequired()])
password = StringField('password', validators=[DataRequired()])
repeatPassword = StringField('repeatPassword', validators=[DataRequired(), EqualTo('password')])
def validate_username(self, username):
""" Ensure email isn't in use """
user = Users.objects(username=username.data).first()
if user != None:
raise ValidationError('Please use a different username.')
The only downside to this is I had to turn off the csrf_token within the config with:
WTF_CSRF_ENABLED = False
I'd like to build a request for testing middleware, but I don't want POST requests to always assume I'm sending form data. Is there a way to set request.body on a request generated from django.test.RequestFactory?
I.e., I'd like to do something like:
from django.test import RequestFactory
import json
factory = RequestFactory(content_type='application/json')
data = {'message':'A test message'}
body = json.dumps(data)
request = factory.post('/a/test/path/', body)
# And have request.body be the encoded version of `body`
The code above will fail the test because my middleware needs the data to be passed as the document in request.body not as form data in request.POST. However, RequestFactory always sends the data as form data.
I can do this with django.test.Client:
from django.test import Client
import json
client = Client()
data = {'message':'A test message'}
body = json.dumps(data)
response = client.post('/a/test/path/', body, content_type='application/json')
I'd like to do the same thing with django.test.RequestFactory.
RequestFactory has built-in support for JSON payloads. You don't need to dump your data first. But you should be passing the content-type to post, not to the instantiation.
factory = RequestFactory()
data = {'message':'A test message'}
request = factory.post('/a/test/path/', data, content_type='application/json')
I've tried Jay's solution and didn't work, but after some reseach, this did (Django 2.1.2)
factory = RequestFactory()
request = factory.post('/post/url/')
request.data = {'id': 1}
Here's what worked for me in Django 4.1:
from django.contrib.sessions.middleware import SessionMiddleware
from django.test import TestCase, RequestFactory
from customauth import views
class RegistrationViewTest(TestCase):
def setUp(self):
self.factory = RequestFactory()
def test_post_request_creates_new_user(self):
data = {
'email': 'new_user#email.com',
'screen_name': 'new_user',
'password1': 'new_user_password',
'password2': 'new_user_password',
}
request = self.factory.post('/any/path/will/do/', data )
middleware = SessionMiddleware(request)
middleware.process_request(request)
request.session.save()
response = views.registration_view(request)
self.assertEqual(response.status_code, 302)
# ok
This test passes. The form was successfully processed in views.registration_view.
Note:
When I included content_type='application/json' in the call to self.factory.post (as the accepted answer suggests), request.POST had no content in the view. Without that, it worked. I don't know why but would be happy to learn.
I needed to manually added SessionMiddleware to request.
In later version of Django (tested on 4.0) this is no longer an issue. On the other hand, to pass data to request.POST might be.
In default, when passing content-type to a RequestFactory, data goes into request.body and when you don't, data goes into request.POST.
request_factory = RequestFactory()
# provide content-type
request = request_factory.post(f'url', data={'foo': 'bar'}, content_type="application/json")
print(request.body) # b'{"foo": "bar"}'
# don't provide content type
request = request_factory.post(f'url', data={'foo': 'bar'})
print(request.POST) # <QueryDict: {'foo': ['bar']}>
I'm having problems to do test on Django. I've been reading the documentation of the responses and I can't do the same as they explain on the documentation.
When I get the response, I only have access to response.status_code and can't access to context or redirect_chain when I write response.(and now PyCharm shows all available options).
I've checked on settings.py and I've 'BACKEND': 'django.template.backends.django.DjangoTemplates' to be sure that I'm using Django templates so I don't know why don't work the test. I need configure something?
The code of the test I'm trying to do it's:
from django.test import TestCase
from django.test.client import Client
class Test(TestCase):
def testLogin(self):
client = Client()
headers = {'X-OpenAM-Username': 'user', 'X-OpenAM-Password': 'password', 'Content-Type': 'application/json'}
data = {}
response = self.client.post('/login/', headers=headers, data=data, secure=True, follow=True)
assert (response.status_code == 200)
# self.assertRedirects(response, '/menu/', status_code=301, target_status_code=200)
I'm not using Django authentication, the login form sends the data to an IDP and if the IDP sends with a correct answer, the "login" it's successful:
def login(request):
logout(request)
message = None
if request.method == "POST":
form = LoginForm(request.POST)
if form.is_valid():
username = request.POST['username']
password = request.POST['password']
headers = {'X-OpenAM-Username': username, 'X-OpenAM-Password': password, 'Content-Type': 'application/json'}
data = {}
req = requests.post('http://openam.idp.com:8090/openamIDP/json/authenticate', headers=headers, params=data)
if req.status_code == 200:
respJson = json.loads(req.content)
tokenIdJson = respJson['tokenId']
request.session['tokenId'] = tokenIdJson
return render_to_response('menu/menu.html', request)
elif req.status_code == 401:
message = "Invalid username and/or password. Please, try again"
else:
form = LoginForm()
return render_to_response('registration/login.html', {'message': message, 'form': form},
context_instance=RequestContext(request))
The redirect assert it's commented because now it fails, when I do the debug I see an empty redirect_chain. I don't understand why happens this because running the web everything works, all views redirect as expected.
Why I only can check status_code? I'm doing something wrong when I redirect after a successful login that on a normal use it works but on the test not?
Thanks.
The remote authentication url expects the credentials as headers, but your local login view expects them as POST data. Your test passes the credentials as headers to your local view.
As a result, the form is passed an empty dictionary (request.POST contains no actual data), and the form is invalid. You get an empty form as a response, without any redirects.
You have to simply pass the credentials as post data to your local view:
def testLogin(self):
client = Client()
data = {'username': 'user', 'password': 'password'}
response = self.client.post('/login/', data=data, secure=True, follow=True)
assert (response.status_code == 200)
self.assertRedirects(response, '/menu/', status_code=301, target_status_code=200)
I have a modelForm as follows:
class UserProfileForm(forms.ModelForm):
class Meta:
model = FileUploads
fields = ['uploads']
and a view like so:
#csrf_exempt
def upper(request):
form = UserProfileForm(request.POST or None, request.FILES or None)
if form.is_valid():
form.save()
return HttpResponse(status=200)
return HttpResponse(status = 403)
And I have a simple script to send a multipart/encoded file to the view as such:
import requests
f = open('C:\\Users\\myname\\Desktop\\image.jpg', 'rb')
urls='http://localhost:8000/upper'
r=requests.post(urls, files= {'uploads':f})
print(r.status_code)
My question being: everything works fine as long as I have the csrrf_exempt decorator above the receiving view, that's fine for test environment. But what if I wanted the csrf protection in place? Considering the fact that I'm using requests module, how can I provide the csrf token?
You need to pass a cookie and a header with the same value:
import requests
f = open('C:\\Users\\myname\\Desktop\\image.jpg', 'rb')
urls='http://localhost:8000/upper'
cookies = {'csrftoken': 'token'}
headers = {'X-CSRF-TOKEN': 'token'}
r=requests.post(urls, files={'uploads':f}, cookies=cookies, headers=headers)
print(r.status_code)
The value of the token does not matter, you can take any literal, as long as they are the same.
I have created an angular resource as
var services = angular.module('Services', ['ngResource']).
// SEND_REPLY_SMS
factory('SendSMS', ['$resource', function($resource){
return $resource('/bulk-sms/reply/', null,
{
send: {method: 'POST'},
}
);
}]);
I used it as
var data = $scope.data;
SendSMS.send({},data,
function(data){
console.log(data);
},function(error){
console.log(error);
}
);
I have checked with console.log(data), data contains the data and the browser shows that the post request has submitted the data.
But When I receive it in django view, I can not get the data in django view and my django view is
class ReplySMSView(View):
def post(self, request):
data = request.POST.copy()
print 'post data', request.POST # here data is not printed
data = dict(data.items())
return self.process(request, data)
def get(self, request):
data = request.GET.copy()
print 'get data', request.GET # here data is not printed
data = dict(data.items())
return self.process(request, data)
def process(self, request, data):
dct = {}
print data
model = IncomingMessage
account = request.user.account
contacts = data.get('contacts', '')
contacts = contacts if contacts else get_contacts_by_filter(model, data)
# TODO: get_contacts_by_filter is not working here for IncomingMessage
message = data.get('message', '')
identity = data.get('identity', '')
if not contacts:
dct['contacts'] = 'No contacts found.'
if not message:
dct['message'] = 'Message is required.'
if not identity:
dct['identity'] = 'Identity is required.'
if dct:
return HttpResponse(json.dumps(dct), content_type='application/json')
response = send_bulk_sms(contacts, message, identity, account, module='bulk')
return HttpResponse(response)
I am not getting where is problem in this code ?
AngularJS will post that data serialized into JSON, but django is expecting to receive form data. If you want to receive that data, you can change default behavior of AngularJS, fetch data not using POST, but rather request.body or you can use some third-party package, like Django REST framework to do job for you.
When calling ajax, you recieve encoded json string in request body, so you need to decode it using python's json module to get python dict.
As django is a web framweork, it expect data from a form.
I really reccomend using this framework http://www.django-rest-framework.org/
Anyway, to get post data will be like this in your view:
(Pdb) request.POST
<QueryDict: {}>
(Pdb) import json
(Pdb) json.loads(request.body)
{u'operator': u'pepe', u'password': u'1234', u'transport': u'LUUAAA'}
import json
class ReplySMSView(View):
def post(self, request):
data = json.loads(request.body)
print 'post data', request.POST # here data is not printed
data = dict(data.items())
return self.process(request, data)