We Keep Coding

Flask form data empty [duplicate]

This question already has answers here:
Sending data from HTML form to a Python script in Flask
(2 answers)
Closed 1 year ago.
I've created a form with flask. This is the python code that handles the requests that come from this form:
#app.route("/submitclips/", methods=['GET', 'POST'])
def submitclips():
print(request)
print(request.method)
result = request.form.to_dict(flat=False)
print(result)
print(request.data)
HTML code:
<form action="/submitclips" method="POST">
<input type="hidden" name="taskpath" value="paththotask">
<table>
<th>Video</th>
<th>Mute</th>
<th>Delete</th>
<th>Keep</th>
<tr>
<td>
<video width="320" height="240" controls>
<source src="videosource.mp4" type="video/mp4">
</video>
</td>
<td>
<input type="radio" id="name$mute" name="name" value="mute">
</td>
<td>
<input type="radio" id="name$delete" name="name" value="delete">
</td>
<td>
<input type="radio" id="name$keep" name="name" value="keep" checked>
</td>
</tr>
</table>
<input type="submit">
</form>
This is a table and has more than one row, I changed the values to make it more readable. I guarantee that all names are unique where they are supposed to be unique though.
It has some code below it, but it crashes there because the form seems to be empty. I have checked the HTML and it's all correct, all inputs have a name. I also recorded the network traffic on google chrome to test this and I can see that all data in the form is present in the request data. So I'm pretty sure it's correct on the front-end. The problem is that it just doesn't show up in this method. In the code you can see several print statements and all of these show me empty data and not the data that I could see in my chrome browser. The method is GET, I don't know if it should be GET. In HTML I selected it to be a POST method, but if I remove the GET option from the python code, it gives me a "method not allowed" error.
I don't know what to do, this is some older code that used to work on my windows machine. I didn't change anything about it, but now I'm working on linux and it's completely broken, none of the forms work, it's all empty data.

Here you have mentioned that the endpoint /submitclips has methods GET and POST. It means that whenever i hit the url "baseurl/submitclips" then this function will be executed. Inside the function you have not specified for which method you will have what response, i.e if i call the endpoint using GET or POST you will process it the same way. This is not a good practice. I suggest you put something like :
if request.method == 'POST' or if request.method == 'GET' to separate the execution based on the type of method.
Now coming to the HTML, you must have the HTML from where you are sending the request to the server. If that data is coming from a form, then as part of the form you can add two attributes,
<form method="post"> and <form action="/submitclips"> to specify that on submit of this form,you will be sending the form data through POST method to the "/submitclips" url. It will look like this.
<form method="post" action="/submitclips"">
For the Server side,
def submitclips():
if request.method == 'POST' :
print(request)
print(request.method)
result = request.form.to_dict(flat=False)
print(result)
print(request.data)
It should work after that.

I have found the error. My HTML code would submit the form to "/submitclips" while the python code received requests from "/submitclips/". I don't know why this is wrong though, the tutorial that I followed for flask told me specifically that putting a slash at the end meant that it could receive requests from both "/submitclips" and "/submitclips/". This also worked earlier on my windows machine, but doesn't work anymore on my linux machine. I'm glad it's solved, but if anyone has an explanation for why this is wrong, feel free to answer.

Redirect user in Telegram Bot to an external link with POST request

Since I'm new to this POST/GET HTTP stuff, I might be getting things wrong, that's why I'll put my question in 2 ways. Maybe one way will be better than the other :)
I'm developing a Telegram Bot using PyTelegramBotAPI, and it needs to include an online payment.
For the online payment I need the user to follow a link with POST method (it's an external link + I need to pass form data), but that's what causes difficulties for me.
I.
In my code I perform the following:
req = requests.post(url=url, data=data)
Where url is the URL of the website to which the client must be redirected, and data is the data that it needs to pass with the POST request when redirecting.
It works fine as a request in Python, but obviously it can't redirect the client to the website needed.
I tried to generate a URL and pass it to the client using
url = url + urlencode(data=data)
Where url is again the URL of the website. But in this case the website tells me that the method used is incorrect. I guess the link becomes a GET request, instead of a POST request.
How can I redirect the client to that link with POST method?
II.
Another way of putting this question is this:
The company which processes the online payments requires them to be performed using the following HTML form:
<form action=”https://securesandbox.webpay.by/” method="post">
<input type=”hidden” name=”*scart” >
<input type=”hidden” name=”wsb_storeid” value=”11111111”>
<input type=”hidden” name=”wsb_order_num” value=”ORDER-12345678”>
<input type=”hidden” name=”wsb_currency_id” value=”BYN”>
<input type=”hidden” name=”wsb_version” value=”2”>
<input type=”hidden” name=”wsb_seed” value=”1242649174”>
<input type=”hidden” name=”wsb_signature” value=”124264917411111111ORDER-123456781BYN10123456”>
<input type=”hidden” name=”wsb_test” value=”1”>
<input type=”hidden” name=”wsb_invoice_item_name[0]” value=”Товар 1”>
<input type=”hidden” name=”wsb_invoice_item_quantity[0]” value=”2”>
<input type=”hidden” name=”wsb_invoice_item_price[0]” value=”10”>
<input type=”hidden” name=”wsb_total” value=”10”>
<input type="submit" value="Купить">
</form>
This would work well if I used HTML pages, but since my web app is a Telegram Bot, hence this wouldn't work. Therefore I need to generate this HTML form automatically in Python (namely, I need to change the "value" fields for every payment).
How can I imitate this HTML form in my Telegram Bot and redirect the client after some trigger?

web scraping a webpage which has dynamic contents loaded via ajax

Say I wish to scrape products on this page(http://shop.coles.com.au/online/national/bread-bakery/fresh/bread#pageNumber=2&currentPageSize=20)
But the products is loaded from a post request. A lot of posts here suggest to simulate a request to get dynamic contents, but in my case the Form Data is unknown for me, i.e. catalogId, categoryId.
I'm wondering is it possible to get the response after the ajax call is finished?

You can get the catalogId and other parameter values needed to make the POST request from the form with id="search":
<form id="search" name="search" action="http://shop.coles.com.au/online/SearchDisplay?pageView=image&catalogId=10576&beginIndex=0&langId=-1&storeId=10601" method="get" role="search">
<input type="hidden" name="storeId" value="10601" id="WC_CachedHeaderDisplay_FormInput_storeId_In_CatalogSearchForm_1">
<input type="hidden" name="catalogId" value="10576" id="WC_CachedHeaderDisplay_FormInput_catalogId_In_CatalogSearchForm_1">
<input type="hidden" name="langId" value="-1" id="WC_CachedHeaderDisplay_FormInput_langId_In_CatalogSearchForm_1">
<input type="hidden" name="beginIndex" value="0" id="WC_CachedHeaderDisplay_FormInput_beginIndex_In_CatalogSearchForm_1">
<input type="hidden" name="browseView" value="false" id="WC_CachedHeaderDisplay_FormInput_browseView_In_CatalogSearchForm_1">
<input type="hidden" name="searchSource" value="Q" id="WC_CachedHeaderDisplay_FormInput_searchSource_In_CatalogSearchForm_1">
...
</form>
Use the FormRequest to submit this form.
I'm wondering is it possible to get the response after the ajax call is finished?
Scrapy is not a browser - it does not make additional AJAX requests to load the page and there is nothing built-in to execute JavaScript. You may look into using a real browser and solve it on a higher level - look into selenium package. There is also the related scrapy-splash project.
See also:
selenium with scrapy for dynamic page

tornado.web.stream_request_body: _xsrf missing error even with _xsrf input within html

Utilizing the Tornado library within Python I have come across a very unusual error. It seems that when I have decorated my file upload handler with '#tornado.web.stream_request_body' the webserver throws the error:
WARNING:tornado.general:403 POST /upload (ip-address): '_xsrf' argument missing from POST
WARNING:tornado.access:403 POST /upload (ip-address) 1.44ms
The code governing the upload is as follows:
#tornado.web.stream_request_body
class Upload(BaseHandler):
def prepare(self):
print self.request.headers
def data_received(self,chunk):
print chunk
#tornado.web.authenticated
def post(self):
self.redirect("/")
where my BaseHandler is a web.RequestHandler subclass with various helper functions (retrieving user info from cookies and whatnot).
Within my HTML template, I have the appropriate xsrf function call as seen here:
<form enctype="multipart/form-data" action="/upload" method="post" id="upload_form" class="form-upload">
{% raw xsrf_form_html() %}
<input type="file" name="upFile" required/>
<button class="btn btn-lg btn-primary btn-block-submit" type="submit">Submit</button>
</form>
and is generating the proper xsrf input within the browser:
<form enctype="multipart/form-data" action="/upload" method="post" id="upload_form" class="form-upload">
<input type="hidden" name="_xsrf" value="2|787b7c6e|4a82eabcd1c253fcabc9cac1e374e913|1430160367"/>
<input type="file" name="upFile" required/>
<button class="btn btn-lg btn-primary btn-block-submit" type="submit">Submit</button>
</form>
When I turn off xsrf_cookies within the webserver settings, all is well and everything functions as normal. However I feel that this is not ideal.
While xsrf_cookies is set to False, if given a text file called "stuff.txt" with a body of "testfile" the output is:
------WebKitFormBoundary4iHkIqUNgfqVErRB
Content-Disposition: form-data; name="_xsrf"
2|787b7c6e|4a82eabcd1c253fcabc9cac1e374e913|1430160367
------WebKitFormBoundary4iHkIqUNgfqVErRB
Content-Disposition: form-data; name="upFile"; filename="stuff.txt"
Content-Type: text/plain
testfile
------WebKitFormBoundary4iHkIqUNgfqVErRB--
From that output, my guess is that the xsrf value is being captured by the stream_request_body and not passed to the appropriate xsrf validation class.
Any help on this would be greatly appreciated. Thank you in advance!

Tornado does not currently (as of version 4.1) support streaming multi-part uploads. This means that uploads you wish to stream must be simple PUTs, instead of a POST that mixes the uploaded data with other form fields like _xsrf. To use XSRF protection in this scenario you must pass the XSRF token via an HTTP header (X-Xsrf-Token) instead of via a form field. Unfortunately this is incompatible with non-javascript web form uploads; you must have a client capable of setting arbitrary HTTP headers.

Creating an object with an external form

I have a upload form that is going to S3 --
<form action="https://test.s3.amazonaws.com/" method="post" enctype="multipart/form-data">
<input type="file" name="file" id="id_file" />
<input type="submit" value="Upload to Amazon S3" name="upload">
</form>
When the form is submitted to S3, I also need to create an object in my db and get the id of the object. How would I do this (without redirecting to my view)? Thank you.

You could use an iframe to send the form and js to search for the id you are looking for in the retrieved page.
You could also use another script to perform the post method like php with the curl library. You could grab all the data retrieved from there, search for what you are looking for and add what you need to your database.