We Keep Coding

How to "click" a button using requests

So, im trying to make a program that can click a button from multiple links, links that i will get from a list, but first i need to understand how can I do this with only one link. They all have the same HTML structure, but I dont know how to do this.
HTML:
<div class="_55wr">
<form method="post">
<input type="hidden" name="fb_dtsg" value="AQG7lSxYN2mb:AQFMMcWJcZtZ" autocomplete="off">
<input type="hidden" name="jazoest" value="22090" autocomplete="off">
<table class="btnBar">
<tbody>
<tr>
<td>
<button type="submit" value="Bloquear" class="_54k8 _52jg _56bs _26vk _56b_ _56bu" name="confirmed" data-sigil="touchable"><span class="_55sr">Bloquear</span></button>
</td>
<td>
<button type="submit" value="Cancelar" class="_54k8 _52jg _56bs _26vk _56b_ _56bt" name="canceled" data-sigil="touchable"><span class="_55sr">Cancelar</span></button>
</td>
</tr>
</tbody>
</table>
</form>
</div>
The idea is to click the first button ('<button type="submit" value="Bloquear"...').
Current code:
import requests
auth = ('email#email.com', 'pass')
payload = {}
url = 'https://www.example.com'
s = requests.Session()
res = s.get('https://www.example.com')
cookies = res.cookies
r = requests.post(url, cookies = cookies, auth = auth, verify = False, payload = payload)
I searched for similar questions, but every question was using some "id" ({'id':'value'}), which I don't have here. So, what value should i use in payload?

The requests library makes HTTP requests which means that it does not render the JS and it can not click buttons. Monitor your network behavior using google dev tools to know what data are sent to the server when you click a button, and then make a POST request sending the same data by the params keyword in the request. For example
data = {'button' : 'clicked1'}
r.requests.post('your_url.com', params = data)
For clicking buttons, I would personally use the selenium library which emulates the browser and provides its automation

I would use Selenium.
The code would be like so:
driver = webdriver.Firefox()
button = driver.find_element_by_xpath("//button[#value = \'Bloquear\']")
button.click()

Django/Python: Pulling value from template into views.py

I currently have a search function in my views.py file like so:
def json_search(request):
query = request.GET.get('query')
api_key = locu_api
url = 'https://api.locu.com/v1_0/venue/search/?api_key=' + api_key
locality = query.replace(' ', '%20')
category =
final_url = url + "&locality=" + locality + "&category=" + category
json_obj = urllib2.urlopen(final_url)
decoded_data = json.load(json_obj)
return render(request, 'loc_search.html',
{'objects': decoded_data['objects']})
What I have set up is a drop-down search bar whereby I want the category variable within my json_search() function to automatically be assigned to the selected option on the drop-down bar before the form is submitted using the submit button. The search bar looks like this :
And the code like this:
<form action="{% url 'search' %}">
<div class="input-group">
<input name="query" input id="address" type="textbox" placeholder="City or Zipcode" class="form-control datebox">
<div class="input-group-btn">
<button class="btn btn-default" type="submit" id="addressSearch">Search</button>
<button name = "category_query" tabindex="-3" data-toggle="dropdown" class="btn btn-default dropdown-toggle" type="button">
<span class="caret"></span>
<span class="sr-only">Toggle Dropdown</span>
</button>
<ul class="dropdown-menu" >
<li>Resturant</li>
<li>Activities</li>
<li>Bar / Club</li>
<li class="divider"></li>
<li>other</li>
</ul>
</div>
</div>
</form>
Is this even possible?

You would have to get the value back to the backend view code in some fashion in order for that to happen.
It would be possible to do so prior submitting the form. For example, you could use an Ajax call within the template code to hit the same URL serviced by the json_search function, passing category in the URL, and then pulling it out of request.GET.
If you wanted it to be assigned upon dropdown selection, you would want to attach a click event handler via jQuery to that dropdown, and then in that handler's function, get the selected value, and then add it to the Ajax call back to your json_search function.
In your json_search code, you'll want to differentiate handling the submit (which should be a POST) vs. general GET handling (perhaps based on whether various parameters are present in the URL).
Edit in response to comment from OP:
It's certainly not trivial, especially if you've not worked with Ajax before, but it shouldn't be too bad overall (and once you get the hang of it, this paradigm can be used for all sorts of interaction with other modules like Datatables and many others, not to mention your own Django backend).
While there are many different ways to do this, I'm a fan of using jQuery's when in conjunction with done (used in examples on the same page). The when lets you fire off multiple asynchronous Ajax requests, and the done acts as a join point where you wait for them to finish before proceeding.

Yes, this is possible, you would make all the links in the dropdown have a onclick handler which would need to save the category. Then instead of a url for the form you would use a submit function which would send your form data + the category.
That is something easy to do with angular + ui.bootstrap.
with jQuery
http://plnkr.co/edit/iBY2n9dq8Tn95IUGwNAB?p=preview
You need to transform your links not to have a valid href and instead call a function, e.g.:
Restaurant
and add a hidden field for the category
<input name="category" input="" id="category" type="hidden" placeholder="Category" class="form-control" />
and some easy javascript
function setCategory(category) {
alert('category (hidden) = ' + category);
$('#category').val(category);
}

Sending data to server flask html

I am trying to create a simple checkbox that sends the data to server here is my html code.
<form action="." method="POST">
<div class="checksheet">
<input id="XML Parser" class="checkbox" type="checkbox"/>XML Parser
<input id="Feed Parser" class="checkbox" type="checkbox"/>Feed Parser
<input id="Text Parser" class="checkbox" type="checkbox"/>Text Parser
<input id="Case Normalization" class="checkbox" type="checkbox"/>Case Normalization
<input id="Stemmer" class="checkbox" type="checkbox"/> Stemmer
</div>
<div class="submit"><input type="submit" value="Send" name="raw_text"></div>
</form>
What I am trying to do is very similar to the question asked here: Send Data from a textbox into Flask?
But except with the text box.. I have checkboxes.
But I get this error:
Not Found
The requested URL was not found on the server.
If you entered the URL manually please check your spelling and try again.
MY server side code (in flask) is:
#app.route('/raw_text.html')
def home ():
file = "sample.xml"
contents = open(file).read()
contents = contents.decode('utf-8')
return render_template('raw_text.html', contents=contents,file=file)
#app.route('/raw_text.html',methods=['POST'])
def get_data():
print "REQUEST ",request.form()
data = request.form['raw_text']
print data
return "Processed"
Any suggestions.
Thanks

A few things:
Your checkbox elements need a name attribute, this is what is used when the data is sent to the back end. Each checkbox that is related to each other needs to have the same name.
Your action attribute needs to point to a URL. If you are posting it to the same page as the form, you can remove the attribute.
ID's cannot contain spaces.
To be accessible the check boxes need <label>s,

Displaying results from search API

I'm trying to get to grips with web2py/python. I want to get the user to fill in a search form, the term they search for is sent to my python script which should send the query to the blekko API and output the results to them in a new HTML page. I've implemented the following code but instead of my normal index page appearing, I'm getting the html response directly from blekko with '%(query)' /html appearing in it's search bar. Really need some help with this!
HTML form on the default/index.html page
<body>
<div id="MainArea">
<p align="center">MY SEARCH ENGINE</p>
<form name="form1" method="get" action="">
<label for="SearchBar"></label>
<div align="center">
<input name="SearchBar" type="text" id="SearchBar" value="" size = "100px"><br />
<input name="submit" type="submit" value="Search">
</div>
</form>
<p align="center"> </p>
Python code on the default.py controller
import urllib2
def index():
import urllib2
address = "http://www.blekko.com/?q='%(query)'+/html&auth=<mykey>"
query = request.vars.query
response = urllib2.urlopen(address)
html=response.read()
return html

I think you are misunderstanding how string formatting works. You need to put the address and query together still:
address = "http://www.blekko.com/?q='%(query)s'+/html&auth=<mykey>" % dict(query=request.vars.query)

Add a hidden field to your form, call it "submitted". Then reformat your controller function as such:
import urllib2
def index():
if request.vars.submitted:
address = "http://www.blekko.com/?q='%(query)'+/html&auth=<mykey>"
query = request.vars.query
response = urllib2.urlopen(address)
html=response.read()
return html
else:
return dict()
This will show your index page unless the form was submitted and the page received the "submitted" form variable.

The /html doesn't do anything. Glad your question got answered. There is python client code for the blekko search api here: https://github.com/sampsyo/python-blekko

Submitting a form in mechanize

I'm having issues submitting the result of a form submission (I can submit a form, but I can't submit the form on the page that follows the first).
I have:
browser = mechanize.Browser()
browser.set_handle_robots(False)
browser.open('https://www.example.com/login')
browser.select_form(nr=0)
browser.form['j_username'] = 'username'
browser.form['j_password'] = 'password'
req = browser.submit()
This works, as print req results in
`
<body onload="document.forms[0].submit()">
<noscript>
<p>
<strong>Note:</strong> Since your browser does not support JavaScript,
you must press the Continue button once to proceed.
</p>
</noscript>
<form action="https://www.example.com/Shibboleth.sso/SAML2/POST" method="post">
<div>
<input type="hidden" name="RelayState" value="cookie:95ca495c"/>
<input type="hidden" name="SAMLResponse" value="really long encoded value"/>
</div>
<noscript>
<div>
<input type="submit" value="Continue"/>
</div>
</noscript>
</form>
</body>
`
But I get errors when I try to use req.select_form(nr=0)
I assume this is probably from something along the lines of how mechanize returns objects from submit() and that I'm going about this the wrong way.
Any input or guidance would be appreciated :)

try again browser.select_form(nr=0) instead of req.select_form(nr=0). (after submitting or clicking a link or so, the new response is considered as an actual browser page - like in a browser :) )