I'm looking at the html form of an external website (not maintained by myself) in the following format :
<form onsubmit="return check(this)" method=post action=address.aspx target=ttPOST>
....
</form>
I wish to post data to this external website without having to go through the form and pressing submit.
Can I simply go to the url 'address.aspx' and append it with the required input parameters ?
My goal is to automate the periodic posting of information, chosen from a list of frequently changing values on the website. (Using Python and AutoIt)
You can use JQuery.post()
<form action="#" class="myForm" method="post">
<input type="text" id="field1" name="field1" value="" runat="server" />
</form>
// Submit
<div onclick="submit();return false;">Submit</div>
Then the submit() function looks like
function submit() {
$.post("address.aspx", $("form.myForm").serialize(), function(data, textStatus)
{
// Handle success or error
});
}
In codebehind, you can access the post variables
Request.Form["field1"]
I should note that I'm unclear if you were wanting to automate the posting of data from outside a web browser or not. Others have answered doing it with script and such like from the web page so I thought I'd cover how it works when you are doing it from a standalone program.
For most languages you can get things that will help you simulate web requests. Most of these would probably allow you to make a post request and supply post data. I dont' know python and autoit but in teh general sense you'd just need to get the name value pairs by looking at the HTML of the form (or by examining a request being made to the server is probably better) and then make a post request.
As others have said if you want to just append the values to teh url then the server will need to be happy to accept a GET request instead of a post. Sometimes you will find that servers will do this happily (they don't care how form submission is done, as long as they get data), other times it will be specifically looking for a post and will ignore parameters passed as part of the querystring.
Change the method attribute from POST to GET.
Read about the differences here.
You can get away with changing the URL to go to the external site using the same syntax of GET parameter (?param1=val1¶m2=val2...), but to achieve this you will need to write this logic yourself in javascript.
Why do you want to get around the form submit? Submit the form and use the values in HTTP_POST. You can use HTTP_GET by changing the method to GET in the above html but the form will still submit.
One way to submit the params to address.aspx would be to use javascript....build a string of all the params and submit that to address.aspx when the user clicks on the submit button. You'll need to disable the submit buttons default behaviour.
You could make AJAX GET requests (appending the data to the URL) and execute them with a certain time interval.
I would recommend jQuery, but it may not be ideal in a .NET environment.
jQuery.ajax() - http://api.jquery.com/jQuery.ajax/
jQuery.get() - http://api.jquery.com/jQuery.get/
NOTE:
I wish to post data to this external website without having to go through the form and pressing submit.
I took this as meaning you didn't want to actually submit the form, do you want to submit the form but simply not by pressing the submit button? If you actually want the form to submit, then you should indeed just change the form's method attribute to get instead of post.
If however you do wish to stay on the form page and need to post the data to some other resource with the data in the URL string (there are reasons for and aganist doing this that you should look into -- scroll down to 9.3 GET), then just make an AJAX GET request, otherwise just POST the data using AJAX. Either way, you'll have to make some sort of an asynchronous call.
POST is not the same as GET -- if you append the data that you want to the URL, the page will not see your variables. That is to say GETTING (a.k.a going to) http://www.the-other-site.com/address.aspx?variable1=a&variable2=b will not work.
You need to build POST request and submit it to address.aspx.
If you want to do it yourself then you'll need Python's urllib2 module (More specifically, the Request object). If you want to use a pre-built solution, you'll need something like mechanize to POST your variables.
Related
I'm trying to write a small python script that will get a list of URLs (which are mostly just a search bar with some other elements), "write" something in the search bar and "press" enter.
My goal is to get to the next page, where all the search results are, and look at the new URL
The sites are different, so I can't just get the query parameter since I don't know it, and every site is different.
I was thinking about searching for the "input" part of the page (since the search bar is supposed to be the only input there), and send something to it. Then wait for the new URL.
Is that possible? Is there a smarter way?
I'm trying to avoid using something else but python for now (selenium, etc..)
I was searching every possible answer here and on the web, but nothing was possible so I was thinking about using the input part somehow...
Without selenium and similar softwares, you'll have to understand what is actually happening when you click such a button.
I'll take an example on a famous site (hint: if you read this you'll know which site I mean). In the HTML source I can see that (truncated) piece of code:
<form id="search" role="search" action=/search method="get" class="grid--cell fl-grow1 searchbar px12 js-searchbar " autocomplete="off">
<div class="ps-relative">
<input name="q"
type="text"
...
What it means is that when you submit the form, the browser would hit the URL /search (which here means https://stackoverflow.com/search), with a get request, wrapping all of the form's fields in the URL, after a ?. If I searched for the term python it would lead to http://stackoverflow.com/search?q=python.
Note that according to their content, the parameters would need to be URL-encoded.
If the form contains more input fields, you'll have to wrap them too, separating them by & signs, in such a way: param1=value1¶m2=value2&...
To sum up, searching only inputs won't be sufficient, you'll have to parse the forms.
Not knowing more about your data, I cannot elaborate, but I think you might be able to do something with that.
I need to view post parameters with python. The website in question has a post parameter of business_number where number changes after each successful action. I need to be able to get this number and then submit a POST request with a value.
A GET request does not give me the needed information. (I have tried with requests.) I have used a firefox addon called httprequester and used the "Submit" button and within the response is the information I need. I'm a real novice with HTTP :) so is there a way I can submit with Python (click the submit button) and then save the response in a variable that I will sort through.
I used the requests module to solve my problem:
http://docs.python-requests.org/en/latest/
I am trying to make it possible to update search results with Dajax as you change the search parameters. I have a GET request for the word search, and would like to modify it using Dajax.
I understand this query has to be in POST format, but how can I combine the word search (currently GET) with the additional parameters (POST)?
So far I am trying this, where variable 'request' is the GET request.
<p onClick="Dajaxice.myapp.get_by_topic(request,{'data':{'mytopic':this.innerHTML}});">mytopic</p>
The Dajax project is pretty much dead. He said it himself on the Github page. https://github.com/jorgebastida/django-dajax/ It really offers nothing you're not capable of accomplishing on your own using a simple form post with jquery ajax.
Why not post your search form to your view method using jquery Ajax which returns JSON and display that in your search results? You can make the post on every space or every few letters etc...
It may take some playing around with to get used to, but you're better off doing that than learning a dead library.
I'm trying to accomplish a little bit of automation which includes submitting a form on a webpage. The values for the form are already coded per item in the list.
I've tried many different modules with Python and nothing seems to give me an answer. I don't have access to Visual Basic and I've personally never dealt with .aspx pages before.
This is the Form name
And I thought I was set and ready to go when I found the parameters for the form:
function ShowEditForm(id, param1, param2, param3, param4) #actual parameter names removed for security
And this is the part that's the major headache:
<INPUT id=__EVENTTARGET type=hidden name=__EVENTTARGET> <INPUT id=__EVENTARGUMENT type=hidden name=__EVENTARGUMENT> <INPUT id=__VIEWSTATE type=hidden value=/wEPDw... #This continues for 800+ characters
I believe this is the cause of my failure of code, am I on a witchhunt trying to post to an .aspx form in python?
Thanks
you would need to parse/parameterize your post headers and contents. this can be non-trivial.
check out mechanize for access at the HTTP level, with some form handling convenience.
check out selenium, for driving a real browser in Python.
I don't think aspx has anything to do with it.
Have you tried http://pypi.python.org/pypi/selenium ?
Indeed, the server-side handling of the POST request won't work if those hidden values aren't present. ASP.NET uses that stuff to track statefulness across multiple requests. Reverse-engineering ASP.NET Web Forms HTTP requests isn't a fun endeavor.
You'll probably need to request the page, scrape the hidden values it gives you, and include those in the POST.
Stepping through a manual interaction with the page and capturing requests/responses in something like FireBug will also give you a good idea of the values being sent back and forth between the client and the server. It wouldn't surprise me if there's some JavaScript emitted to the response which dynamically modifies some hidden fields in server-pre-determined ways as well, helping to indicate which button was pressed or which control was in some way modified.
Asp.net has a feature called viewstate (encrypted page state settings) which you can't fake, and which the page may be using by default and will expect to see on post to the form when submitting back to itself (called post back).
If you control the .aspx code it likely has an associated .cs or .vb file with the code to do the form processing. You can change the code to get values from posted form or URL parameters instead of (or in addition to) controls on the original form. If the site is compiled and you don't see any .vb or .cs files to change you would need to locate the original source files for the solution.
I'm working on a Plone template. Currently I'm implementing a search on the template. There is a form on the template which submits to the same template i.e. the same page.
I need to enable certain parts of the page to be viewed i.e. the results DIV, only when the form in the page has been submitted. I am not sure of how to check if a page has been submitted and display certain portions of the page.
I've looked at this link on using form.submitted, but it is really not helpful. I'm guessing whatever I have to do will be done in tal:condition tag.
Any help will be appreciated.
You can add an hidden field, name for example "submitedform" with value="1" and then, make a condition on :
request.get('submitedform', 0)
I found a way to make it work.
I used a tal:condition and checked for data that is inside the request object upon form submission.
To examine what is in the request object, simply put the following on your page:
<div tal:replace="structure request" />
Note: when the request object is called, it renders a readable, HTML version of the data. We use "structure" to prevent escaping the HTML.
Now, for my case, I needed to check whether my search form was submitted. So inside the DIV tag, I checked if the query string was empty or not. Below is a sample of the code:
<!-- Form has been submitted -->
<div tal:condition="python:request.environ.get('QUERY_STRING') != ''"></div>
<!-- Form has not been submitted -->
<div tal:condition="python:request.environ.get('QUERY_STRING') == ''"></div>
Note that you can use any variable you want. I simply chose to use the QUERY_STRING from the request object.
This might not be perfect, but I hope it will give direction to someone else.