Can I have any highlight kind of things using Python 2.7? Say when my script clicking on the submit button,feeding data into the text field or selecting values from the drop-down field, just to highlight on that element to make sure to the script runner that his/her script doing what he/she wants.
EDIT
I am using selenium-webdriver with python to automate some web based work on a third party application.
Thanks
This is something you need to do with javascript, not python.
[NOTE: I'm leaving this answer for historical purposes but readers should note that the original question has changed from concerning itself with Python to concerning itself with Selenium]
Assuming you're talking about a browser based application being served from a Python back-end server (and it's just a guess since there's no information in your post):
If you are constructing a response in your Python back-end, wrap the stuff that you want to highlight in a <span> tag and set a class on the span tag. Then, in your CSS define that class with whatever highlighting properties you want to use.
However, if you want to accomplish this highlighting in an already-loaded browser page without generating new HTML on the back end and returning that to the browser, then Python (on the server) has no knowledge of or ability to affect the web page in browser. You must accomplish this using Javascript or a Javascript library or framework in the browser.
Related
I work with python and data mine some content which I categorize into different categories.
Then I go to a specific webpage and submit manually the results.
Is there a way to automate the process? I guess this is a "form-submit" thread but I haven't seen any relevant module in Python. Can you suggest me something?
Selenium Webdriver is the most popular way to drive web pages, but Python also has beautifulsoup; Either library will work.
If you want make this automatic yo have to see which params are send in the form and make a request with this params to the endpoint but directly from your python app, or search a package that simulate a browser and fill the form, but I think that the correct way is making the request directly from your app
I am trying to scrape a web site using python and beautiful soup. I encountered that in some sites, the image links although seen on the browser is cannot be seen in the source code. However on using Chrome Inspect or Fiddler, we can see the the corresponding codes.
What I see in the source code is:
<div id="cntnt"></div>
But on Chrome Inspect, I can see a whole bunch of HTML\CSS code generated within this div class. Is there a way to load the generated content also within python? I am using the regular urllib in python and I am able to get the source but without the generated part.
I am not a web developer hence I am not able to express the behaviour in better terms. Please feel free to clarify if my question seems vague !
You need JavaScript Engine to parse and run JavaScript code inside the page.
There are a bunch of headless browsers that can help you
http://code.google.com/p/spynner/
http://phantomjs.org/
http://zombie.labnotes.org/
http://github.com/ryanpetrello/python-zombie
http://jeanphix.me/Ghost.py/
http://webscraping.com/blog/Scraping-JavaScript-webpages-with-webkit/
The Content of the website may be generated after load via javascript, In order to obtain the generated script via python refer to this answer
A regular scraper gets just the HTML document. To get any content generated by JavaScript logic, you rather need a Headless browser that would also generate the DOM, load and run the scripts like a regular browser would. The Wikipedia article and some other pages on the Net have lists of those and their capabilities.
Keep in mind when choosing that some previously major products of those are abandoned now.
TRY THIS FIRST!
Perhaps the data technically could be in the javascript itself and all this javascript engine business is needed. (Some GREAT links here!)
But from experience, my first guess is that the JS is pulling the data in via an ajax request. If you can get your program simulate that, you'll probably get everything you need handed right to you without any tedious parsing/executing/scraping involved!
It will take a little detective work though. I suggest turning on your network traffic logger (such as "Web Developer Toolbar" in Firefox) and then visiting the site. Focus your attention attention on any/all XmlHTTPRequests. The data you need should be found somewhere in one of these responses, probably in the middle of some JSON text.
Now, see if you can re-create that request and get the data directly. (NOTE: You may have to set the User-Agent of your request so the server thinks you're a "real" web browser.)
I tried web.seeother("link"), but this does not open it in a new tab. Now I can generate a link with a _blank tag, but then the user has to click on the link separately that is separate button for generating the link and another button to follow that link. I want to perform both with a single click. A server side method to do this would be best.
I am using the web.py framework.
As the document says web.seeother() is used for redirecting a user to another page. So a more clear way for asking your question is: "how to make web.seeother() open a link in a new tab"?
As I have observed the documents, There is no way to do that on server-side.
Not a web.py issue. Cannot be done from server-side by any python or non-python framework, must be done in the Client.
From the client, you can set target="_blank" in the HTML, or use javascript with something like window.open(url). Javascript will allow you to set size and position of second window.
I need to develop a simple login page using Python language with two fields and a button, like:
Username, Password, Login
I know there are some beautiful Python frameworks like
Django, Grok, WebPy, TurboGears
which support web development using Python, but mine is a basic requirement consisting of only 3 screens (pages):
1st page - Login page (Redirects to 2nd page when login button is clicked)
2nd page - Page with records in the form of a list, with an option for adding new records (Redirects to 3rd page when "Add Records" button is clicked)
3rd page - Page with fields, which are saved as records for the list on 2nd page (After entering details and clicking Submit)
So, I have decided to develop the above functionality using Python without using any framework, so that I can have flexibility as well as write my own code.
Is it possible to create a login page using Python without using a framework?
I haven't worked on web services and don't know the basics of web development in Python.
If possible, can you provide me an example on how to create a login page using Python and achieve the functionality described above?
Firstly, it's good to learn how to do things without the frameworks, but if you're doing this for anything but educational purposes it's most likely a mistake to say "my requirements are too simple to bother with frameworks." What you describe is on the simpler end of the spectrum, but already I can see some complications you're probably glossing over.
Anyway, to implement your 3 pages, you will need:
a web server
2 static HTML files
python handlers for GET on 2nd Page, POST from 1st and 3rd Page
storage (either flat file or database)
Look at A Simple Python CGI Server Tutorial to get started, and there are more details at TutorialsPoint: Python - CGI Programming.
Your HTML forms just need to have method="POST" and action set to the URL you're serving from python for each form. Your POST handlers should do what they need to do and then serve a 302 redirect.
If you don't know the basics of python web development, you're going to have your hands full and should probably start with a single user flat-file system. Maybe you can even get away with storing that in memory for now. Then if/when that stops doing it for you, try sqlite. Configuring web servers is a pain, too, so if you can just stick with BaseHTTPServer.
Finally, load your records in another python endpoint, make some HTML for each, slap it in a big string with surrounding HTML, and serve that puppy.
You requirement is very trivial. Though people may suggest micro frameworks like flask, bottle I would say you should try Django and have a look at Django Admin Panel. I think it's possible for you to meet your requirement using the Django admin feature.
Updated
Have a look at this http://docs.python.org/howto/webservers.html
Without a framework and if you don't want to mess with CGI you can take a look at http://webob.org/ .
I would like to change the logo of a website based on which menu is currently activated/seen by the user browsing the website.
For instance I have www.urltowebsite.com/menu1 = Header Logo 1
And then I have www.urltowebsite.com/menu2 = Header Logo 2
And on top of this I want to add an else statement stating that: If any other menu is selected, use header logo 3.
How can I make this possible with Python? I cant seem to wrap my head around what to define where and how to call up the different functions on the HTML website.
Oh and I insist doing this with Python. And preferably without any framework such as Django. But if needs be I can install web.py
EDIT:
Am I forced to go with php then? I would like to once and for all start utilizing Python on my web projects.
The website is made in simple HTML as I said first. The Javascript functions are only used to serve the HTML menu's through AJAX. Again this does not matter much for what I am trying to do, as menu's have classes and I can define those in php and thus change my logo/header.
What I want to do is to use Python in this instance. Here is a code snippet from the site:
<div id="header">
<span class="title"><img src="http://www.url.com/subfolder/images/logo.png"/>
</span>
</div>
And some more relevant to this:
<div id="menu">
<ul>
<li>001</li>
<li>002</li>
<li>003</li>
<li>004</li>
<li>005</li>
<li>006</li>
<li>007</li>
<li>008</li>
</ul>
</div>
So can I use python here?
You're asking to do the wrong thing the wrong way.
In order to change the logo based on the URL in Python , you need Python to generate the page and know what that url is.
There are two ways to do that in Python:
Use an existing Web Framework
Write your own Web Framework
"Python" doesn't know or care what your URL is - the frameworks and support libraries ( Django, Pyramid, Bottle, Flash, Tornado, Twisted, etc) figure out what the URL is by an integration with an underlying web server ( though some have their own webserver coupled in ). Similarly, PHP doesn't really know or care what the URL is - that information comes from an integration with Apache or FCGI/Nginx/etc. PHP tends to ship with most/all of that integration done. It's also worth noting that PHP is not just a language, but a web framework. Python is just a language.
Most Python frameworks will be written to the WSGI spec and have a "request" object that has all the data you want ( and many use the WebOb librbary for that ).
If you plan on doing everything with static HTML files, then you have a few options:
have a single static directory. use javascript to figure out the addressbar location, and render the corresponding logo / write the headers & footers.
have a "template" directory of all your HTML. use a Python script build a static version of each website with the custom headers/footers and configure your webserver to serve a different one for each domain.
No, Python cannot run inside the HTML web page. If you're really serving plain HTML pages then you must use javascript to execute code in the browser once the page is loaded. However, since you mention using AJAX, it sounds like it's not really true that you're serving plain HTML but rather have some server side code. If so, that server side code is the place to put your HTML-construction logic. To know the best way to do that, you would have to describe what's happening on the server.
Although I haven't used it, I have heard that the pyhp project more or less provides php-like embedded functionality for python.