I'm trying to automate a specific task using python. I need to execute a specific task when a specific colour pops up on the display. For example:
During the automation process, when the code to open a web browser is executed and a web browser pops up,I need to make sure that it actually worked, by detecting the colour at the exact coordinate where the logo of the browser is situated. So that it confirms the browser is open, then I need to use that information to execute another code.
As we know, when we try to open a browser, there will be a lag to open up the browser. Since that lag will be different, I need to execute the next code only after opening the browser.
I have gone too far with my explanation, but if you got any idea from what I have just explained, please help me with what you know?
I am using helium library for web scraping dynamic websites
I found it much faster to use tabs than using many windows yet some websites when I open them in a new tab they show some ads and I could not find a way to close them. That made me look for a lot of JS codes, which I do not understand, and most of the times they are not working when executed using helium library in python even though they work fine in the Console of Google Chrome.
However, the code should be running headless and I am doing all of this only for testing and here is my main question
Does it matter if used new tab or new window if I am going to use headless browser in python helium
is it going to be faster just like what happens when it runs normal or since it is headless it makes no difference ?
not to forget to mention that it is way easier for me to work with many windows since I won't have to use JS codes.
driver.execute_script('''$x('xpath')[0].remove();''')
driver.execute_script('''$x('xpath')[0].click();''')
$x('xpath')[0].remove();
$x('xpath')[0].click();
selenium.common.exceptions.JavascriptException: Message: javascript error: $x is not defined
they both work just fine in Google Chrome Console but for some reason, I can not execute them python helium.
I tried using to give more time using time.sleep(5) to load the page but it did not work
not to mention that I tried using the click() method from helium it gives me a LookupError()
I found a way to test it for myself it might not be accurate but the results gave me a clear winner I used the windows task manager to see the percentage of CPU usage in all the cases when running headless tabs were faster and used way less CPU than new windows
Essentially what I want to do is be able to click a button and have the webpage state be stored somewhere on the HDD so it doesn't need to just sit in RAM, and when it's loaded again at some later time the page pops up exactly as it was before as if it had never been closed without the need to download anything over the internet to restore it (although additional resource requests that didn't exist when the page was saved should still download properly).
(as an example, firefox does this when it crashes, all the tabs are restored, text you've typed is still in the textboxes, etc..)
I don't care if that button is in a firefox plugin, chrome, or even a custom browser that I program myself with something like webkit perhaps.
I've been trying for days to find a way to do this. I made WebKit programs in both C++ and Python but every time I think I'm getting close there is some deficiency in webkit or a build-in security measure that prevents me from doing this. I tried creating MHTML archives but they don't allow javascript to download new data over the internet, I tried pickling the entire WebKit.WebView object in python, I tried looking through webkit code to see if I could patch the behavior into the source code myself
I'm running out of ideas and the only one I see left is to just post this online. Is there any way that I can do this, in any programming or scripting language, using any libraries at all?
I just have no idea where to turn next.
I have done a python webkit navigator, with GTK.
And, as I better know html/css for rendering than others ways, I have done a software using python, wekbit and GTK.
I got some questions. I have read documentation that I found, and did a lot of researchs on google and stackoverflow. I still got lot of questions.
First, in my app, I change the title of the window to communicate between javascript and python. I wish to do the same in my navigator, but I can not (I need title). Is their any others ways ?
I would love to bind javascript events listeners to python, without changing the title.
EDIT
I have found a solution. We can bind some events to python.
You can have more documentation about events in python :
import webkit
help(webkit.WebView)
I have tried with console-message. This events returns me 4 args : webview, webframe, int, msg. What is the int ? In most of messages it is valued to 13... If someone know what it means.
Second, my linux version of my navigator plays really well media element (audio, video...). I assume, it is because linux rulz and depedencies are pretty well installed on my computer.
But on windows it is another things ...
I have seen that I can build webkit for windows with these dependencies.
But I have found some javascript codecs for reading media elements (https://github.com/audiocogs). Should it be better than I inject this javascript, or compiling webkit in my own way ?
Third, can I handle cache settings ? I am pretty sure that now, there is no cache in my browser. (my code is really light now on).
Fourth, can I handle HTTP request ? (cookies, apache auth, ...)
Fifth, I use WebView.zoom_in(), and zoom_out functions. And definitively it has not the same behaviour than firefox or chrome when I zoom_in or zoom_out.
With firefox or chrome, it's like if zoom_out make you have more pixels than before. I mean if you zoom out on chrome, you can have different media queries than before.
With WebView zooms functions, it's like if there is only the font size who change.
How could I do zooms like firefox and chrome ?
Sixth, I could use gecko engine instead of webkit. But I do not know, how to choose between those twos.
It seems that webkit is nicely imported in python and gtk, and linux. But gecko probably too. How could I, in a cleverway, choose ?
Seventh, I got some streaming problems. For instance, if I want to hear some long music, or some videos, and pause them for some times, when I play back the media, my browser bug. There is no error in console, and the webkit.webview is all blank. I can reload, and it works again... How can I handle this error ?
Some relevant samples of my light code :
class nav:
def __init__(self):
self.browser = webkit.WebView()
self.browser.connect("create-web-view",self.set)
self.browser.set_full_content_zoom(True)
self.browser.get_settings().set_property("enable-webaudio",True)
self.browser.open(url)
def on_zoom_in(self, widget):
self.browser.zoom_in()
def on_zoom_out(self, widget):
self.browser.zoom_out()
def on_zoom_n(self, widget):
self.browser.set_zoom_level(1.0)
Thank you,
Not the answers for all the questions you have, but this will help.
There is no need to change the title for communicating between javascript and python. You can alert mechanism. Some examples can be found https://github.com/nhrdl/notesMD - the tool I wrote few days back. In simplest terms, your script uses alert function and python gets the callback. You can parse the text of alert message and decide on action.
Your code has nothing to do with webkit cache. Its function of what pages your application is visiting and what server prefers. Server can ask for some resources to be cached (e.g. images/javascript) and others not to be cached. I know webkit gtk 2 supports some more functions for caching, but don't recall much in Webkit gtk 1. I have seen it caching the files in your home directory though.
For cookies look at python webkit webview remember cookies?. Webkit also has various methods to get request and response and you can listen to various soup events for the things that interest you.
I have not read about python bindings for geco engine. That does not mean it does not exists, only I have not seen it.
I've tried asking this question on the KDE development forum, but haven't received a satisfying answer so far.
I've developed a Python script for Amarok 1.4 which retrieves upcoming events for the currently playing artist and displays them in the context browser. The user can click each event to know more about it, but so far clicking takes him to another Amarok tab, in which he must then click another button to finally get the link to open in the external browser.
What I'd like to know is whether there is a way to open the link directly in the external browser.
Update: I recently started using Pana, which actually opens the links directly in the Wikipedia tab, so I guess I'll stick to that player in the future.
I know nothing about Amarok, but in general you can spawn the platform's default web browser on a URL:
on modern open-source desktops (KDE, GNOME, Xfce) by spawning the xdg-open command;
on OS X with the open command;
on Windows with the built-in os.startfile method.
There is also the webbrowser module, but it will often pick the wrong browser and everything about it is outmoded and ugly.