For my exercice I must with selenium and Chrome webdriver with python 2.7 click on the link :
https://test.com/console/remote.pl
Below structure of the html file :
<div class="leftside" >
<span class="spacer spacer-20"></span>
<img class="logo" src="https://img.test.com/frontoffice.png" />
<span class="spacer spacer-20"></span>
<img src="https://img.test.com/icons/fichiers.png" alt="" /> Mes fichiers
<img src="https://img.test.com/icons/publication.png" alt="" /> Gestion FTP
<img src="https://img.test.com/icons/telechargement-de-liens.png" alt="" /> Remote Upload
<img src="https://img.test.com/icons/profil.png" alt="" /> Mon profil
<img src="https://img.test.com/icons/parametres.png" alt="" /> Paramètres
<img src="https://img.test.com/icons/abonnement.png" alt="" /> Services Payants
<img src="https://img.test.com/icons/af.png" alt="" /> Af
<img src="https://img.test.com/icons/v.png" alt="" /> V
<img src="https://img.test.com/icons/logs.png" alt="" /> Jour
<img src="https://img.test.com/icons/deconnexion.png" alt="" /> Déconnexion
<span class="spacer spacer-20"></span>
<img src="https://img.test.com/btns/reverse.png">
</div>
I use : driver.find_element_by_xpath() as explained here : enter link description here
driver.find_element_by_xpath('//[#id="leftside"]/a[3]').click()
But I have this error message :
SyntaxError: Failed to execute 'evaluate' on 'Document': The string
'//[#id="leftside"]/a[3]' is not a valid XPath expression.
Who can help me please ?
Regards
I think you were pretty close. But as it is a <div> tag with class attribute set as leftside you have to be specific. But again the <a[3]> tag won't be the immediate child of driver.find_element_by_xpath("//div[#class='leftside'] node but a decendent, so instead of / you have to induce // as follows :
driver.find_element_by_xpath("//div[#class='leftside']//a[3]").click()
The issue is that you need to have a tag name also. So you should either use
driver.find_element_by_xpath('//*[#id="leftside"]/a[3]').click()
When you don't care about which tag it is. Or you should use the actual tag if you care
driver.find_element_by_xpath('//div[#id="leftside"]/a[3]').click()
I would suggest against using the second one, with the div tag. As xpath are slower and using a * may be slightly slower
i have got a similar problem but i didn't post it so thanks for the post.
the xpath is "//[#class="leftside"]/a[3]" there is no id with the name leftside in your html.
Related
So I am using Python Selenium to Login through a Webpage, i have done that before, my code was working pretty fine in that case:
driver.find_element_by_name("username").send_keys("user")
driver.find_element_by_name("password").send_keys("password")
driver.find_element_by_xpath("//input[#type='submit' and #value='Login / Register']").click()
But now, as I would like to do that again, it doesnt work anymore. I have tried some variations but none of them have worked for me, I also tried to access through layers, which I have seen in some older cases:
wait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.ID, "loginmask")))
driver.find_element_by_id("yourName").send_keys('username')
.....
I also tried something like that:
frame = driver.find_element_by_id("mainBody")
driver.switch_to.frame(frame)
driver.find_element_by_id("yourName").send_keys('username')
But somehow the find_element_by_id function isnt working for me, but the id is the only thing I can search for in that case.
enter image description here
this image of the login page might be helpful, as you can see the layers i might have to bypass. Maybe you can help me how to get the Element 'yourName'
Better Picture of HTML Body:
enter image description here
HTML Body:
<body id="mainBody" style="visibility: visible;">
<div id="treeLeft" style="height: 882px;">
<img id="logoLeft" src="img/ax_01.png" alt="">
<div id="buttonAreaLeft">
<img id="btnLogout" src="img/logoff.gif" title="logout" alt="" style="float:right">
<img id="btnHome" src="img/home.gif" title="startpage" alt="" style="float:left">
<br clear="all">
</div>
<div id="myMenuAccordion"></div>
</div>
<div id="mainLayer" style="left: 230px; width: 1125px; height: 860px; top: 12px;"><h1>AX Controller WEB-Interface</h1>
<div id="loginLayer" style="position:absolute;top:120px;left:120px;">
<h1>PLEASE LOGIN</h1>
<div class="hspacer5"></div>
<div class="hspacer5"></div>
<div id="loginmask" class="form">
<form method="post" action="index.php" onsubmit="venue.login();return false;">
<span class="label block">Your Name:</span><input type="text" id="yourName" maxlength="20" style="width:110px;" autocomplete="off"><br>
<div class="hspacer5"></div>
<span class="label block">Password:</span><input type="password" id="authCode" maxlength="20" style="width:110px;" autocomplete="off"><br>
<div class="hspacer5"></div>
<span class="label block"> </span><input type="submit" id="btnLogin" value="login" style="width:112px;padding:1px 0px;">
</form>
</div>
</div>
With this HTML code, I would like to extract the text ".Fermé(e)" :
<A onclick="return false;" onkeydown=campaignSPTree.onKeyDown(this); href="#r2">
<SPAN>
<LABEL class=hidden-label>Tree level 0, 3 of 3. </LABEL>
<SPAN id=r2c0Content name="r2c0Content">
<IMG border=0 alt="" src="/wfo/bbm/images/icons/campaign.gif?v=11.1.1.7673"> UAT Sud Est
</SPAN>
<LABEL id=campaignSPTreer2ChildInfo class=hidden-label childInfo=", node has 24 items. Press the right arrow key to show/hide this row's child items.">.Fermé(e)</LABEL>
</SPAN>
<LABEL class=hidden-label name="SelectInfo"></LABEL>
</A>
I thought I could do it using the following command, but it doesn't work :
infoOpen = browser.find_element_by_id("campaignSPTreer2ChildInfo").text
I don't have an error message but the value infoOpen is empty.
Could you please help me ?
I am trying to do my first web scraper. What I want to do is to get the status of RainbowSix's servers. Inspecting the webpage I found how to look for the state of the PC servers:
<div class="col-sm-2 col-md-2" id="dvPC">
<img class="" id="onlinePC" src="https://.../online.png"/>
<img class="hidden" id="interruptedPC" src="https://.../interrupted.png"/>
<img class="hidden" id="degradedPC" src="https://.../degradation.png"/>
<img class="hidden" id="maintenancePC" src="https://.../maintenance.png"/>
<span class="consoleTitle online" id="hdPC" style="width: 50px;"> PC </span>
</div>
Here, as the server is online, the img with "onlinePC" id has no class "hidden" and the span class is "consoleTitle online", otherwise it would be "consoleTitle interrupted" (or any of the other options).
So, my code (not finished, of course):
import requests
from bs4 import BeautifulSoup
url = 'https://.../status/'
response = requests.get(url)
soup = BeautifulSoup(response.content, "html5lib")
div = soup.find('div', attrs={'id':'dvPC'}) #Here I look for the div with the PC status, because the webpage also shows the status of PS4 and Xbox servers.
print(div)
Output print(div):
<div class="col-sm-2 col-md-2" id="dvPC">
<img class="hidden" id="onlinePC" src="https://.../online.png"/>
<img class="hidden" id="interruptedPC" src="https://.../interrupted.png"/>
<img class="hidden" id="degradedPC" src="https://.../degradation.png"/>
<img class="hidden" id="maintenancePC" src="https://.../maintenance.png"/>
<span class="consoleTitle" id="hdPC" style="width: 50px;"> PC </span>
</div>
As you can see, everything is hidden and the span class is incomplete. I guess there is a script who determines the hidden images and the span class. How can I "render" that script to get the correct html output?
Hope I am being clear.
I am using this xpath //a[contains(., 'Download Python')]
on www.python.org/downloads
in chrome developer tools I get 4 matches, each one match the following
<a class="button" href="https://www.python.org/ftp/python/3.7.4/python-3.7.4-macosx10.9.pkg">Download Python 3.7.4</a>
<a class="button" href="https://www.python.org/ftp/python/3.7.4/Python-3.7.4.tar.xz">Download Python 3.7.4</a>
<a class="button" href="https://www.python.org/ftp/python/3.7.4/python-3.7.4.exe">Download Python 3.7.4</a>
<a class="button" href="https://www.python.org/ftp/python/3.7.4/python-3.7.4.exe">Download Python 3.7.4</a>
the problem is only the third is clickable, the others give an error ElementNotInteractableException: Message: element not interactable
if I print the text of each element only the third one displays the text 'Download Python 3.7.4'
the rest print blank
elems = driver.find_elements_by_xpath(
"//a[contains(., 'Download Python')]")
for elem in elems:
print("-------")
print(elem.text)
print(elem.tag_name)
-------
a
-------
a
-------
Download Python 3.7.4
a
-------
a
as a result only elems[3].click works
This problem is not with your code, the python download page detects the OS you are browsing from and shows you the download link for that OS only. It has total 4 such buttons
<div class="download-os-mac-osx" style="display: none;">
<h1 class="call-to-action">Download the latest version for Mac OS X</h1>
<p class="download-buttons">
<a class="button" href="https://www.python.org/ftp/python/3.7.4/python-3.7.4-macosx10.9.pkg">Download Python 3.7.4</a>
</p>
</div>
<div class="download-os-source" style="display: none;">
<h1 class="call-to-action">Download the latest source release</h1>
<p class="download-buttons">
<a class="button" href="https://www.python.org/ftp/python/3.7.4/Python-3.7.4.tar.xz">Download Python 3.7.4</a>
</p>
</div>
<div class="download-os-windows" style="">
<h1 class="call-to-action">Download the latest version for Windows</h1>
<p class="download-buttons">
<a class="button" href="https://www.python.org/ftp/python/3.7.4/python-3.7.4.exe">Download Python 3.7.4</a>
</p>
</div>
<div class="download-unknown" style="display: none;">
<h1 class="call-to-action">Download the latest version of Python</h1>
<p class="download-buttons">
<a class="button" href="/downloads/release/python-374/">Download Python 3.7.4</a>
</p>
</div>
You need to select the one which has style = "" rest others wont be displayed on the page.
You can also use the xpath with index and it will point to the button corresponding to the index
(Xpath)[index]
Eg: (//*[#class="button"])[0] -for first button
Let me know if it was helpful
I got a problem to use "find_element_by_css_selector" to get the element "Select" (a href).
I tried the methods below but all of them didn't work:
driver.find_element_by_css_selector("div.plan.right > a.select.").click()
driver.find_element_by_xpath("//div[#class='plan right']/div[2]/a/select").click()
Could anyone kindly give me some suggestions? Thanks!!
<div class="choose_plan">
<h1>Sign up now for <strong>UNLIMITED</strong> access <br/>to all </h1>
<div class="plans">
<div class="plan left">
<div class="head">
<p>MONTHLY</p>
</div>
<div class="body">
<p>annually</p>
</div>
<hr />
SELECT
</div>
<div class="plan right">
<img alt="Popular-right" class="popular" src="/assetse8.png" />
<div class="head">
<p>14</p>
</div>
<div class="body">
<p>Unlimited</p>
</div>
<hr />
SELECT
</div>
</div>
</div>
I know you already have an answer but there's a simple alternative that may be helpful in the future. From the HTML you provided, it looks like the data-planId attribute is unique for each A tag. You can take advantage of that using the code below.
driver.find_element_by_css_selector("a[data-planId='31']")
See this CSS Selector reference for more info.
It would help to have well formed HTML, as line 15 (<div class="choose_plan">) appears to be unclosed. This solution below was done with this line removed, but the rest of the HTML as shown. You can test online XPath here.
driver.find_element_by_xpath("//div[#class='plan right']/a").click()
yields the following:
Element='SELECT'
I would try to make it simple:
driver.find_element_by_css_selector("div.right a.select")
Or:
driver.find_elements_by_link_text("SELECT")[-1]
Here we are basically getting the last a element having SELECT text.