We Keep Coding

send keys to 'display : block' modal in python selenium

I am having trouble sending keys using selenium.
What I want to do is, I want to send_keys('samsung')
BUT, since 'style = display' is changing from 'none' to 'block' when I click it, I cannot send keys.
<div class="modal-search modal-search-analysis" id="analysisSearchModal" style="display: none; transform: translate(0px, 0px); opacity: 1;">
after click
<div class="modal-search modal-search-analysis" id="analysisSearchModal" style="display: block; transform: translate(0px, 0px); opacity: 1;">
Here are the element
this webpage's modal is not in iframe, it's in div.
<div class="modal-search modal-search-analysis" id="analysisSearchModal" style="display: block; transform: translate(0px, 0px); opacity: 1;">
<div class="modal-search-analysis-keyword" data-tooltip-container="">
<input type="text" name="searchKeywordf714s6nggsw" id="searchKeywordf714s6nggsw" data-index="f714s6nggsw" class="input-style type1 x-small modal-search-analysis-keyword-input search-input input-text" placeholder="Type search keyword" maxlength="15" tabindex="1">
Here are codes that I've been trying
browser.find_element(By.CLASS_NAME, "vbtn btn-submit").click() #click search button
time.sleep(3)
input = browser.find_element(by=By.XPATH, value = '//*[#id="searchKeywordn7ssjvuykn"]')
input.send_keys('samsung')
I've also tried this code
browser.find_element(By.CLASS_NAME, "vbtn btn-submit").click()
time.sleep(3)
container = browser.find_element(by=By.XPATH, value = '//*[#id="analysisSearchModal"]')
browser.execute_script("argments[0].style.display = 'block';", container)
input = browser.find_element(by=By.XPATH, value = '//*[#id="searchKeywordn7ssjvuykn"]')
input.send_keys('samsung')
Thank you for those who can help me solve this problem:)

I can see in your html page id = "searchKeywordf714s6nggsw"
but you are using id="searchKeywordn7ssjvuykn"

I have 4 nested div tags and when I print text using find_all, it prints the text 4 times

I am extracting text from an html file which contains a lot of div tags. However, at some places there are say 4 nested div tags and when I print text, it prints it 4 times.
<div>
<div id="PGBRK" style="TEXT-INDENT: 0pt; WIDTH: 100%; MARGIN-LEFT: 0pt; MARGIN-RIGHT: 0pt">
<div id="PN" style="PAGE-BREAK-AFTER: always; WIDTH: 100%">
<div style="TEXT-ALIGN: center; WIDTH: 100%"><font style="DISPLAY: inline; FONT-FAMILY: Times New Roman; FONT-SIZE: 10pt">27</font></div>
</div>
</div>
</div>
For example, here if I do:
for item in page_soup.find_all('div'):
if "27" in item.text:
print(item)
It prints the number 27 four times and therefore messes up whole text.
How can I get my code to only print the nested text once?
EDIT 1:
This works well for this part of the code. But like I said, this is only true at some places. For example, when I do:
for item in page_soup.find_all('div', recursive = False):
print(item)
It does not print anything. For reference, this is the document I am trying to scrape.
EDIT 2:
From the given html, I am trying to extract the section "ITEM 1A. RISK FACTORS".
should_print = False
for item in page_soup.find_all('div'):
if "ITEM 1A." in item.text:
should_print = True
elif "ITEM 1B." in item.text:
break
if should_print:
print(item)
So I am printing everything starting from ITEM 1A. until it finds ITEM 1B.
Here at some places there are nested div tags, which gets printed multiple times with this piece of code.
If I do, recursive = False, it does not print anything.

Here is one option
import bs4, re
html = '''<div>
<div id="PGBRK" style="TEXT-INDENT: 0pt; WIDTH: 100%; MARGIN-LEFT: 0pt; MARGIN-RIGHT: 0pt">
<div id="PN" style="PAGE-BREAK-AFTER: always; WIDTH: 100%">
<div style="TEXT-ALIGN: center; WIDTH: 100%"><font style="DISPLAY: inline; FONT-FAMILY: Times New Roman; FONT-SIZE: 10pt">27</font></div>
</div>
</div>
</div>
</div>'''
soup = bs4.BeautifulSoup(html,'html.parser')
elements = soup.find_all(text=re.compile('27'))
print(elements)
output
[u'27']

printing everything starting from ITEM 1A. until it finds ITEM 1B
Trough .string attribute (https://www.crummy.com/software/BeautifulSoup/bs4/doc/#string)
import requests
from bs4 import BeautifulSoup
url = 'https://www.sec.gov/Archives/edgar/data/4904/000000490412000013/ye11aep10k.htm'
html_doc = requests.get(url).content
page_soup = BeautifulSoup(html_doc, 'html.parser')
do_print = False
for el in page_soup.find_all('div'):
if el.string:
if "ITEM 1A" in el.string:
do_print = True
elif "ITEM 1B" in el.string:
break
if do_print:
print(el)
The output (I'll show the representative start and end blocks without middle part, to make a short dump):
<div align="justify" style="TEXT-INDENT: 0pt; DISPLAY: block; MARGIN-LEFT: 0pt; MARGIN-RIGHT: 0pt"><font style="DISPLAY: inline; FONT-FAMILY: Times New Roman; FONT-SIZE: 12pt; FONT-WEIGHT: bold"><font style="DISPLAY: inline; TEXT-DECORATION: underline">ITEM 1A.   RISK FACTORS</font></font></div>
<div style="TEXT-INDENT: 0pt; DISPLAY: block"><br/>
</div>
<div align="justify" style="TEXT-INDENT: 0pt; DISPLAY: block; MARGIN-LEFT: 0pt; MARGIN-RIGHT: 0pt"><font style="DISPLAY: inline; FONT-FAMILY: Times New Roman; FONT-SIZE: 12pt; FONT-WEIGHT: bold">GENERAL RISKS OF OUR REGULATED OPERATIONS</font></div>
<div style="TEXT-INDENT: 0pt; DISPLAY: block">
<div align="justify" style="TEXT-INDENT: 0pt; DISPLAY: block; MARGIN-LEFT: 0pt; MARGIN-RIGHT: 0pt"><font style="FONT-STYLE: italic; DISPLAY: inline; FONT-FAMILY: Times New Roman; FONT-SIZE: 12pt; FONT-WEIGHT: bold"> </font></div>
<div align="justify" style="TEXT-INDENT: 0pt; DISPLAY: block; MARGIN-LEFT: 0pt; MARGIN-RIGHT: 0pt"><font style="FONT-STYLE: italic; DISPLAY: inline; FONT-FAMILY: Times New Roman; FONT-SIZE: 12pt; FONT-WEIGHT: bold">The regulatory environment in Ohio has recently become unpredictable and increasingly uncertain. – Affecting AEP and OPCo</font></div>
<div style="TEXT-INDENT: 0pt; DISPLAY: block"><br/>
.....
<div style="TEXT-ALIGN: center; WIDTH: 100%"><font style="DISPLAY: inline; FONT-FAMILY: Times New Roman; FONT-SIZE: 10pt">37</font></div>
<div style="TEXT-ALIGN: center; WIDTH: 100%">
<hr noshade="" size="2" style="COLOR: black"/>
</div>
<div id="HDR">
<div align="right" id="GLHDR" style="WIDTH: 100%"><font style="DISPLAY: inline; FONT-FAMILY: Times New Roman; FONT-SIZE: 8pt">  </font></div>
</div>
<div align="right" id="GLHDR" style="WIDTH: 100%"><font style="DISPLAY: inline; FONT-FAMILY: Times New Roman; FONT-SIZE: 8pt">  </font></div>
<div style="TEXT-INDENT: 0pt; DISPLAY: block"> </div>

You can provide the option text = "27" to search the divs by text and identify only that exact div. The below code should work fine. If you want to get all the divs then just remove the text = "27" or replace it with what text that you want to find. You can also use recursive = False to get only the top level divs.
Edit 1:
from bs4 import BeautifulSoup
t = '''
<div>
27
</div>
<div>
<div id="PGBRK" style="TEXT-INDENT: 0pt; WIDTH: 100%; MARGIN-LEFT: 0pt; MARGIN-RIGHT: 0pt">
<div id="PN" style="PAGE-BREAK-AFTER: always; WIDTH: 100%">
<div style="TEXT-ALIGN: center; WIDTH: 100%"><font style="DISPLAY: inline; FONT-FAMILY: Times New Roman; FONT-SIZE: 10pt">27</font></div>
</div>
</div>
</div>
</div>
'''
page_soup = BeautifulSoup(t, 'html.parser')
for item in page_soup.find_all('div', text="27"):
print(item.text)
Edit 2:
I have added a specific code that works for your problem specifically. Try the below code. The div range that you are expecting is from 567 - 715 with page numbers removed.
import requests
from bs4 import BeautifulSoup
resp = requests.get(
r'https://www.sec.gov/Archives/edgar/data/4904/000000490412000013/ye11aep10k.htm')
t = resp.text
page_soup = BeautifulSoup(t, 'html.parser')
s = 'body > div:not(#PGBRK)'
for i in page_soup.select(s)[567:715]:
print(i.get_text(strip=True))

Well I think that is a cool question, and I don't see a simple answer if you want to generalize it to find out what text there is at each level without resorting to searching for a specific number like 27. Beautiful Soup doesn't seem to have a function for showing only the text in the top , and recursive=False simply prevents the search from delving below the first level but will still include everything below the first level as contents, so if at the top level of tags then it will capture it and everything below it
So I think you'd actually have to recurse down the tree of divs and compare the text at each level. I figure this out. It prints in reverse order as it bubbles up from the recursion but that could be stored in a list and output in forward order.
from bs4 import BeautifulSoup
soup = BeautifulSoup('<div>1A<div>2A</div>1B<div>2B<div>3A</div><div>3A</div>2C</div>1C</div>', 'html.parser')
def mangle(node):
divs = node.find_all('div')
if len(divs):
result = [divs[0]] + [n for n in divs[0].next_siblings if n.__class__.__name__ == 'Tag']
txt = []
for r in result:
txt.append(r.__repr__())
for c in mangle(r):
txt[-1] = txt[-1].replace(c.__repr__(), '')
print(''.join(BeautifulSoup(t, 'html.parser').text for t in txt))
return result
else:
return []
if __name__ == '__main__':
mangle(soup)
Basically it walks down the branches of divs and builds lists at each fork of the tree, including the tags, then the caller removes anything found below it leaving just the text that is defined at that level. I keep the tags in place so that text patterns appearing at multiple levels don't get removed by mistake.
Output from the html 1A2A1B2B3A3A2C1C was
3A3A
2A2B2C
1A1B1C
which is the 3rd, 2nd and 1st nesting levels respectively. Hope this helps.

I will answer my own question since I finally got it to work.
The solution was easy, I was just thinking it too hard.
I just added the condition that the parent of the item should not be "div". Now the program does not print the text multiple times.
should_print = False
for item in page_soup.find_all('div'):
if item.name == "div" and item.parent.name != "div"
if "ITEM 1A." in item.text:
should_print = True
elif "ITEM 1B." in item.text:
break
if should_print:
print(item)
Thank you everyone for your contributions. Appreciated...

Python XPath that starts with X and ends with zero or the lowest number

I would be really grateful if someone could give me a hand with writing an XPath that achieves the following:
Finds an element with the following id: 'revit_form_Button_50' where the number constantly changes (could be 30/40 but it seems like the first one on the page always ends with 0) and also has the title "Daily Tech Totals / Daily Tech Totals / Employee Daily Totals Report"
OR Finds and element like this: 'revit_form_Button_'+'mininumnumber' and has the tile: "Daily Tech Totals / Daily Tech Totals / Employee Daily Totals Report"
Any approach would help me tremendously.
My first two failed attempts:
browser.find_element_by_xpath("//*[starts-with(#id,'revit_form_Button_' ) and ends-with(#id,'0')]").click()
browser.find_element_by_xpath("//*[boolean(number(substring-before(substring-after(#id, 'revit_form_Button_'), '0')))]").click()
<div id="reporting_grid_CellReportTitle_10" class="rptCellReportTitle" widgetid="reporting_grid_CellReportTitle_10">
<div dojoattachpoint="divReportTitle" class="rptCellReportTitleSpanNode"><span class="dijit dijitInline dijitReset revitButtonHideBackground revitButton" data-dojo-attach-point="focusNode,titleNode,stateNode" role="button" aria-labelledby="revit_form_Button_30_label" data-dojo-attach-event="ondijitclick:_onClick" tabindex="0" id="revit_form_Button_30" widgetid="revit_form_Button_30" style="position: relative; user-select: none;">
<span class="dijitReset revitIconNode dijitNoIcon" data-dojo-attach-point="iconNode" style="display: none;"></span>
<span class="dijitReset dijitButtonText" id="revit_form_Button_30_label" data-dojo-attach-point="containerNode" style="padding-left: 0px; text-transform: none;"><b>Daily Tech Totals / Daily Tech Totals / Employee Daily Totals Report</b></span>
<span class="dijitReset revitIconNode revitIconRightNode dijitNoIcon" data-dojo-attach-point="iconRightNode"></span>
<input type="button" value="" class="dijitOffScreen" tabindex="-1" role="presentation" data-dojo-attach-point="valueNode">
</span></div>
<div dojoattachpoint="divNotes" class="rptCellReportTitleNotesNode"></div>
<div dojoattachpoint="divSchedule" class="rptCellReportTitleScheduleNode"></div>
</div>
<div id="reporting_grid_CellReportTitle_11" class="rptCellReportTitle" widgetid="reporting_grid_CellReportTitle_11">
<div dojoattachpoint="divReportTitle" class="rptCellReportTitleSpanNode"><span class="dijit dijitInline dijitReset revitButton revitButtonHideBackground" data-dojo-attach-point="focusNode,titleNode,stateNode" role="button" aria-labelledby="revit_form_Button_31_label" data-dojo-attach-event="ondijitclick:_onClick" tabindex="0" id="revit_form_Button_31" widgetid="revit_form_Button_31" style="position: relative; user-select: none;">
<span class="dijitReset revitIconNode dijitNoIcon" data-dojo-attach-point="iconNode" style="display: none;"></span>
<span class="dijitReset dijitButtonText" id="revit_form_Button_31_label" data-dojo-attach-point="containerNode" style="padding-left: 0px; text-transform: none;"><b>Daily Tech Totals / Daily Tech Totals / Employee Daily Totals Report</b></span>
<span class="dijitReset revitIconNode revitIconRightNode dijitNoIcon" data-dojo-attach-point="iconRightNode"></span>
<input type="button" value="" class="dijitOffScreen" tabindex="-1" role="presentation" data-dojo-attach-point="valueNode">
</span></div>
<div dojoattachpoint="divNotes" class="rptCellReportTitleNotesNode"></div>
<div dojoattachpoint="divSchedule" class="rptCellReportTitleScheduleNode"></div>
</div>

There is no ends-with in XPath 1.0, but you can use
"//*[starts-with(#id,'revit_form_Button_' ) and substring(#id,string-length(#id)) = '0']"
To satisfy the additional condition
and also has the title "Daily Tech Totals / Daily Tech Totals / Employee Daily Totals Report"
add a clause like
child::*[text() = 'full content of the child element here']
or
child::*[contains(text(),'text to match here')]
to your XPath. In you example, the child element was <b>. So, you could also write child::b instead of child::*.
Putting it all together:
"//*[starts-with(#id,'revit_form_Button_')
and substring(#id,string-length(#id)) = '0'
and child::*[text() = 'Daily Tech Totals / Daily Tech Totals / Employee Daily Totals Report']
]"
This is one single XPath expression. Broken into several lines for readability, but could be a single line.

Python Selenium print out value of textfield is showing empty. The value is not printing

I am trying to print out the value of a textfield to the console.
The webpage has the value 1,000.000 in the textfield. 1,000.000
should be printed but my method is printing blank.
I am using Python Webdriver. I am using the .text which should get the text value of the textfield.
My method is:
from selenium.webdriver.common.by import By
# max records textfield has the value 1,000.000 as default
def print_maxrecords_textfield(self):
max_records_textfield = self.driver.find_element((By.XPATH, '//span[#class="gwt-InlineLabel myinlineblock marginbelow" and contains(text(), "Max records")]/following-sibling::*'))
print "max_records_textfield = "
print max_records_textfield.text
I call the method from my test case class as dp.print_maxrecords_textfield()
The output to the console is as follows:
max_records_textfield =
It should say max_records_textfield = 1,000.00
The HTML snippet is:
<div class="padding gwt-TabLayoutPanelContent" style="position: absolute; left: 0px; top: 0px; right: 0px; bottom: 0px;" aria-hidden="false">
<div class="clear">
<span class="gwt-InlineLabel marginbelow myinlineblock" style="width: 8em;">Location</span>
<input class="gwt-TextBox marginbelow" type="text" style="width: 30em;"/>
</div>
<div class="clear">
<span class="gwt-InlineLabel myinlineblock marginbelow" style="width: 8em;">Max records</span>
<input class="gwt-IntegerBox marginbelow" type="text"/>
</div>

Actually try getting the value instead of text.
from selenium.webdriver.common.by import By
# max records textfield has the value 1,000.000 as default
def print_maxrecords_textfield(self):
max_records_textfield = self.driver.find_element((By.XPATH, '//span[#class="gwt-InlineLabel myinlineblock marginbelow" and contains(text(), "Max records")]/following-sibling::*'))
print "max_records_textfield = "
print max_records_textfield.get_attribute('value')

Selenium2Library: move mouse position for click

I am new to the whole Robotframework and the Selenium2Library and I have a problem.
I have two divs: rasterContainer and anlageContainer.
They have the same x- and y-offset. The anlageContainer has a z-offset of 3 and the rasterContainer has 0. The anlageContainer lies on top of the rasterContainer.
Those two build a time bar.The anlageContainer got just one id and the rasterContainer contains many other divs, each of them with an id.
If you mouse over those divs, the rasterContainer shows you the time. If you click there, you just click on the anlageContainer and some other methods calculate the offset to get the time and opens a window with this time in a textbox.
What I want to do:
I want to move my mouse to an element of the rasterContainer and click at the same position on the anlageContainer.
What I have tried:
I began to write my own library in python. I have just one method which gets an instance of the Selenium2Library, the vertical value of the mouse position (the mouse is on top of the anlageContainer) and the vertical value of the rasterContainer's element.
def click_on_element(self, vertEl, vertMo, se2lib):
v = vertEl - vertMo
#Get Webdriver
driver = se2lib._current_browser()
#ActionChains instance
ac = webdriver.ActionChains(driver)
ac.move_by_offset(0, v)
ac.click().perform()
return "On my way"
With move_by_offset: The window opens but with the wrong time (07:00). I wanted to have 09:30.
I also tried:
#Get Element
elmfinder = ElementFinder()
elm = elmfinder.find(driver, "5_09_30")[0]
ac.move_to_element(elm)
ac.move_to_element_with_offset(elm, 461, 422)
The window opened neither with move_to_element nor with move_to_element_with_offset.
I really don't know what I am missing here.
Any hints would help.
EDIT:
HTML code:
<div id="resource_id_5_2013-07-30" class="resource" daylenght="720" loaded="false" date="2013-07-30" time="07:00" style="top: 0px; height: 1540px; width: 309.75px; left: 619.5px;">
<div class="terminContainer"></div>
<div class="overlapContainer" style="width: 10%; position: absolute; left: 90%; height: 1560.0px; top: 0px;"></div>
<div id="5" class="anlageContainer" style="width: 10%; height: 1440px; top: 0px;" title="08:53"></div>
<div class="rasterContainer" style="width: 10%; height: 1440px; top: 0px;">
<div id="5_07_00" class="rasterLabel" style="position: absolute; top: 0px;">7:00</div>
<div id="5_07_15" class="rasterLabel" style="position: absolute; top: 30px;">7:15</div>
<div id="5_07_30" class="rasterLabel" style="position: absolute; top: 60px;">7:30</div>
etc...
</div>
</div>
CSS style:
.rasterContainer{
position: absolute;
background-color: #EEEEEE;
}
.anlageContainer:hover + .rasterContainer{
background-color: #e3e3e3;
}
.rasterLabel{
z-index: 2;
font-size: 0.7em;
color: #000;
border-top: solid 1px #888;
}
.anlageContainer{
z-index: 3;
cursor: pointer;
position: absolute;
}
There you can see that the anlageContainer is above the rasterContainer. And between them are the rasterLabels --> z-index.
The anlageContainer has
dojo.connect(anlageContainer, 'onclick', function(clickevt){
addTermin(resourceId, getOffsetY(clickevt)/g_terminMultiplikator, datum);
});
Two links to images:
Time bar
3D time bar

element = find_element_by_xpath (driver, ".//div[#id='resource_id_5_2013-07-30']//div[#class='anlageContainer']")
element.click()