We Keep Coding

Get text from element immediately following an element with specific text

I am trying to scrape job postings and I'm having Selenium go to each individual posting to get some text. The problem is the structure of the page isn't the same for every posting so I'm trying to tell Selenium to grab the text from the element that immediately follows a div that contains the text Job Scope since that is where the text always is.
Here is one site, again it's not the same for every one but is always after Job Scope/Duties: https://recruitingbypaycor.com/career/JobIntroduction.action?clientId=8a87142e46c6fe710146f995773e6461&id=8a78839e812f7de70181456c3ad709ff&source=&lang=en
Here is my code:
job_descriptions = WebDriverWait(browser, 10).until(EC.presence_of_all_elements_located((By.ID,"gnewtonJobDescriptionText")))
for job in job_descriptions:
job_description.append(job.find_elements(By.XPATH, "//div[contains(text(),'Job Scope')]/following-sibling::div"))
I've got the script to work here but it produces empty list. So when I add .text to the end it errors our saying 'list' object has no attribute 'text' (obviously since it's producing an empty list. The only thing I can think of is that the text Job Scope is actually within a b tag inside the preceding div so maybe that's why?

Selenium + Python - entering text in form with no label/id?

I"m trying to enter some text in the box as shown below. I've narrowed the element down to SMDMainText. I've tried numerous ways of finding the element, however when I try to send_keys(), the result is that I get the error "element not interactable". The issue that I'm facing is that I'm trying to find an element called <input automplete="on"> and don't quite understand how to send text to it.
element = browser.find_element_by_class_name("styledModalDialogueDIV")
element = browser.find_element_by_css_selector("form")
element = browser.find_element_by_id("SMDMain")
element = browser.find_element_by_id("SMDMainText")
element = browser.find_element_by_css_selector("input")
# comment all but 1 line above, and try below:
element.send_keys("Hello World") # same error is produced for each try

You need to target the input tag.
element = browser.find_element_by_css_selector("#SMDMainText>div>input")

How to get an element by pressing TAB in selenium (Python)?

I want to fill out some personal data on a website. The first input element can be accessed by find_element_by_id but the id of the next text field has a different id every time I access the website. In a Browser, I can simply press the TAB key two times to get to the desired text input field. Is there a way to achieve similar behavior with selenium?
i tried the following:
input1 = browser.find_element_by_id('id_email')
input1.send_keys("email#something.com")
input2 = browser.send_keys(Keys.TAB).send_keys(Keys.TAB)
input2.send_keys("Something else")
But Line 3 gives me:
AttributeError: 'WebDriver' object has no attribute 'send_keys'

You cannot send keys against the browser object.
This line:
input2 = browser.send_keys(Keys.TAB).send_keys(Keys.TAB)
is not valid. It even tells you as such: 'WebDriver' object has no attribute 'send_keys'
That says your webdriver object (which you called "browser") does not have a attribute ( a method/ a function) called Send_Keys.
Top tip to avoid this sort of problem is to use a good IDE with intellisense. That will tell you the methods you can use.
In vscode, you get this:
As you type, it tells you valid commands - and you can see the driver has no send keys!
What you need to do is use .sendKeys(..) against a web element.
In your code you already have input1 - that is a web element. You can send keys against that.
Something like this:
input1 = browser.find_element_by_id('id_email')
input1.send_keys("email#something.com")
input1.send_keys(Keys.Tab)
If you want to do multiple tabs from the same object, you can just add multiple. This will tab 3 times
.send_keys(Keys.TAB + Keys.TAB + Keys.TAB)
When i run a sample script on google.com,
I tab 3 times i go to the googlesearch button. (first tab is the clear, second tab is the microphone, 3rd tab goes to the button):
Finally, using tabs to navigate is a LAST RESORT. They can be flaky and inconsistent. You REALLY should get an identifier for your object.
If you can share your URL or the page DOM then I'm help you identify a working identifier. I know you say there is no ID but there many ways to access objects.

welcome!
i can only guess, since i'm not able to test the code out. But seems like you are not getting the right element with the 2 Tabs, since it starts from the start of the html page and not from the input1 field. You'd better getting the password by its id or parents id, rather then with Key.TAB.

Find number of check-boxes on webpage, when page source contains 'type' as a string also, using selenium python

This is my code which finds number of check-boxes on the current webpage
checkboxes=driver.find_elements(By.XPATH("html/body/div[2]/div/section[8]/div/div/div[3]/div/div[2]/table/tbody/tr[1]/td[1]/input[#type='checkbox']"))
but it gives following error:
'str' object is not callable
As HTML DOM contains string "checkbox" in it.
Thus I am unable to find a way to calculate total number of check-boxes.

You should send two arguments to find_elements, like checkboxes=driver.find_elements(By.XPATH, xpath) while there is only one argument in your code- checkboxes=driver.find_elements(By.XPATH(xpath)). So you should use:
checkboxes=driver.find_elements(By.XPATH, "html/body/div[2]/div/section[8]/div/div/div[3]/div/div[2]/table/tbody/tr[1]/td[1]/input[#type='checkbox']")
UPDATE
Regarding OP comment: If you want to get only checkboxes visible on page, use following code:
checkboxes_list = [check for check in driver_find_elements_by_xpath('//input[#type="checkbox"]') if check.is_displayed()]

Selenium get textarea error

I'm trying to scrap in a website using Selenium and Python, I got stucked in a field called 'textarea' by the website. This is how the website HTML calls the area where I'm trying to extract text:
<textarea class="script" onclick="this.focus();this.select()" readonly="readonly" id="script">
After this code comes the text that I want to get. Here is the code that I'm using:
getCode = driver.find_elements_by_tag_name('textarea')
My problem is that It does not recognize the text by the following codes:
getCode.submit()
getCode.click()
getCode.text()
This is the code error that I always get:
Traceback (most recent call last):
File "ScprL.py", line 55, in module
print (repr(getCode.text))
AttributeError: 'list' object has no attribute 'text'
I would apreciate your help!

You should use driver.find_element_by_tag_name instead
When you use driver.find_elements you get a list of webElements. You should extract the element from the list
elem = driver.find_elements_by_tag_name('textarea')[0]
print element.text
If you have multiple textareas on the page, then you should try to finding the one you need like below
textareas = driver.find_elements_by_tag_name('textarea')
for i, textarea in enumerate(textareas):
print '{} is at index {}'.format(textarea.text, i)
And then use the appropriate i value to get textareas[i]

As you are using driver.find_elements_by_tag_name('textarea') it will retrieve list of web elements. Need to collect these web elements then iterate one by one then get text of each of web element. below is the example in java,
List<WebElement> ButtonNamelist = driver.findElements(By.cssSelector(".locatorHere"));
System.out.println(ButtonNamelist.size());
for(int i=0;i<ButtonNamelist.size();i++){
System.out.println(ButtonNamelist.get(i).getText());
}
Thank You,
Murali

There are two functions for each locator in selenium: "find_elements" and "find_element". The difference is pretty simple: the first one return a list of elements that satisfy selector and the second one returns first found element. You can read more about locating elements here.
So you either need to change your function to find_element_by_tag_name or to retrieve first element from list: find_element_by_tag_name()[0].