We Keep Coding

Selenium Webdriver returns None when trying to access text

I'm trying to scrape some data from a website and I can't seem to get the text in between two tags as it keeps returning None. This is my code, could somebody tell me what is wrong?
uq = "https://my.uq.edu.au/programs-courses/requirements/program/2451/2021"
driver = webdriver.Firefox(executable_path="/Users/Connor/Downloads/geckodriver")
driver.get(uq)
groups = driver.find_element_by_xpath("/html/body/div/div[2]/div/div/div[3]/div[1]/div[2]/div[4]/div[1]/div[2]/a[1]/span[1]")
print(groups)
print(groups.text) #This won't get the text between the span tags e.g. <span>hello</span>

To get that text you'll need to click the parent element first.
However, you can avoid it with .get_attribute("innerHTML") and waiting for the presence of the element. Also, make sure you are using stable locators.
from selenium import webdriver
from selenium.webdriver.firefox.options import Options as FirefoxOptions
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
options = FirefoxOptions()
options.add_argument('window-size=1920x1080')
driver = webdriver.Firefox(options=options)
uq = "https://my.uq.edu.au/programs-courses/requirements/program/2451/2021"
driver.get(uq)
wait = WebDriverWait(driver, 15)
wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "a[title='Theory of Computing']>span:nth-of-type(1)")))
group = driver.find_element_by_css_selector("a[title='Theory of Computing']>span:nth-of-type(1)").get_attribute("innerHTML")
print(group)
Prints: COMP2048

You have to set an action to get the value of the WebElement.
So you have to use this code instead:
print(groups.getText());

Yes, same for me. I have tried every way possible. Trying this:
print(speed.get_attribute('outerHTML'))
I get
<span data-download-status-value="NaN" class="result-data-large number result-data-value download-speed"> </span>

Confirm whether to write comments with xpath

Could you make an example with the site? I want to ignore the posts with comments and just click the ones without comments.
from selenium import webdriver
import time
path = "C:\chromed\chromedriver.exe"
driver = webdriver.Chrome(path) #path
'''
'''
driver.get("http://cineaste.co.kr/") #url
time.sleep(0.5)
postclick = driver.find_element_by_xpath("//h3[.='영화이야기']/following::div[#class='widget-small-box'][1]//li[#class='ellipsis'][not(contains(.,'+'))]") #로그인창 활성화
postclick.click()
driver.close()
I tried this, but there was an error.
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//h3[.='영화이야기']/following::div[#class='widget-small-box'][1]//li[#class='ellipsis'][not(contains(.,'+'))]"}
(Session info: chrome=81.0.4044.138)
I want to click on a post that doesn't have any comments yet. And I want to skip posts with comments.
I'm still a beginner. Help me.

The Element you are looking for is not available.
You need to recreate the Xpath. The Xpath you created is incorrect. See following image :-

You'll need these imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Example with the first entry with no comments in the "movie-reviews" block :
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//h3[.='영화감상평']/following::div[#class='widget-small-box'][1]//li[#class='ellipsis'][not(contains(.,'+'))][1]"))).click()

How to click a button on a website by finding its id

When I run the code, the website loads up fine but then it won't click on the button- an error appears saying the element is not interacterble. What do I need to do to click the button? I am relatively new to this and would be grateful for any help.
I have already tried finding it by id and tag.
page = driver.get("https://kenpreston.co.uk/author/")
element = driver.find_element_by_id('mk-button-31')
element.click()
SOLVED:
I used driver.find_element_by_link_text and this worked fine.

I have checked the website and noticed that mk-button-31 is an id for a div tag and inside it there is an a tag. Try getting the url from the a tag and do another driver.get instead of clicking on it.
Also the whole div tag is not clickable so that is why you are getting this error.

Use sleep from time library to be sure page fully loaded
from time import sleep
page = driver.get("https://kenpreston.co.uk/author/")
sleep(2)
element = driver.find_element_by_id('mk-button-31')
element.click()

Looks like your element is not clickable you need to replace this id selector with the css and need to wait for the element before click on it.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
page = driver.get("https://kenpreston.co.uk/author/")
element = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#mk-button-31 span"))
element.click();

Consider adding Explicit Wait to your script as it might be the case the DOM had finished loading and the button you're looking for is still not there.
The classes you're looking for are:
WebDriverWait
expected_conditions
Suggested code change:
#your other imports here
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions
#your other code here
page = driver.get("https://kenpreston.co.uk/author/")
element = WebDriverWait(driver, 10).until(expected_conditions.element_to_be_clickable((By.ID, "mk-button-31")))
element.click()
More information: How to use Selenium to test web applications using AJAX technology

Finding dynamically named elements on a page using Selenium

I'm trying to locate and click on a section of a web page using Selenium so I can add a comment. I'm having a bit of trouble figuring out how to do this, though.
It seems like the class of the element changes from page-to-page. It also doesn't help that there are multiple similar elements in the document. Here's what I've come up so far:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--user-data-dir=/Users/me/Library/Application/Support/Google/Chrome/Default")
chrome_options.add_argument('--profile-directory=Profile 1')
driver = webdriver.Chrome(chrome_options=chrome_options)
driver.get("https://link-i-want-to-visit")
comment_section = driver.find_elements_by_xpath(("//input"))
print comment_section
comment_section.click()
Here's some of the markup from the relevant page:
<input class="sc-iKpIOp igoGaM" placeholder="Add a comment…">
On each different URL on this site, the class name appears to change. How can I circumvent that limitation, click into the input field, and send my comment?
Any guidance would be much appreciated. If it helps, this input field appears to be the last on the page, but I don't know if that's relevant (semantically).

Can't you use absolute xpath not including class?
e.g. /html/body/div/div/div/div/div/div/div/div/div/p[1] is your "I'm trying to..." paragraph on this page.
Also, try //input[placeholder="Add a comment…"]

comment_section = driver.find_elements_by_xpath("//input")
This will return list.So you can't click on list. You should use driver.find_element_by_xpath("//input") to click on the element.
However for a best practice Use WebDriverWait and Wait for the element element_to_be_clickable
and then click.
comment_section=WebDriverWait(driver,30).until(EC.element_to_be_clickable((By.XPATH,"//input[#placeholder='Add a comment…']")))
comment_section.click()
OR
comment_section=WebDriverWait(driver,30).until(EC.element_to_be_clickable((By.XPATH,"//input[contains(#placeholder,'Add a comment')]")))
comment_section.click()
You need to use following imports to execute above code.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

how to click the login button using selenium xpath

I am trying to login to a website using python so that I can get some of their text from the website.
Here is my code. There always an error at the end of the code after the id and password code.
import os
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver = webdriver.Chrome(executable_path=r"C:\chromedriver\chromedriver.exe")
driver.get('https://www.saramin.co.kr/zf_user/auth')
driver.implicitly_wait(3)
driver.find_element_by_name('id').send_keys('<<my_id>>')
driver.find_element_by_name('password').send_keys('<<my_password>>')
driver.find_element_by_xpath('//*[#id="frmNIDLogin"]/fieldset/input').click()
HTML source of the button:

Eventually I figured it out! Thanks for your answer though.
Here is the final code.
driver = webdriver.Chrome(executable_path="C:\chromedriver\chromedriver.exe")
browser = webdriver.Chrome('C:\chromedriver\chromedriver.exe')
driver.get('https://www.saramin.co.kr/zf_user/auth')
driver.implicitly_wait(3)
driver.find_element_by_name('id').send_keys('ID') driver.find_element_by_name('password').send_keys('PW')
driver.find_element_by_xpath( '//*[#class="btn-login"]' ).click()

The xpath is incorrect, it doesn't match anything in the page. Try
driver.find_element_by_xpath('//form[#id="login_frm"]//button[#class="btn-login"]').click()
or simply use submit() function on the <form>
form = driver.find_element_by_id('login_frm')
form.submit()

In the first case you were using 'id' ('//*[#id="frmNIDLogin"]) for click button, because 'id' changes every time page loads it was giving error. But in the second case when you used class ( '//*[#class="btn-login"]' ) it worked because it remains same every time page is loaded. Also as mentioned above the value of id in first case was wrong.