We Keep Coding

Using Selenium to Scrape Java-Heavy Website - Returning None

New coder here. I've been trying to scrape just one piece of text on a very java based website for a while now using Selenium. Not sure what I am doing wrong that this point.
from selenium import webdriver
driver = webdriver.Chrome()
driver.get("https://explorer.helium.com/accounts/13pm9juR7WPjAf7EVWgq5EQAaRTppu2EE7ReuEL9jpkHQMJCjn9")
earnings = driver.find_elements_by_class_name('text-base text-gray-600 mb-1 tracking-tight w-full break-all')
print(earnings)
driver.quit()
Image of attempted element to scrape :
I am trying to scrape that dollar amount in this container so I can eventually use it in a daily report that I am building.
Everything I have tried has resulted in it returning none. Even when I try to grab the text from that element.
Here is website link: https://explorer.helium.com/accounts/13pm9juR7WPjAf7EVWgq5EQAaRTppu2EE7ReuEL9jpkHQMJCjn9

You should wait until javascript loads, page loads, elements loads.
_ = driver.Manage().Timeouts().ImplicitWait;
You can create condition until element appers.
ExpectedConditions ...... define selenium conditions
//This is how we specify the condition to wait on.
wait.until(ExpectedConditions.alertIsPresent());
You can use XPATH ! The DOLLAR XPATH IS
/html/body/div[1]/div/article/div[2]/div/div[2]/div/div[2]/div[1]/div[2]/div[2]
FIREFOX XPATH FINDER
https://addons.mozilla.org/en-US/firefox/addon/xpath_finder/

You can use this xpath
//*[#id="app"]/article/div[2]/div/div[2]/div/div[2]/div[3]/div[1]/div[1]/div[3]

Python Edge Driver Web Automation Help - Cannot find Xpath

Inspect Youtube Page Element
I am new to Python and I am learning how to automate webpages. I under the basics around using the different locators under the inspect element tab to drive my code.
I have written some basic code to skip youtube ads however I am stuck on finding the correct page element to agree to the privacy policy pop up box in Youtube. I have used ChroPath to try and find the xpath of the page however there doesn't appear to be one. I was unable to locate any other page elements and I was wondering if anyone has any ideas on how I can automate the click of the 'I Agree' button?
Python Code:
from msedge.selenium_tools import Edge, EdgeOptions
options = EdgeOptions()
options.use_chromium = True
driver = Edge(options=options)
driver.get('http://www.youtube.com')
def agree():
while True:
try:
driver.find_element_by_xpath('/html/body/ytd-app/ytd-popup-container/paper-dialog/yt-upsell-dialog-renderer/div/div[3]/div[1]/yt-button-renderer/a/paper-button').click()
driver.find_elements_by_xpath('.<span class="RveJvd snByac">I agree</span>').click()
except:
continue
if __name__ == '__main__':
agree()
Youtube Inspect Element Screeshot is below:

I don't know if the xpath in your code is right as I can't see the whole html structure of the page. But you can use F12 dev tools in Edge to find the xpath and to check if the xpath you find is right:
Open the page you want to automate and open F12 dev tools in Edge.
Use Ctrl+Shift+C and click the element you want to locate and find the html code of the element.
Right click the html code and select Copy -> Copy XPath.
Then you can try to use the xpath you copy.
Besides, find_elements_by_xpath(xpath) will return a list with elements if any was found. I think you need to specify which one element of the list to click with. You need to pass in the value number of the elements list like this [x], for example:
driver.find_elements_by_xpath('.<span class="RveJvd snByac">I agree</span>')[0].click()

When inspecting the page elements I overlooked the element of iframe. After doing some digging I came across the fact I had to tell the Selenium Driver to switch from the main page to the iframe. I added the following code and now the click to the 'I Agree' button is automated:
frame_element = driver.find_element_by_id('iframe')
driver.switch_to.frame(frame_element)
agree2 = driver.find_element_by_xpath("/html/body/div/c-wiz/div[2]/div/div/div/div/div[2]/form/div/span/span").click()
driver.switch_to.default_content()

Using Selenium and conditional xpath to click element

Currently I am learning to use selenium to automate testing. One of my task is to click to the next page on a website. The xpath I copied from the specific button is the following:
xpath = '//*[#id="pagination"]/div/div[1]/a[4]'
So when I use driver.find_element_by_xpath(xpath).click() it will bring me to the next page.
To click through multiple pages I would like to find the specified element based on the condition that the text is equal to the correct page.
For this I tried the following conditional xpath:
xpath = //*[#id="pagination"]/div/div[1]/a[text()='page_num']
where page_num is the specific page I want to click on.
Example:
for the follwing element I would use the xpath:
element = 2
xpath = //*[#id="pagination"]/div/div[1]/a[text()='2']
I would expect that selenium would click to the specified page but instead I get an Error message that the xpath doesn't exist.
What should be the correct conditional xpath name?

How can I find Selenium XPath for the element placed in input tag?

I'm trying to find Selenium Xpath for element in the input tag. But, I'm not getting the value. I have used following code :
WebDriverWait(driver,
10).until(EC.presence_of_element_located((By.TAG_NAME("input"))))
searchBox = driver.findElement(By.tagName("input"))
But, it is not working!!!

You can use a more specific finder like By.ID this will look for an id on the web page since id is unique this should work better.
The id of the search box is input_0 you can look it up via the inspect element from your browser, or via the development tools usually hidden under the F12 key
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID("input_0"))))
searchBox = driver.find_element_by_id('input_0')
Have a read over here here for more information about locators.
Let me know if it helped and if you learned something today :)

Here is the Answer to your Question:
I don't see any issue with your line of code or locator. But you may consider trying the line of code below along with a few points:
The URL https://paytm.com/shop takes a bit of time to load properly, so we need to wait for the HTML DOM to be completely loaded. Using ExplicitWait is the right choice.
You can consider increasing the time limit from 10 to 20 seconds.
Instead of TAG_NAME locator consider using XPATH locator.
Your line of code will look like:
search_box = WebDriverWait(browser, 20).until(
EC.presence_of_element_located((By.XPATH, "//input[#id='input_0']"))
)
search_box.click()
Let me know if this Answers your Question.

Web Scraping using Selenium with two drop down menus

I am trying to make a web scraper that cycles through two drop down menus, but I cannot seem to locate the first drop down box using selenium. I was to cycle through all the names, and years in the drop down box and export a table of all pages and values to a csv. The webpage is :http://surge.srcc.lsu.edu/s1.html
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
driver.get("http://surge.srcc.lsu.edu/s1.html")
element = driver.find_element_by_xpath('//select[#id="storm_name"]')
all_options = element.find_elements_by_tag_name("option")
My error is:
NoSuchElementException: Unable to locate element:
{"method":"xpath","selector":"//select[#id=\"storm_name\"]"}

So because of #HumphreyTriscuit I will post this as an answer. Please take the time to mark it as an answer in case it was.
So there you go:
Its because you are not loading the right webpage to find your element. Your webpage contains an iFrame that points to the actual content but Selenium I believe is not able to pre-load iFrames. Get this page and your code should work.
And also a little bonus right here: If you are unsure about your xPath you should consider using xPath Checker for Firefox.