We Keep Coding

How to Fetch href links in Chromedriver?

I am trying to scrape the link from a button. If I click the button, it opens a new tab and I can't navigate in it. So I thought I'd scrape the link, go to it via webdriver.get(link) and do it that way since this will be a background program. I cannot find any tutorials on this using the most recent version of selenium. This is in Python
I tried using
wd.find_element("xpath", 'xpath here')
but that just scrapes the button title. Is there a different tag I should be using?
I've also tried just clicking the button but that opens a new tab and I don't know how to navigate on it, since it doesn't work by default and I'm still fairly new to Chromedriver.
I can't use beautifulsoup to my knowledge, since the webpage must be logged in.

You need to get the href attribute of the button. If your code gets the right button you can just use
button.get_attribute("href")
Of course if you get redirected using Javascript this is a different story, but since you didn't specify I will assume my answer works

You can use swith_of function to manage multiple windows(tabs) in same test case session
driver.switch_to.window(name_or_handler)
An extra information: If you want to get attribute value from element, you can use get_attribute() function
link_value = driver.find_element(By, selector).get_attribute("href")
P.S: example code written in Python. If you use another language, you can use equivalent Selenium functions for them.

How can I click "invisible" reCAPTCHA buttons using Selenium web automation?

I am using Python and Selenium to automate this website: https://prenotami.esteri.it
The script I made fills out a form and then clicks a button to advance to the next page. These actions are carried out using Selenium's find_element_by_xpath() function. Recently, the website added a reCAPTCHA that pops up after the button is clicked, and must be completed before advancing.
I have already written a Python script that is capable of surpassing this type of captchas by using the audio option. However, in this particular website, I am not able to find the xpath to the audio button of the reCAPTCHA. Although there is an iframe that contains the reCAPTCHA, there seems not to be anything inside it.
In the first attached image you can see how this website's reCAPTCHA looks like in HTML, compared to other website that is visible in the second image, where a #document can be seen inside the iframe.
My intention is to run this program using headless Chrome, so I can't relay in any mouse control functions offered by pyautogui for example.
I've been scratching my head around this problem for a while, so any advice is useful. Thanks!
Edit: after some research I have found that this type of reCAPTCHA that doesn't need to check a "I am not a robot" checkbox is called "invisible reCAPTCHA". The captcha only pops up if the detected activity is suspicious (for example clicking too fast). I have tried adding random waits and movements to mimic human behaviour, but the captcha still appears after some tries. Since I don't think there is a way to avoid the captcha from appearing 100% of the times, the question of how to click the buttons using Selenium's find_element_by_xpath() function remains the same. Leaving this as a note just in case someone finds it useful.

Ever tried to use the following function:
add_argument("-auto-open-devtools-for-tabs")
I managed to interact with captcha

If the position is always fixed, you can use PyAutoGUI to move the mouse and click on it
import pyautogui
pyautogui.click(100, 100) # button coordinates

Since, it is in iframe, we need to move our selenium pointing to iframe and then use your xpath.
driver.switch_to.frame("c-la7g7xqfbit4")
capchaBtn = driver.find_element_by_xpath("(//button[#id='recaptcha-audio-button'])[2]")

Is there a way to get HTML data from an active browser (Python)

I'm just messing around with python automation and attempting to create a Cookie Clicker bot.
I was wondering if you can get HTML data (e.g ID's Names, etc) from an already open browser,
and use them into locating where to click, if you should click, etc. Don't know if it's a dumb question, but I'm a beginner so help would be appreciated.
*I have tried selenium, but the point is to be able to control my main chrome browser

I'd check out stuff like Selenium and robotic process automation for that. Try this: https://realpython.com/modern-web-automation-with-python-and-selenium/

How to automate chrome extension that interacts with the parent webpage?

I want to automate the testing for a chrome-extension.
I have used selenium-python to automate the parent web-page , but i am unable to use selenium to automate the chrome-extension because selenium is bound to the web-page view,so i am not able to click the chrome extension icon present in the title bar of the google chrome. Due to this limitation, i tried using sikuli to click on the chrome-extension icon ,but i couldn't get the DOM of the chrome-extension popup HTML that appear when the extension's icon is clicked, as i had to access the text displayed by the extension's pop up html page and then evaluated it.My test failed.
I thought i could use shift_to_window() but in vain,because its a pop up html, and hence i cant shift.If instead i use sikuli ,as the test cases increased , the amount of pictures needed to automate it will also increase, so it won't be feasible.
Sample Screenshot,
In this picture,the web-store page is the parent page and the small popup in the right top corner,is the pop-up i am talking about, that appear after clicking the chrome extension . So basically i have to interact with that popup HTML.

I didn't quite understand why you can't automate whatever webpage you have with Selenium and what is the limitation you are describing. Perhaps an example such as screenshot or even a link to the relevant webpage can help. Furthermore, I don't understand why do being unable to extract DOM from a page is a limitation while using Sikuli. Sikuli has nothing to gain from the DOM, it purely based on visual content of the screen.
I would attempt to handle the popup you are describing using using standard Selenium functionality so something like this:
driver.switchTo().alert().dismiss();
There are additional alert handling options so you can choose what suits you most.

Python Selenium -- Searching for a link but finding cards

Using Python + Selenium to create a web crawler/scraper to notify me when new homework is posted. Managed to log into the main website, but you need to click a link to select your course.
After searching through the HTML manually, I found this information about the link I usually click (The blue box is the link).
However, no button that seems clickable. So I searched the page for the link I knew it should redirect me to, and I found this:
It looks like a card, which is a new data structure/object for me. How can I use an automated web crawler to click this link?

Try the following:
ui.WebDriverWait(self.driver, timeout).until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".title.ellipsis")))
driver.find_element_by_css_selector(".title.ellipsis").click()
Hope it helps you!