How to load this dynamic website with Selenium - python

Trying to load this dynamic website but failed. Anyone can help?
driver = webdriver.Chrome()
driver.get('https://apply.95559.com.cn/personbank/portals/ptQuotaQryCoins.do')

Not that super clear about the issue you are facing while loading the dynamic website. However I was able to load the website using the following code block:
Code Block:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
options = Options()
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('excludeSwitches', ['enable-logging'])
options.add_experimental_option('useAutomationExtension', False)
options.add_argument('--disable-blink-features=AutomationControlled')
s = Service('C:\\BrowserDrivers\\chromedriver.exe')
driver = webdriver.Chrome(service=s, options=options)
driver.get('https://apply.95559.com.cn/personbank/portals/ptQuotaQryCoins.do')
Browser Snapshot:

Related

Can't load a website with selenium chomedriver Python3

I'm trying to get a html from the url below.
https://reserve.tokyodisneyresort.jp/restaurant/search/?useDate=20220714&adultNum=4&childNum=0&childAgeInform=&restaurantType=4&restaurantType=5&restaurantType=3&restaurantType=1&restaurantType=2&restaurantType=7&nameCd=&wheelchairCount=0&stretcherCount=0&keyword=&reservationStatus=0
But after starting chromedriver, it continues to load all the time and nothing appears.
How can I fix it?
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.options import Options
import time
def get_html():
url = 'https://reserve.tokyodisneyresort.jp/restaurant/search/?useDate=20220714&adultNum=4&childNum=0&childAgeInform=&restaurantType=4&restaurantType=5&restaurantType=3&restaurantType=1&restaurantType=2&restaurantType=7&nameCd=&wheelchairCount=0&stretcherCount=0&keyword=&reservationStatus=0'
options = Options()
options.add_experimental_option('excludeSwitches', ['enable-logging'])
options.add_argument("--disable-blink-features")
options.add_argument("--disable-blink-features=AutomationControlled")
driver = webdriver.Chrome(
executable_path='./chromedriver.exe', chrome_options=options)
driver.implicitly_wait(60)
driver.get(url)
get_html()

How to make chrome tab executed by selenium/chromedriver to not close

How does one make the google chrome tab to not close immediately after its opened.
Could use time.sleep or use a while loop in some type of way?
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.service import Service
chrome_options = webdriver.ChromeOptions()
prefs = {"profile.default_content_setting_values.notifications": 2}
chrome_options.add_experimental_option("prefs", prefs)
chrome_options.add_experimental_option("useAutomationExtension", False)
chrome_options.add_experimental_option("excludeSwitches", ["enable-automation"])
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=chrome_options)
driver.get("https://www.google.com")
Found it myself.
Use this
chrome_options.add_experimental_option("detach", True)

Selenium: Website always opens with selenium but then the site goes completely white immediately and keeps loading forever

I try to open the following site using selenium:
https://www.honestdoor.com/
Normally this works fine with every site with the following code:
(I am currently using google-chrome version 98.0.4758 - using ChromeDriverManager for downloading the version - see below in the code)
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.support.ui import WebDriverWait
from webdriver_manager.chrome import ChromeDriverManager
from fake_useragent import UserAgent
import time
if __name__ == '__main__':
ua = UserAgent()
userAgent = ua.random
options = Options()
# options.add_argument('--headless')
options.add_experimental_option ('excludeSwitches', ['enable-logging'])
options.add_argument("start-maximized")
options.add_argument('window-size=1920x1080')
options.add_argument('--no-sandbox')
options.add_argument('--disable-gpu')
options.add_argument(f'user-agent={userAgent}')
srv=Service(ChromeDriverManager().install())
driver = webdriver.Chrome (service=srv, options=options)
waitWebDriver = WebDriverWait (driver, 10)
link = "https://www.honestdoor.com/"
# link = "https://www.bcassessment.ca/"
# driver.minimize_window() # optional
driver.get (link)
time.sleep(1000)
The site opens with selenium as allways but then the site goes immediately complete white and is still loading forever with the cicle going around in the top left corner (I can only kill the chrome-task in the task manager).
When I open the site in normal chrome or incognito chrome everything works fine - it seem to only crash when I open it with selenium. With other sites (like https://www.bcassessment.ca/ I have no problems at all and the open with selenium as allways)
Why is this not working for this particular website?
Not that super clear about the exact issue you are facing while loading the website. However I was able to load the website using the following code block:
Code Block:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
options = Options()
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('excludeSwitches', ['enable-logging'])
options.add_experimental_option('useAutomationExtension', False)
options.add_argument('--disable-blink-features=AutomationControlled')
s = Service('C:\\BrowserDrivers\\chromedriver.exe')
driver = webdriver.Chrome(service=s, options=options)
driver.get("https://www.honestdoor.com/")
Browser Snapshot:

Specific sites doesn't render using Selenium (python)

I've been using selenium with python on both Chrome and Firefox. This specific website stays blank on both browsers when I try to run it with selenium - I'd appreciate any help. Here's my code for chrome:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_experimental_option("useAutomationExtension", False)
options.add_experimental_option("excludeSwitches",["enable-automation"])
path = r'C:\Program Files (x86)\chromedriver.exe'
driver = webdriver.Chrome(executable_path=path, chrome_options=options)
driver.get('https://main.knesset.gov.il/Activity/committees/pages/allcommitteesagenda.aspx')
Add the argument --disable-blink-features=AutomationControlled
Code Block:
options = Options()
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('useAutomationExtension', False)
options.add_argument('--disable-blink-features=AutomationControlled')
driver = webdriver.Chrome(executable_path=r'C:\BrowserDrivers\chromedriver.exe', options=options)
driver.get("https://main.knesset.gov.il/Activity/committees/pages/allcommitteesagenda.aspx")
Browser Snapshot:
References
You can find a couple of relevant detailed discussion in:
How to evade blocking by Walmart using Selenium Webdriver
Selenium can't open a second page

Selenium chromedriver how disable translation bar in headless mode

When opening some web pages in Selenium using Chrome driver I found that chrome shows the translation bar in headless mode, but it doesn't show it when headless is disabled.
Example (headless enabled)
from selenium import webdriver
from selenium.webdriver.chrome.options import Options as ChromeOptions
chrome_options = ChromeOptions()
chrome_options.headless = True
chrome_options.page_load_strategy = 'normal'
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--lang=en')
chrome_options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome(options=chrome_options)
driver.get('https://blog.bellostes.com/?attachment_id=5392')
driver.save_screenshot('headless_true.png')
driver.quit()
Result:
Example (headless disabled)
from selenium import webdriver
from selenium.webdriver.chrome.options import Options as ChromeOptions
chrome_options = ChromeOptions()
chrome_options.headless = False
chrome_options.page_load_strategy = 'normal'
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--lang=en')
chrome_options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome(options=chrome_options)
driver.get('https://blog.bellostes.com/?attachment_id=5392')
driver.save_screenshot('headless_false.png')
driver.quit()
Result:
What I have already tried:
chrome_options.add_argument('--lang=en')
chrome_options.add_argument('--disable-features=TranslateUI')
chrome_options.add_argument('--disable-translate') # Not supported any more on Chrome
No difference. Is any I can do to get rid of the translation bar in headless mode?

Categories