We Keep Coding

Selenium_Webdriver_Python: Send_Keys (Keys + Keys.Enter ) is not working

I'm trying to send the keys by passing a variable inside it (var name : NextDayFormatted).Because it is a string and to avoid any future error I'm using the below code-
browser.find_element_by_css_selector("...").send_keys(NextDayFormatted + Keys.Enter)
Whenever I'm running this single line of code independently, it works fine. But, when I'm running all , it fails. It is passing the value that I have assigned to my variable (NextDayFormatted) but doesn't hit ENTER due to which the next line of code is failing.
PS: This issue persists in ENTER and RETURN both.
Can you please guide me on this?
Full code:
browser.get("url")
username = "abcd"
pwd = "def"
email = browser.find_element_by_name('email')
email.send_keys(username)
password = browser.find_element_by_name('password')
password.send_keys(pwd)
browser.find_element_by_tag_name('button').click()
#Click on filter icon
browser.find_element_by_xpath('....').click()
#Click Add filter
browser.find_element_by_xpath('....').click()
#Select the fields from the dropdown - Appointment Start Date & After
browser.find_element_by_xpath('....').click()
#The file needs to be extracted on every fridays therefore apply the appointment strt date logic in the search box.
Next_Date = datetime.datetime.today() + datetime.timedelta(days = 2)
Next_Date_Formatted = next_date.strftime('%m/%d/%Y')
#Applying the above variable in the search box
browser.find_element_by_css_selector('....').send_keys(Next_Date_Formatted + Keys.ENTER)`enter code here`
#Click on submit button
browser.find_element_by_css_selector('....').click()

How to use selenium to handle and select elements from the popup/hidden form/table?

I would like to select the popup date table/calendar from the below website by using selenium. i tried to add double click function in it, but it was failed to select the date that i wanted.
from selenium.webdriver.common.action_chains import ActionChains
ccass = driver.get('http://www.hkexnews.hk/sdw/search/searchsdw_c.aspx')
ticker = '00001'
menu = driver.find_element_by_xpath("#date-picker-popup").click()
ccass_search_year = driver.find_element_by_xpath('//*[#id="date-picker"]/div[1]/b[1]/ul/li[2]/button').click()
actions.double_click(ccass_search_year)
ccass_search_month = driver.find_element_by_xpath('//*[#id="date-picker"]/div[1]/b[2]/ul/li[4]/button').click()
actions.double_click(ccass_search_month)
ccass_search_day = driver.find_element_by_xpath('//*[#id="date-picker"]/div[1]/b[3]/ul/li[4]/button').click()
actions.double_click(ccass_search_day)
ccass_search = driver.find_element_by_xpath('//*[#id="txtStockCode"]').send_keys(ticker) #Keys.ENTER)
ccass_search_click = driver.find_element_by_xpath('//*[#id="btnSearch"]').click()

The date you are trying to select is disabled. You can't select 3rd April 2018. You can only select from 10th April and I'm guessing that it will be disabled tomorrow.
Sorry to say, you are too late. Which the class name for the disabled dates also.

i edited for a little bit, the problem was that although i found all those buttons from the popup date picker, but i could not get the result of the date that i selected:
year = year_list['2019']
month = month_list['4']
day = day_list['4']
year_list = {'2018':'//*[#id="date-picker"]/div[1]/b[1]/ul/li[1]/button', '2019':'//*[#id="date-picker"]/div[1]/b[1]/ul/li[2]/button'}
month_list = {'1':'//*[#id="date-picker"]/div[1]/b[2]/ul/li[1]/button', '2':'//*[#id="date-picker"]/div[1]/b[2]/ul/li[2]/button', '3':'//*[#id="date-picker"]/div[1]/b[2]/ul/li[3]/button', '4':'//*[#id="date-picker"]/div[1]/b[2]/ul/li[4]/button', '5':'//*[#id="date-picker"]/div[1]/b[2]/ul/li[5]/button', '6':'//*[#id="date-picker"]/div[1]/b[2]/ul/li[6]/button', '7':'//*[#id="date-picker"]/div[1]/b[2]/ul/li[7]/button', '8':'//*[#id="date-picker"]/div[1]/b[2]/ul/li[8]/button', '9':'//*[#id="date-picker"]/div[1]/b[2]/ul/li[9]/button', '10':'//*[#id="date-picker"]/div[1]/b[2]/ul/li[10]/button', '11':'//*[#id="date-picker"]/div[1]/b[2]/ul/li[11]/button', '12':'//*[#id="date-picker"]/div[1]/b[2]/ul/li[12]/button'}
day_list = {'1':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[1]/button','2':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[2]/button','3':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[3]/button','4':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[4]/button','5':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[5]/button','6':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[6]/button','7':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[7]/button','8':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[8]/button','9':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[9]/button','10':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[10]/button','11':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[11]/button','12':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[12]/button','13':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[13]/button','14':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[14]/button','15':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[15]/button','16':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[16]/button','17':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[17]/button','18':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[18]/button','19':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[19]/button','20':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[20]/button','21':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[21]/button','22':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[22]/button','23':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[23]/button','24':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[24]/button','25':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[25]/button','26':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[26]/button','27':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[27]/button','28':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[28]/button','29':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[29]/button','30':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[30]/button','31':' //*[#id="date-picker"]/div[1]/b[3]/ul/li[31]/button'}
ccass = driver.get('http://www.hkexnews.hk/sdw/search/searchsdw_c.aspx')
popup_datepicker = driver.find_element_by_xpath('//*[#id="txtShareholdingDate"]').click()
ccass_search_year = driver.find_element_by_xpath(year).click()
actions.double_click(ccass_search_year)
ccass_search_month = driver.find_element_by_xpath(month).click()
actions.double_click(ccass_search_month)
ccass_search_day = driver.find_element_by_xpath(day).click()
actions.double_click(ccass_search_day)
ccass_search = driver.find_element_by_xpath('//*[#id="txtStockCode"]').send_keys(ticker) #Keys.ENTER)
ccass_search_click = driver.find_element_by_xpath('//*[#id="btnSearch"]').click()

Stop a loop if information is not found

I created a web scraping program that open several URLs, it checks which one of the URLs has information related to "tomorrow"s date and then it prints some specific information that is on that URL. My problem is that sometimes none of the URLs in that list has information concerning "tomorrow". So I would like that in such case, the program prints other innformation like "no data found". How could I accomplish that? Other doubt I have, do I need the while loop at the beginning? Thanks.
My code is:
from datetime import datetime, timedelta
tomorrow = datetime.now() + timedelta(days=1)
tomorrow = tomorrow.strftime('%d-%m-%Y')
day = ""
while day != tomorrow:
for url in list_urls:
browser.get(url)
time.sleep(1)
dia_page = browser.find_element_by_xpath("//*[#id='item2']/b").text
dia_page = dia_page[-10:]
day_uns = datetime.strptime(dia_page, "%d-%m-%Y")
day = day_uns.strftime('%d-%m-%Y')
if day == tomorrow:
meals = browser.find_elements_by_xpath("//*[#id='item2']/span")
meal_reg = browser.find_element_by_xpath("//*[#id='item_frm']/span[1]").text
sopa2 = (meals[0].text)
refeicao2 = (meals[1].text)
sobremesa2 = (meals[2].text)
print(meal_reg)
print(sopa2)
print(refeicao2)
print(sobremesa2)
break

No need for a while loop, you can use the for-else Python construct for this:
for url in list_urls:
# do stuff
if day == tomorrow:
# do and print stuff
break
else: # break never encountered
print("no data found")

How to check current date and move to next date

I'm having a python issue which I cannot seem to understand. Not sure if I need to use if statements but because I'm new to python, I'm not actually sure how to code this little issue.
Virtually this is the issue I have. For the departure calendar, I want python to be able to do the following:
View 'Your date'. If there's a flight (doesn't matter if lowfare or normal), click it. If not then move onto the next available date that does have a flight and click that.
Will need to be able to move to the next month if no date is available in the current month (I have an example code for this).
For the return calendar, I want it to do the same thing but ensure it selects a date at least 7 days after the selected departure date.
That's virtually my question, how to do that?
Below is the html of the depature calendar (return calendar is exactly the same except it's inboundsearchresults rather than outbound search results):
Below I have a sample code which works when selecting from an ordinary date picker (this is used in the page before the url) if you want to use that template and manipulate it:
# select depart date
datepicker = driver.find_element_by_id("departure-date-selector")
actions.move_to_element(datepicker).click().perform()
# find the calendar, month and year picker and the current date
calendar = driver.find_element_by_id("departureDateContainer")
month_picker = Select(calendar.find_element_by_class_name("ui-datepicker-month"))
year_picker = Select(calendar.find_element_by_class_name("ui-datepicker-year"))
current_date = calendar.find_element_by_class_name("ui-datepicker-current-day")
# printing out current date
month = month_picker.first_selected_option.text
year = year_picker.first_selected_option.text
print("Current departure date: {day} {month} {year}".format(day=current_date.text, month=month, year=year))
# see if we have an available date in this month
try:
next_available_date = current_date.find_element_by_xpath("following::td[#data-handler='selectDay' and ancestor::div/#id='departureDateContainer']")
print("Found an available departure date: {day} {month} {year}".format(day=next_available_date.text, month=month, year=year))
next_available_date.click()
except NoSuchElementException:
# looping over until the next available date found
while True:
# click next, if not found, select the next year
try:
calendar.find_element_by_class_name("ui-datepicker-next").click()
except NoSuchElementException:
# select next year
year = Select(calendar.find_element_by_class_name("ui-datepicker-year"))
year.select_by_visible_text(str(int(year.first_selected_option.text) + 1))
# reporting current processed month and year
month = Select(calendar.find_element_by_class_name("ui-datepicker-month")).first_selected_option.text
year = Select(calendar.find_element_by_class_name("ui-datepicker-year")).first_selected_option.text
print("Processing {month} {year}".format(month=month, year=year))
try:
next_available_date = calendar.find_element_by_xpath(".//td[#data-handler='selectDay']")
print("Found an available departure date: {day} {month} {year}".format(day=next_available_date.text, month=month, year=year))
next_available_date.click()
break
except NoSuchElementException:
continue

The idea is to define a reusable function - calling it select_date() that receives a "calendar" WebElement and an optional minimum date. This function would first look for the Your date in the calendar and if it is there and it is more than minimum (if given) click it and return the date. If there is no Your date, look for the available "flight" days and, if minimum date is given and the date is more than or equal to it, click it and return the date.
Working implementation:
from datetime import datetime, timedelta
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
def select_date(calendar, mininum_date=None):
try:
# check if "Your Date" is there
your_date_elm = calendar.find_element_by_class_name("your-date")
your_date = your_date_elm.get_attribute("data-date")
print("Found 'Your Date': " + your_date)
your_date_elm.click()
# check if your_date against the minimum date if given
your_date = datetime.strptime(your_date, "%Y-%m-%d")
if mininum_date and your_date < mininum_date:
raise NoSuchElementException("Minimum date violation")
return your_date
except NoSuchElementException:
flight_date = None
flight_date_elm = None
while True:
print("Processing " + calendar.find_element_by_css_selector("div.subheader > p").text)
try:
if mininum_date:
flight_date_elms = calendar.find_elements_by_class_name("flights")
flight_date_elm = next(flight_date_elm for flight_date_elm in flight_date_elms
if datetime.strptime(flight_date_elm.get_attribute("data-date"), "%Y-%m-%d") >= mininum_date)
else:
flight_date_elm = calendar.find_element_by_class_name("flights")
except (StopIteration, NoSuchElementException):
calendar.find_element_by_partial_link_text("Next month").click()
# if found - print out the date, click and exit the loop
if flight_date_elm:
flight_date = flight_date_elm.get_attribute("data-date")
print("Found 'Flight Date': " + flight_date)
flight_date_elm.click()
break
return datetime.strptime(flight_date, "%Y-%m-%d")
driver = webdriver.Firefox()
driver.get("http://www.jet2.com/cheap-flights/leeds-bradford/antalya/2016-03-01/2016-04-12?adults=2&children=2&infants=1&childages=4%2c6")
wait = WebDriverWait(driver, 10)
# get the outbound date
outbound = wait.until(EC.visibility_of_element_located((By.ID, "outboundsearchresults")))
outbound_date = select_date(outbound)
# get the inbound date
inbound = driver.find_element_by_id("inboundsearchresults")
inbound_minimum_date = outbound_date + timedelta(days=7)
inbound_date = select_date(inbound, mininum_date=inbound_minimum_date)
print(outbound_date, inbound_date)
driver.close()
For the provided in the question URL, it prints:
Processing March 2016
Found 'Flight Date': 2016-03-28
Processing April 2016
Found 'Flight Date': 2016-04-04
2016-03-28 00:00:00 2016-04-04 00:00:00
The two dates printed at the end are the departure and the return dates.
Let me know if you need any clarifications and hope it helps.

Python Google Calendar sort by date

I've got a script that grabs every event off of a Google Claendar, and the searches through those events, and prints the ones that contain a search term to a file.
The problem I'm having is that I need them to be put in order by date, and this doesn't seem to do that.
while True:
events = calendar_acc.events().list(calendarId='myCal',pageToken=page_token).execute()
for event in events['items']:
if 'Search Term' in event['summary']:
#assign names to hit, and date
find = event['summary']
date = event['start'][u'dateTime']
#only print the first ten digits of the date.
month = date[5:7]
day = date[8:10]
year = date[0:4]
formatted_date = month+"/"+day+"/"+year
#write a line
messy.write(formatted_date+" "+event['summary']+"\n\n")
I think there is a way to do this with the time module maybe, but I'm not sure. Any help is appreciated.

Just in case anyone else needs to do this. With the help of jedwards.
I ended up creating an empty list: hits
And then appending the ['start']['dateTime'] as an datetime.datetime object,
and ['summary'] to the list for each event that contained my "Search Term". Like so:
hits = []
while True:
events = calendar_acc.events().list(calendarId='My_Calendar_ID', pageToken=page_token).execute()
for event in events['items']:
if "Search Term" in event['summary']:
hits.append((dateutil.parser.parse(event['start']['dateTime']), event['summary']))
page_token = events.get('nextPageToken')
if not page_token:
break
The you just sort the list, and in my case, I cut the datetime object down to just be the date. And then wrote the whole line to a file. But this code just prints it to the console.
hits.sort()
for x in hits:
d = x[0]
date = "%d/%d/%d"%(getattr(d,'month'),getattr(d,'day'), getattr(d,'year'))
final = str(date)+"\t\t"+str(x[1])
print final
Thanks again to jedwards in the comments!

You can return a sorted list (by date ascending) from the API by using the "orderBy" parameter and setting it to "updated".
page_token = None
while True:
events = service.events().list(calendarId=myID, orderBy='updated', pageToken=page_token).execute()
for event in events['items']:
print(event)
page_token = events.get('nextPageToken')
if not page_token:
break
For more information see: https://developers.google.com/calendar/v3/reference/events/list
Hope this helps.