trying to do this on Instagram. there is code before it like importing the libraries and chrome driver path and logging in..so below is the loop that is failing...what's your suggestion? it works when I put in a list of exact URLs..so there is definitely something wrong in the loop
users=['instagramuser1','instagramuser2','instagramuser3']
user=-1
for user in users:
user+=1
webdriver.get('https://www.instagram.com/'+str(users)+'/')
sleep(5)
webdriver.find_element_by_css_selector('the_amazing_css_path').click()
I think what you want is:
users=['instagramuser1','instagramuser2','instagramuser3']
for user in users:
webdriver.get('https://www.instagram.com/' + user + '/')
sleep(5)
webdriver.find_element_by_css_selector('the_amazing_css_path').click()
Note user instead of users in the URL string.
I assume you use exact code posted in the question.
so you did not follow python indent rule in this particular code
it sould be fix like this:
users=['instagramuser1','instagramuser2','instagramuser3']
for user in users:
webdriver.get('https://www.instagram.com/'+user+'/')
sleep(5)
webdriver.find_element_by_css_selector('the_amazing_css_path').click()
Related
Description:
I am trying to make a job ad parser which works on the indeed.com site (I am using python + selenium + chromedriver)
I am able to login with my facebook credentials and then, I am redirected to the default site which is hu.indeed.com (as I am living in Hungary).
I would like to search for jobs available in London, therefore get selenium driver to change to the uk.indeed.com site.
Then I get selenium to locate and input my job search criteria in the position input field and the locality as well in the locality field. Up untill now everything works smoothly.
The problem:
After pressing the search button I am able to see the results window, but after a very short time I am automatically redirected to the hu.indeed.com site. As you can see from my code below, I have no such commands, I have no clue whatsoever why and how this is happening. My print statements show that driver.current_url changes at a moment in time and I dont understand why is that happening and how could I prevent that.
Could you please let me know why does the url change and how could I prevent that?
Code:
driver.get("https://uk.indeed.com/")
time.sleep(1)
job_type_input=driver.find_element_by_xpath('//*[#id="text-input-what"]')
search_text=f"{jobs[0]} {extra_info}"
job_type_input.send_keys(search_text)
time.sleep(1)
print(f"1 print:{driver.current_url}") #<--- 1. print
job_location_input=driver.find_element_by_xpath('//*[#id="text-input-where"]')
job_location_input.send_keys(cities[0])
search_button=driver.find_element_by_xpath('//*[#id="jobsearch"]/button')
search_button.click()
time.sleep(5)
print(f"2 print:{driver.current_url}") #<--- 2. print
print(f"3 print:{driver.current_url}") #<--- 3. print
try:
moaic_element=driver.find_element_by_id("mosaic-provider-jobcards")
html=mosaic_element.get_attribute('innerHTML')
print("success")
except:
print("error in try")
print(f"4 print:{driver.current_url}") #<--- 4. print
Output:
1 print:https://uk.indeed.com/
2 print:https://hu.indeed.com/
3 print:https://hu.indeed.com/
error in try
4 print:https://hu.indeed.com/
I am the one who wrote the original post and found I found the solution to this problem. As Max Daroshchanka mentioned in his answer, the problem was claused by indeed.com as it reloaded due to some plugin (or something). Therefore my solution was to use the input field only after some time passed (using time.sleep(2))
So I have a selenium script that will automatically enter a series of numbers into a website, and the website will redirect the user to another website based off if the numbers match a PIN. However, the browser takes a short time to redirect the user, in which the next line of code would have already run and returned an error.
I was thinking something like this would work but it doesn't, I don't know why.
def checkElement():
try:
xpath = '//*[#id="name"]'
print("Page is ready!")
except TimeoutException:
print("failed")
checkElement()
I believe that you are looking for WebDriverWait. You can add specific condition in it. Please find the sample code below.
first_result = wait.until(presence_of_element_located((By.XPATH, "//*[#id='name']")))
So, my goal was to write a script, that scrapes users, that used a specific hashtag on Instagram and writes their accounts into a .txt-file and it mostly works!
My problem is, that even though some accounts posted plural pictures, my script does show each name only once. Any idea, how it might be able to kind of count them or get my script to not delete doublets?
I looked for everything but can't find a solution.
This is my part of writing code:
def generate_initial_information_txt(initial_information):
initial_information_txt = open("initial_information", "w+")
for user in initial_information:
initial_information_txt.write(user + "\n")
This is the part to find the name:
for user in range(30):
el = self.driver.find_element_by_xpath('/html/body/div[4]/div[2]/div/article/header/div[2]/div[1]/div[1]')
el = el.find_element_by_tag_name('a')
time.sleep(2)
profile = el.get_attribute('href')
open_recent_posts_set.add(profile)
time.sleep(2)
next_button = self.driver.find_element_by_xpath('/html/body/div[4]/div[1]/div/div/a[2]')
next_button.click()
time.sleep(2)
THE URL would be
https://instagram.com/explore/tags/hansaviertel_ms
So I'm starting to scrape the "Recent" Posts and e.g. the "Hansaforum" posted like 5 of the first 6. If I insert a range of 6 it just throws out a .txt-file with two accounts, not 5 times the "Hansaforum". And I'd like to get the amount of times in any kind of way. –
Thanks :)
This is from the book "automate the boring stuff with python".
At first I made a .bat file and ran it with arguments from cmd, didnt open any pages in chrome, looked up on here, changed up the code, still it executes perfectly and prints the print line but it doesnt open tabs as it should.
What am I doing wrong? Thanks in advance
#! python3
# lucky.py opens several google search matches
import requests,sys,webbrowser,bs4
searchTerm1 = 'python'
print('Googling...')
res = requests.get('https://www.google.com/search?={0}'.format(searchTerm1))
res.raise_for_status()
#retrieve top search result links
soup = bs4.BeautifulSoup(res.text,"html.parser")
#open a browser tab for each result.
linkElems = soup.select('.r a')
numOpen = min(5,len(linkElems))
for i in range(numOpen):
webbrowser.open('http://google.com' + linkElems[i].get('href'))
The short answer is that your URL is not returning results. Here's a URL that provides results: https://www.google.com/search?q=python.
I changed the one line in your code to use this template: "https://www.google.com/search?q={0} and I saw linkElems was non-trivial.
In short, webbrowser is not opening any pages because numOpen is 0, so the for loop tries to iterate over 0 items, which results in the code within that for loop block (webbrowser.open) to not get executed.
The longer, more detailed explanation of why the numOpen = 0 is due to a redirect that occurs with the initial GET request given your custom Google query. See this answer for how to circumvent these issues as there are numerous ways- the easiest is probably to use the Google search API.
As a result of the redirect, your BeautifulSoup search will not return any successful results, causing the numOpen variable to be set to 0 as there will be no list elements. As there are no list elements, the for loop does not execute.
You can debug things like this on your own the quick and dirty, but not perfect, way by simply adding print statements throughout the script to see which print statements fail to execute as well as looking at the variables and their returned values.
As an aside, the shebag should also be set to #!/usr/bin/env python3 rather than simply #! python3. Reference here.
Hope this helps
I am trying to type a float number into a textbox with default value 0.00.But it tries to get appended instead of overwriting it.I tried with .clear() and then send_keys('123.00') but still it gets appended.
Then i tried with send_keys(Keys.CONTROL+'a','123.00').It updates 0.00 only.
Any help is really appreciated.
For more info ..
URL : http://new.ossmoketest.appspot.com
userid: senthil.arumugam#mycompanyname.com -- mycompanyname = orangescape (sorry to avoid spam mails)
password not needed now.
click purchaseorder... in the form please new product and new price... sample application for automation.. thanks
I've had good results with:
from selenium.webdriver.common.keys import Keys
element.send_keys(Keys.CONTROL, 'a')
element.send_keys('123.00')
If that doesn't work it may have something to do with the code in the web page.
Unless you have custom editbox, click() should work for you:
from selenium.webdriver import Firefox
b = Firefox()
b.get('http://google.com')
e = b.find_element_by_id('lst-ib')
e.click() # is optional, but makes sure the focus is on editbox.
e.send_keys('12.34')
e.get_attribute('value')
# outputs: u'12.34'
e.click()
e.clear()
e.get_attribute('value')
# outputs: u''
e.send_keys('56.78')
e.get_attribute('value')
# outputs: u'56.78'
I just found the clear() command - see here:
If this element is a text entry element, this will clear the value. Has no effect on other elements. Text entry elements are INPUT and TEXTAREA elements.
EDIT:
So your approach would be:
element.clear();
element.sendKeys('123.00');
I've experienced issues with all the examples given in other answers.
el.send_keys(Keys.CONTROL + 'a' + Keys.NULL, 'your string')
Has worked in all the projects I've worked in, so much I've wrapped it into my own implementation of the Webdriver class with more robust operations.