Related
I am trying to add screenshots of failed tests to html reports generated by pytest-html plugin,pytest libraries.I followed this How do I include screenshot in python pytest html report. However i always ended up with the below error.
Error occuring:
feature_request = item.funcargs['request']
KeyError: 'request'
conftest.py
import pytest
from datetime import datetime
from selenium import webdriver
import pytest_html
#pytest.hookimpl(hookwrapper=True)
def pytest_runtest_makereport(item, call):
timestamp = datetime.now().strftime('%H-%M-%S')
pytest_html = item.config.pluginmanager.getplugin('html')
outcome = yield
report = outcome.get_result()
extra = getattr(report, 'extra', [])
if report.when == 'call':
feature_request = item.funcargs['request']
driver = feature_request.getfixturevalue('browser')
driver.save_screenshot('Screenshots/scr' + timestamp + '.png')
extra.append(pytest_html.extras.image('Screenshots/scr' + timestamp + '.png'))
extra.append(pytest_html.extras.url('http://www.example.com/'))
xfail = hasattr(report, 'wasxfail')
if (report.skipped and xfail) or (report.failed and not xfail):
# only add additional html on failure
extra.append(pytest_html.extras.image('Screenshots/scr.png'))
extra.append(pytest_html.extras.html('<div>Additional HTML</div>'))
report.extra = extra
#pytest.fixture(scope='session')
def setup():
print('Starting')
url = 'https://www.google.com/'
driver = webdriver.Firefox()
driver.get(url)
return driver
test_google.py
class Test_One():
def test_login(setup):
print('Opend google')
assert False
Whats wrong with my code ? how can i append screenshot on failed test cases ?
you should have a fixture with request argument on your conftest.py, specifically your driver.
#pytest.fixture(scope='session')
def setup(request):
then use driver = feature_request.getfixturevalue('setup') #referencing your def setup
I'm working on an automation project. I am automating the login page. There is a problem with the login page in the website that I'm working on, i.e, it fails the first time I log in, even if the credentials are correct, and then successfully logs in if I log in the second time. So, to overcome this I have included a try except for method. The code:
driver=webdriver.Chrome("C:\\temp\\chromedriver.exe")
url = 'site-address'
driver.get(url)
driver.maximize_window()
wait = WebDriverWait(driver, 120)
class Demo
def func1()
user=driver.find_element_by_xpath("//")
user.send_keys('Name')
pas = driver.find_element_by_xpath("//")
password.send_keys('pas')
btn = driver .find_element_by_xpath("//")
btn.click()
try:
chk=wait.until(EC.presence_of_element_located((By.ID, 'lop')))
assert check.text == "Some Text"
print('PAGE VERIFIED : ', check.text)
except:
print('LOGIN UN-SUCCESSFUL')
print('Re-Trying...')
user=driver.find_element_by_xpath("//")
user.send_keys('Name')
pas = driver.find_element_by_xpath("//")
pas.send_keys('pas')
btn = driver .find_element_by_xpath("//")
btn.click()
So, as you can see I am repeating the process of entering username, password and click twice, once in the beginning and then the same process in the except block. My question is, how to create a method, call it setup() such that I can initialize the driver, URL, username, password in setup(). And then call the method in func1() and then in the except block. I'm new to Python, not familiar how to return the values.
Python classes have a constructor called __init__ which you can use to pass information to when creating an object of this class. Something like this, even if the separation of logic is not the best. But gives you an idea of the constructor.
class Demo:
def __init__(self, driver, url, username, password):
""" Constructor of our demo class, set local variables"""
self.driver = driver
self.username = username
self.password = password
self.driver.get(url)
self.driver.maximize_window()
self.wait = WebDriverWait(self.driver, 120)
def func1(self, attempt=1):
""" Find elements in the opened page and send username and pasword,
then clikc the button (submit?) """
user = self.driver.find_element_by_xpath("//")
user.send_keys(self.username)
pas = driver.find_element_by_xpath("//")
pas.send_keys(self.password)
btn = driver .find_element_by_xpath("//")
btn.click()
try:
# Wait for element
chk = self.wait.until(EC.presence_of_element_located((By.ID, 'lop')))
assert chk.text == "Some Text"
print('PAGE VERIFIED : ', chk.text)
except:
# General exception ..
if attempt >= 3: # Start with 1.. so 1, 2, 3.
print("FAILED THREE TIMES, GIVING UP")
else:
print('LOGIN UN-SUCCESSFUL')
print('Re-Trying...')
self.func1(attempt + 1)
if __name__ == '__main__':
driver = webdriver.Chrome("C:\\temp\\chromedriver.exe")
url = 'site-address'
TEST = Demo(driver=driver, url=url, username='Name', password='pas')
TEST.func1()
I have a conftest file to handle the setup and tear down of selenium drivers when running tests in pytest. I'm trying to add in a command line option to determine if I run the local built in selenium and web drivers or a remote selenium server and drivers etc...
I've added a commandline option called "runenv" and I'm trying to obtain the string value from this entered through command line to determine if the system should run local or remote webdriver configuration. This allows testers to develop on their own machines locally, but also means we can script the tests to run on the remote machine as part of a build pipeline.
The issue I have is that my parser.addoption shown in the below file is not processed. It doesn't seem return a value (whether it's the default or the value passed through command line) that I can use.
My conftest.py file is as follows (*note the url and remote IP are just samples to cover company privacy)
#conftest.py
import pytest
import os
import rootdir_ref
import webdriverwrapper
from webdriverwrapper import DesiredCapabilities, FirefoxProfile
#when running tests from command line we should be able to pass --url=www..... for a different website, check what order these definitions need to be in
def pytest_addoption(parser):
parser.addoption("--url", action="store", default="https://mydomain1.com.au")
parser.addoption("--runenv", action="store", default="local")
#pytest.fixture(scope='session')
def url(request):
return request.config.option.url
#pytest.fixture(scope='session')
def runenv(request):
return request.config.option.runenv
BROWSERS = {}
if runenv == 'remote':
BROWSERS = {'chrome_remote': DesiredCapabilities.CHROME}
else:
BROWSERS = {'chrome': DesiredCapabilities.CHROME}
# BROWSERS = {
# #'firefox': DesiredCapabilities.FIREFOX,
# # 'chrome': DesiredCapabilities.CHROME,
# 'chrome_remote': DesiredCapabilities.CHROME,
# # 'firefox_remote': DesiredCapabilities.FIREFOX
# }
#pytest.fixture(scope='function', params=BROWSERS.keys())
def browser(request):
if request.param == 'firefox':
firefox_capabilities = BROWSERS[request.param]
firefox_capabilities['marionette'] = True
firefox_capabilities['acceptInsecureCerts'] = True
theRootDir = os.path.dirname(rootdir_ref.__file__)
ffProfilePath = os.path.join(theRootDir, 'DriversAndTools', 'FirefoxSeleniumProfile')
geckoDriverPath = os.path.join(theRootDir, 'DriversAndTools', 'geckodriver.exe')
profile = FirefoxProfile(profile_directory=ffProfilePath)
# Testing with local Firefox Beta 56
binary = 'C:\\Program Files\\Mozilla Firefox\\firefox.exe'
b = webdriverwrapper.Firefox(firefox_binary=binary, firefox_profile=profile, capabilities=firefox_capabilities,
executable_path=geckoDriverPath)
elif request.param == 'chrome':
desired_cap = BROWSERS[request.param]
desired_cap['chromeOptions'] = {}
desired_cap['chromeOptions']['args'] = ['--disable-plugins', '--disable-extensions']
desired_cap['browserName'] = 'chrome'
desired_cap['javascriptEnabled'] = True
theRootDir = os.path.dirname(rootdir_ref.__file__)
chromeDriverPath = os.path.join(theRootDir, 'DriversAndTools', 'chromedriver.exe')
b = webdriverwrapper.Chrome(chromeDriverPath, desired_capabilities=desired_cap)
elif request.param == 'chrome_remote':
desired_cap = BROWSERS[request.param]
desired_cap['chromeOptions'] = {}
desired_cap['chromeOptions']['args'] = ['--disable-plugins', '--disable-extensions']
desired_cap['browserName'] = 'chrome'
desired_cap['javascriptEnabled'] = True
b = webdriverwrapper.Remote(command_executor='http://192.168.1.1:4444/wd/hub', desired_capabilities=desired_cap)
elif request.param == 'firefox_remote':
firefox_capabilities = BROWSERS[request.param]
firefox_capabilities['marionette'] = True
firefox_capabilities['acceptInsecureCerts'] = True
firefox_capabilities['browserName'] = 'firefox'
firefox_capabilities['javascriptEnabled'] = True
theRootDir = os.path.dirname(rootdir_ref.__file__)
ffProfilePath = os.path.join(theRootDir, 'DriversAndTools', 'FirefoxSeleniumProfile')
profile = FirefoxProfile(profile_directory=ffProfilePath)
b = webdriverwrapper.Remote(command_executor='http://192.168.1.1:4444/wd/hub',
desired_capabilities=firefox_capabilities, browser_profile=profile)
else:
b = BROWSERS[request.param]()
request.addfinalizer(lambda *args: b.quit())
return b
#pytest.fixture(scope='function')
def driver(browser, url):
driver = browser
driver.set_window_size(1260, 1080)
driver.get(url)
return driver
My tests would simply utilise the resulting "driver" fixture after the page has already been setup by conftest. Example test maybe:
import pytest
from testtools import login, dashboard, calendar_helper, csvreadtool, credentials_helper
import time
#pytest.mark.usefixtures("driver")
def test_new_appointment(driver):
testId = 'Calendar01'
credentials_list = credentials_helper.get_csv_data('LoginDetails.csv', testId)
# login
assert driver.title == 'Patient Management cloud solution'
rslt = login.login_user(driver, credentials_list)
.... etc..
I'd then like to run the test suite using a command like:
python -m pytest -v --html=.\Results\testrunX.html --self-contained-html --url=https://myotherdomain.com.au/ --runenv=chrome_remote
So far the url command line option works, I can use it to override the url or let it use the default.
But I can not get a value from the runenv commandline option. In the if statement below it will always default to the else statement. runenv doesn't seem to have a value, even though the default I have for that parser.addoption is 'local'
if runenv == 'remote':
BROWSERS = {'chrome_remote': DesiredCapabilities.CHROME}
else:
BROWSERS = {'chrome': DesiredCapabilities.CHROME}
I tried putting in pdb.trace() before the if statement so I could see what is in the runenv, but it will only tell me it's a function and I don't seem to be able to obtain a value from it, which makes me think it's not getting populated at all.
I'm not really sure how to debug the conftest file because the output generally does not appear in the console output. Any suggestions? Does the pytest_addoption actually accept 2 or more custom command line arguments?
I'm using
Python 3.5.3
Pytest 3.2.1
In a VirtualEnv on windows 10
Here, why are you making url and runenv as fixture? You can use it like below:
In your conftest.py
def pytest_addoption(parser):
parser.addoption('--url', action='store', default='https://mytestdomain.com.au/', help='target machine url')
parser.addoption('--runenv', action='store', default='remote', help='select remote or local')
def pytest_configure(config):
os.environ["url"] = config.getoption('url')
os.environ["runenv"] = config.getoption('runenv')
Now, wherever you want to access url and runenv you just need to write os.getenv('Variable_name') like,
#pytest.fixture(scope='function')
def driver(browser):
driver = browser
driver.set_window_size(1260, 1080)
driver.get(os.getenv('url'))
return driver
Or like in your code,
if os.getenv('runenv')== 'remote':
BROWSERS = {'chrome_remote': DesiredCapabilities.CHROME}
else:
BROWSERS = {'chrome': DesiredCapabilities.CHROME}
Here, url and runenv would be saved in OS environment variable and you can access it anywhere without fixture just by os.getenv()
Hope it would help you !!
BROWSERS are populated at conftest.py import and at import time runenv is a function. If you want to use runenv as a fixture BROWSERS also must be a fixture:
#pytest.fixture(scope='session')
def BROWSERS(runenv):
if runenv == 'remote':
return {'chrome_remote': DesiredCapabilities.CHROME}
else:
return {'chrome': DesiredCapabilities.CHROME}
Okay so after doing a proof of concept it looks like the main part of my problem is that I can't use a command line option to change the output of a function (either a fixture or non fixture function) and then use that as a list for a dynamic paramater on another request fixture function. After reading up about it, it would seem that it's something to do with the order of processing during loading of the fixture functions. I pretty much tried everything apart from playing with the metafunc .
I tried using all of the various pytest.mark.fixture or any variations in the params = section and they would simply not yield an iterable list (on some occasions I could get it to give me the entire list but not iterate over it)
I also tried the lazyfixture model with no success.
I tried using dictionaries in the a fixture function as the out put. I tried them outside the function, I tried the same thing with classes and creating populated objected inside a fixture function . I tried using the pytest.mark.getfixturevalue in the params = , I tried using the pytest.mark.use decorator, I tried the paramatize decorator. None of of it works.
It looks like the only thing that may work here would be the alternative proposed solution here, that has not actually been developed.
https://docs.pytest.org/en/latest/proposals/parametrize_with_fixtures.html
In the end I decided to contain all of the logic one large fixture function, which seems to work for moment, but not ideally how I'd like to do it because unfortunately I can not have variable params based on command line entries for which browsers I want to test. I have to manually update the conftest file to chance whether I not I run one or 2 browsers and have it iterate through both for each test.
# conftest.py
import pytest
import os
import rootdir_ref
import webdriverwrapper
from webdriverwrapper import DesiredCapabilities, FirefoxProfile
# when running tests from command line we should be able to pass --url=www..... for a different website, check what order these definitions need to be in
def pytest_addoption(parser):
parser.addoption('--url', action='store', default='https://mytestdomain.com.au/', help='target machine url')
parser.addoption('--runenv', action='store', default='remote', help='select remote or local')
#pytest.fixture(scope='session')
def url(request):
return request.config.getoption('url')
#pytest.fixture(scope='session')
def runenv(request):
return request.config.getoption('runenv')
BROWSERS = {
# 'firefox': DesiredCapabilities.FIREFOX,
'chrome': DesiredCapabilities.CHROME
}
#pytest.fixture(scope='function', params=BROWSERS.keys())
def browser(request, runenv):
if request.param == 'firefox':
if runenv == 'local':
firefox_capabilities = BROWSERS[request.param]
firefox_capabilities['marionette'] = True
firefox_capabilities['acceptInsecureCerts'] = True
theRootDir = os.path.dirname(rootdir_ref.__file__)
ffProfilePath = os.path.join(theRootDir, 'DriversAndTools', 'FirefoxSeleniumProfile')
geckoDriverPath = os.path.join(theRootDir, 'DriversAndTools', 'geckodriver.exe')
profile = FirefoxProfile(profile_directory=ffProfilePath)
# Testing with local Firefox Beta 56
binary = 'C:\\Program Files\\Mozilla Firefox\\firefox.exe'
b = webdriverwrapper.Firefox(firefox_binary=binary, firefox_profile=profile, capabilities=firefox_capabilities,
executable_path=geckoDriverPath)
elif runenv == 'remote':
request.param == 'firefox_remote'
firefox_capabilities = BROWSERS[request.param]
firefox_capabilities['marionette'] = True
firefox_capabilities['acceptInsecureCerts'] = True
firefox_capabilities['browserName'] = 'firefox'
firefox_capabilities['javascriptEnabled'] = True
theRootDir = os.path.dirname(rootdir_ref.__file__)
ffProfilePath = os.path.join(theRootDir, 'DriversAndTools', 'FirefoxSeleniumProfile')
profile = FirefoxProfile(profile_directory=ffProfilePath)
b = webdriverwrapper.Remote(command_executor='https://selenium.mytestserver.com.au/wd/hub',
desired_capabilities=firefox_capabilities, browser_profile=profile)
else:
b = webdriverwrapper.Firefox()
elif request.param == 'chrome':
if runenv == 'local':
desired_cap = BROWSERS[request.param]
desired_cap['chromeOptions'] = {}
desired_cap['chromeOptions']['args'] = ['--disable-plugins', '--disable-extensions']
desired_cap['browserName'] = 'chrome'
desired_cap['javascriptEnabled'] = True
theRootDir = os.path.dirname(rootdir_ref.__file__)
chromeDriverPath = os.path.join(theRootDir, 'DriversAndTools', 'chromedriver.exe')
b = webdriverwrapper.Chrome(chromeDriverPath, desired_capabilities=desired_cap)
elif runenv == 'remote':
desired_cap = BROWSERS[request.param]
desired_cap['chromeOptions'] = {}
desired_cap['chromeOptions']['args'] = ['--disable-plugins', '--disable-extensions']
desired_cap['browserName'] = 'chrome'
desired_cap['javascriptEnabled'] = True
b = webdriverwrapper.Remote(command_executor='https://selenium.mytestserver.com.au/wd/hub',
desired_capabilities=desired_cap)
else:
b = webdriverwrapper.Chrome()
else:
b = webdriverwrapper.Chrome()
request.addfinalizer(lambda *args: b.quit())
return b
#pytest.fixture(scope='function')
def driver(browser, url):
driver = browser
driver.set_window_size(1260, 1080)
driver.get(url)
return driver
Requirement: Need to open the two instance of chrome, one in normal mode and another in incognito mode. Load the same URL lets say facebook.com on both instance. And try to login with different credential.
I am able to open two instances in normal and incognito mode, but when setting the values it get written to same page. Below is the code:
MyTest script:
#calling the driver
self.driverObj1 = WebAction().get_private_browser_driver(browser)
self.driverObj2 = WebAction().get_browser_driver(browser)
#Opening the browser and load the URL
WebAction().explicit_load_url_in_browser(self.driverObj1, self.url, self.elementforwait)
WebAction().explicit_load_url_in_browser(self.driverObj2, self.url,self.elementforwait)
#Locating the User Input
page1_loginuser = WebModel().get_element_by_xpath(self.driverObj1, self.inputxpath['logintext'])
page1_loginpass = WebModel().get_element_by_xpath(self.driverObj1, self.inputxpath['loginpass'])
page1_loginbutton = WebModel().get_element_by_xpath(self.driverObj1, self.inputxpath['loginbutton'])
page2_loginuser = WebModel().get_element_by_xpath(self.driverObj1, self.inputxpath['logintext'])
page2_loginpass = WebModel().get_element_by_xpath(self.driverObj1, self.inputxpath['loginpass'])
page2_loginbutton = WebModel().get_element_by_xpath(self.driverObj1, self.inputxpath['loginbutton'])
#filling the details and click on button
page1_loginuser.send_keys(self.logininput1['username'])
page1_loginpass.send_keys(self.logininput1['pass'])
page1_loginbutton.click()
page2_loginuser.send_keys(self.logininput2['username'])
page2_loginpass.send_keys(self.logininput2['pass'])
page2_loginbutton.click()
My class WebModel has two methods
# Method - to get chrome driver
def get_chrome_driver(self):
path_of_driver = self.get_chrome_driver_path()
self.chromeDriver = webdriver.Chrome(path_of_driver)
return self.chromeDriver
# Method - to get chrome incognito
def get_private_chrome_driver(self):
path_of_driver = self.get_chrome_driver_path()
chrome_option = webdriver.ChromeOptions()
chrome_option.add_argument("--incognito")
self.chromeDriver = webdriver.Chrome(path_of_driver, chrome_options=chrome_option)
return self.chromeDriver
class WebAction has two methods
def get_browser_driver(self, browserName):
try:
if browserName == 'chrome':
self.browserDriver = self.webmodelObj.get_chrome_driver()
elif browserName == 'firefox':
self.browserDriver = self.webmodelObj.get_firfox_driver()
elif browserName == 'ie':
self.browserDriver = self.webmodelObj.get_ie_driver()
else:
self.browserDriver = self.webmodelObj.get_safari_driver()
return self.browserDriver
except:
print "FAIL: UNABLE TO CONNECT WITH %s"%browserName
return False
def get_private_browser_driver(self,browserName):
self.browserDriver = False
try:
if browserName == 'chrome':
self.browserDriver = self.webmodelObj.get_private_chrome_driver()
return self.browserDriver
except:
print "Fail: Unable to Connect with %s" %(browserName)
return self.browserDriver
#Locating the User Input
page2_loginuser = WebModel().get_element_by_xpath(self.driverObj1, self.inputxpath['logintext'])
page2_loginpass = WebModel().get_element_by_xpath(self.driverObj1, self.inputxpath['loginpass'])
page2_loginbutton = WebModel().get_element_by_xpath(self.driverObj1, self.inputxpath['loginbutton'])
You are using self.driverObj1 to find page2_login elements.
I have tried to correct the following error which basically is about the arguments for 'username' and 'directory'. I have tried all possible ways but no luck. Python does not state the line of code that the following error refers to:
usage: Google_Map.py [-h] [-n NUM_TO_DOWNLOAD] [-l LOG_LEVEL]
username directory
Google_Map.py: error: the following arguments are required: username, directory
Please see the code here:
def __init__(self, username, directory, num_to_download = 10,
log_level='info'):
self.username = username
self.profile_url = self.get_url(username)
self.directory = directory
self.PAUSE = 1
self.user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
self.headers = {'User-Agent': self.user_agent}
self.html_source = None
self.log_level = getattr(logging, log_level.upper())
self.setup_logging(self.log_level)
self.set_num_posts(num_to_download)
self.setup_webdriver()
def get_url(self, path):
return urlparse.urljoin('https://instagram.com', path)
def set_num_posts(self, num_to_download=None):
self.num_posts = int(self.get_posts_count(self.profile_url) or 0)
self.num_to_download = num_to_download
def setup_logging(self, level=logging.INFO):
self.logger = logging.getLogger('instaraider')
self.logger.addHandler(logging.StreamHandler())
self.logger.setLevel(logging.INFO)
def log(self, *strings, **kwargs):
level = kwargs.pop('level', logging.INFO)
self.logger.log(level, u' '.join(str(s) for s in strings))
def setup_webdriver(self):
self.profile = webdriver.FirefoxProfile()
self.profile.set_preference("general.useragent.override", self.user_agent)
self.webdriver = webdriver.Firefox(self.profile)
self.webdriver.set_window_size(480, 320)
self.webdriver.set_window_position(800, 0)
def get_posts_count(self, url):
"""
Given a url to Instagram profile, return number of photos posted
"""
response = requests.get(url)
counts_code = re.search(r'\"media":{"count":\d+', response.text)
if not counts_code:
return None
return re.findall(r'\d+', counts_code.group())[0]
def log_in_user(self):
driver = self.webdriver
self.log('You need to login to access this profile.',
'Redirecting you to the login page in the browser.',
level=logging.WARN)
driver.get(self.get_url('accounts/login/'))
# Wait until user has been successfully logged in and redirceted
# to his/her feed.
WebDriverWait(driver, 60).until(
expected_conditions.presence_of_element_located(
(By.CSS_SELECTOR, '.-cx-PRIVATE-FeedPage__feed'),
)
)
self.log('User successfully logged in.', level=logging.INFO)
self.set_num_posts() # Have to set this again
driver.get(self.profile_url)
def load_instagram(self):
"""
Using Selenium WebDriver, load Instagram page to get page source
"""
self.log(self.username, 'has', self.num_posts, 'posts on Instagram.')
if self.num_to_download is not None:
self.log("The first", self.num_to_download, "of them will be downloaded.")
num_to_download = self.num_to_download or self.num_posts
driver = self.webdriver
# load Instagram profile and wait for PAUSE
self.log("Loading Instagram profile...")
driver.get(self.profile_url)
driver.implicitly_wait(self.PAUSE)
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
try:
el = driver.find_element_by_css_selector(
'.-cx-PRIVATE-ProfilePage__advisoryMessageHeader'
)
except NoSuchElementException:
pass
else:
self.log_in_user()
if (num_to_download > 24):
scroll_to_bottom = self.get_scroll_count(num_to_download)
element = driver.find_element_by_css_selector('div.-cx-PRIVATE-AutoloadingPostsGrid__moreLoadingIndicator a')
driver.implicitly_wait(self.PAUSE)
element.click()
for y in range(int(scroll_to_bottom)):
self.scroll_page(driver)
# After load all profile photos, retur, source to download_photos()
time.sleep(1)
source = driver.page_source
# close Firefox window
driver.close()
return source
def scroll_page(self, driver):
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
time.sleep(0.2)
driver.execute_script("window.scrollTo(0, 0);")
def get_scroll_count(self, count):
return (int(count) - 24) / 12 + 1
def validate(self):
"""
returns True if Instagram username is valid
"""
req = requests.get(self.profile_url)
try:
req.raise_for_status()
except:
self.log('User', self.username, 'is not valid.',
level=logging.ERROR)
return False
if not self.num_posts:
self.log('User', self.username, 'has no photos to download.',
level=logging.ERROR)
return False
return True
def save_photo(self, photo_url, photo_name):
image_request = requests.get(photo_url, headers=self.headers)
image_data = image_request.content
with open(photo_name, 'wb') as fp:
fp.write(image_data)
if "last-modified" in image_request.headers:
modtime = calendar.timegm(eut.parsedate(image_request.headers["last-modified"]))
os.utime(photo_name, (modtime, modtime))
def download_photos(self):
"""
Given source code for loaded Instagram page,
extract all hrefs and download full-resolution photos
source: HTML source code of Instagram profile papge
"""
num_to_download = self.num_to_download or self.num_posts
if self.html_source is None:
self.html_source = self.load_instagram()
# check if directory exists, if not, make it
if not op.exists(self.directory):
os.makedirs(self.directory)
# index for progress bar
photos_saved = 0
self.log("Saving photos to", self.directory)
links = re.findall(r'src="[https]+:...[\/\w \.-]*..[\/\w \.-]*..[\/\w \.-]*..[\/\w \.-].jpg', self.html_source)
for link in links:
photo_url = link[5:]
photo_url = photo_url.replace('\\', '')
photo_url = re.sub(r'/s\d+x\d+/', '/', photo_url)
split = urlparse.urlsplit(photo_url)
photo_name = op.join(self.directory, split.path.split("/")[-1])
# save full-resolution photo if its new
if not op.isfile(photo_name):
self.save_photo(photo_url, photo_name)
photos_saved += 1
self.log('Downloaded file {}/{} ({}).'.format(
photos_saved, num_to_download, op.basename(photo_name)))
else:
self.log('Skipping file', photo_name, 'as it already exists.')
if photos_saved >= num_to_download:
break
self.log('Saved', photos_saved, 'files to', self.directory)
def main():
# parse arguments
parser = argparse.ArgumentParser(description='InstaRaider')
parser.add_argument('username', help='Instagram username')
parser.add_argument('directory', help='Where to save the images')
parser.add_argument('-n', '--num-to-download',
help='Number of posts to download', type=int)
parser.add_argument('-l', '--log-level', help="Log level", default='info')
args = parser.parse_args()
username = args.username
directory = op.expanduser(args.directory)
raider = InstaRaider(username, directory,
num_to_download=args.num_to_download,
log_level=args.log_level)
if not raider.validate():
return
raider.download_photos()
if __name__ == '__main__':
main()
That's not a Python exception message. That's command line help output.
The output is generated by the argparse module, configured here:
def main():
# parse arguments
parser = argparse.ArgumentParser(description='InstaRaider')
parser.add_argument('username', help='Instagram username')
parser.add_argument('directory', help='Where to save the images')
parser.add_argument('-n', '--num-to-download',
help='Number of posts to download', type=int)
parser.add_argument('-l', '--log-level', help="Log level", default='info')
args = parser.parse_args()
The moment parser.parse_args() is called your command line arguments are parsed to match the above configuration.
Specifically, the username and directory positional arguments are required:
parser.add_argument('username', help='Instagram username')
parser.add_argument('directory', help='Where to save the images')
You'll need to specify these on the command line when you run the script:
Google_Map.py some_instagram_username /path/to/directory/to/save/images
The other command line options are optional and start with - or --.
If you can't run this from a console or terminal command line, you could pass in the options to the parser directly:
def main(argv=None):
# parse arguments
parser = argparse.ArgumentParser(description='InstaRaider')
parser.add_argument('username', help='Instagram username')
parser.add_argument('directory', help='Where to save the images')
parser.add_argument('-n', '--num-to-download',
help='Number of posts to download', type=int)
parser.add_argument('-l', '--log-level', help="Log level", default='info')
args = parser.parse_args(argv)
# ....
main(['some_instagram_username', '/path/to/directory/to/save/images'])
Now the arguments are passed in via the argv optional function parameter, as a list.
However, rather than have main() parse arguments, you could just use the InstaRaider() class directly:
raider = InstaRaider('some_instagram_username', '/path/to/directory/to/save/images')
raider.download_photos()