I want to use a tbselenium package for browser automation but however when I try to run the code I get the following error TBDriverPortError: SOCKS port 9050 is not listening.
following is the code snippet that I have used.
import unittest
from time import sleep
from tbselenium.tbdriver import TorBrowserDriver
import tbselenium.common as cm
class TestSite(unittest.TestCase):
def setUp(self):
# Point the path to the tor-browser_en-US directory in your system
tbpath = r'C:\Users\Sachin\Desktop\Tor Browsernew'
self.driver = TorBrowserDriver(tbpath, tbb_logfile_path='test.log')
self.url = "https://check.torproject.org"
def tearDown(self):
# We want the browser to close at the end of each test.
self.driver.close()
def test_available(self):
self.driver.load_url(self.url)
# Find the element for success
element = self.driver.find_element_by_class_name('on')
self.assertEqual(str.strip(element.text),
"Congratulations. This browser is configured to use Tor.")
sleep(2) # So that we can see the page
if __name__ == '__main__':
unittest.main()
can any one help me solve this error have been struggling for days
You need to uncomment ControlPort, HashedControlPassword and CookieAuthentication from Torrc file.
you can get torrc from here
then reload tor services.
if you are facing issues post full trackback in your Question
Related
Following this example: https://opensource.com/article/18/4/metrics-monitoring-and-python
We are using Python to try and create a healthcheck app that scrapes a webpage and tests for a successful login. We then wish to send the state of the healthcheck to Prometheus, using the prometheus_client package.
We have app.py which is basically the Flask api endpoint where Prometheus will scrape metrics. We also came up with the idea of spawning a subprocess to run our infinite test loop.
from flask import Flask, Response
from middleware import setup_metrics
import prometheus_client
import subprocess
CONTENT_TYPE_LATEST = str('text/plain; version=0.0.4; charset=utf-8')
app = Flask(__name__)
setup_metrics(app)
subprocess.Popen(['python', "/usr/src/tasks/selenium_login.py"])
#app.route('/metrics')
def metrics():
return Response(
prometheus_client.generate_latest(),
mimetype=CONTENT_TYPE_LATEST)
if __name__ == '__main__':
# Start up the server to expose the metrics.
app.run(host='0.0.0.0', debug=True) # local host
We have middleware.py which calculates and reports metrics
from flask import request
from prometheus_client import Counter, Histogram
import time
import sys
HEALTHCHECK_STATE = Enum(
'my_task_state',
'Description of enum: healthcheck state',
states=['starting', 'running', 'stopped'])
def set_state_to_running():
HEALTHCHECK_STATE.state('running')
def set_state_to_stopped():
HEALTHCHECK_STATE.state('stopped')
def setup_metrics(app):
app.after_request(post_state)
We have selenium_login.py which runs the test login script and either passes or exceptions out.
#!/usr/bin/env python
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
import base64
from middleware import set_state_to_running, set_state_to_stopped
def login():
time.sleep(20)
try:
driver = webdriver.Chrome()
driver.get('webpage.com')
# do login stuff
driver.quit()
set_state_to_running()
except:
set_state_to_stopped()
if __name__ == "__main__":
while True:
login()
Problem we are trying to solve is passing the state from the selenium_login.py file to middleware.py so that the metric updates as the infinite login loop runs and either passes or fails. So far hitting the metrics endpoint only yields us:
my_task_state{my_task_state="starting"} 1.0
my_task_state{my_task_state="running"} 0.0
my_task_state{my_task_state="stopped"} 0.0
We would expect running to be a "1.0". Can someone help us with the logic here?
You don't need to run the login as a separate script. Try to run the test on a separate threat but still within the same script:
def infinite_loop():
while True:
login()
time.sleep(20)
thread = Thread(target=infinite_loop,
args=(),
kwargs={})
thread.daemon = True
thread.start()
This way you will have access to all the elements you define on your main script
I am trying to test a Flask web app within a docker container, which is new for me. My stack is the following:
firefox
selenium
pytest-selenium
pytest-flask
Here is my Flask app file:
from flask import Flask
def create_app():
app = Flask(__name__)
return app
app = create_app()
#app.route('/')
def index():
return render_template('index.html')
Now, my test file which verifies the title of my index page:
import pytest
from app import create_app
# from https://github.com/pytest-dev/pytest-selenium/issues/135
#pytest.fixture
def firefox_options(request, firefox_options):
firefox_options.add_argument('--headless')
return firefox_options
# from https://pytest-flask.readthedocs.io/en/latest/tutorial.html#step-2-configure
#pytest.fixture
def app():
app = create_app()
return app
# from https://pytest-flask.readthedocs.io/en/latest/features.html#start-live-server-start-live-server-automatically-default
#pytest.mark.usefixtures('live_server')
class TestLiveServer:
def test_homepage(self, selenium):
selenium.get('http://0.0.0.0:5000')
h1 = selenium.find_element_by_tag_name('h1')
assert h1 == 'title'
When I run my tests with:
pytest --driver Firefox --driver-path /usr/local/bin/firefox test_app.py
I get the following error (which seems due to firefox not in headless mode).
selenium.common.exceptions.WebDriverException: Message: Service /usr/local/bin/firefox unexpectedly exited. Status code was: 1
Error: no DISPLAY environment variable specified
I am able to run firefox --headless but it seems my pytest fixture didn't manage to do the setup. Is there a better way to do this?
Now, if I replace selenium.get() by urlopen just to try the correct initialization of the app and its connection:
def test_homepage(self):
res = urlopen('http://0.0.0.0:5000')
assert b'OK' in res.read()
assert res.code == 200
I get the error:
urllib.error.URLError:
Do I need to boot the live server differently? Or should I change my host + port config somewhere?
Regarding the problem with a direct call with urllib:
Pytest's live server uses random port by default. You can add this parameter to pytest invocation:
--live-server-port 5000
Or without this parameter you can make direct calls to live server like:
import pytest
import requests
from flask import url_for
#pytest.mark.usefixtures('live_server')
def test_something():
r = requests.get(url_for('index', _external=True))
assert r.status_code == 200
I suppose you have view function called index. It would add a correct port number automatically.
But this doesn't have to do anything with docker, how do you run it?
Regarding the problem with Selenium itself - I can imagine docker networks related problem. How do you use it? Do you have eg. docker-compose configuration? Can you share it?
The referenced pytest-selenium issue has:
#pytest.fixture
def firefox_options(firefox_options, pytestconfig):
if pytestconfig.getoption('headless'):
firefox_options.add_argument('-headless')
return firefox_options
Note the - (single dash) preceding headless in add_argument()
(Source)
For late comers, it might be worthwhile taking a look at Xvfb and even more helpful can be this tutorial
Then (in Linux shell) you can enter:
Xvfb :99 &
export DISPLAY=:99
pytest --driver Firefox --driver-path /usr/local/bin/firefox test_app.py
This provides a virtual frame buffer (fake screen) for the application and it outputs all the graphical content there.
Note that I did not encountered this problem, just providing a solution that helped me overcome the mentioned error with an another app.
I learned Django functional test from the TDD with Python and adjust to my project.
My FT is really simple, check the title of url.
I use live_server_url to test by selenium.
But it goes to another port number(56458), not 8000.
(When I follow the book, it wasn't)
$ python manage.py runserver &
...
Starting development server at http://127.0.0.1:8000/
...
$ python manage.py test functional_test
...
http://localhost:56458
E
======================================================================
...
My functional_tests/tests.py is :
from django.test import LiveServerTestCase
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import WebDriverException
from time import time, sleep
class NewVistorTest(LiveServerTestCase):
def setUp(self):
self.browser = webdriver.Firefox()
self.browser.implicitly_wait(3)
def tearDown(self):
self.browser.quit()
def test_render_a_list_of_candiates_in_home(self):
self.browser.get(self.live_server_url)
h1_text = self.browser.find_element_by_tag_name('h1').text
self.assertEqual(self.browser.title, 'Voting Dapp')
self.assertEqual(h1_text, 'A Simple Voting Application')
Doc says:
The live server listens on localhost and binds to port 0 which uses a free port assigned by the operating system. The server’s URL can be accessed with self.live_server_url during the tests.
So I try to look listening ports(I think I'm immature for this part):
$ netstat | grep LISTEN
$ # nothing printed!
You use a LiveServerTestCase. It launches a Django server for you. No need to start a server like you did in the previous chapter.
LiveServerTestCase does basically the same as TransactionTestCase with one extra feature: it launches a live Django server in the background on setup, and shuts it down on teardown. This allows the use of automated test clients other than the Django dummy client such as, for example, the Selenium client, to execute a series of functional tests inside a browser and simulate a real user’s actions.
https://docs.djangoproject.com/en/2.0/topics/testing/tools/#django.test.LiveServerTestCase
So it is expected that your test server has some other port than the development server. Also the test server is an empty project with an empty db. So your test needs to create the needed content before you execute the actual test case.
Alternatively you can point to tests to some other environment with --liveserver LIVESERVER. See python manage.py test -h.
I think it is wrong to test against the development server as this data can be altered (manually and by previous tests) and therefore is not reproducible. I believe tests should be entirely self contained, such that it can be run either in isolation or in arbitrary combination with any number of other test cases.
from selenium import webdriver
from django.urls import reverse
import time
class TestApiPages(StaticLiveServerTestCase):
def setUp(self):
self.browser = webdriver.Chrome('functional_test/chromedriver.exe')
def tearDown(self):
self.browser.close()
def test_api_list_displayed(self):
self.browser.get(('%s%s' % (self.live_server_url, '/admin/')))
I got stuck same problem like you. Same exeception.
So what actually problem is my home page url. Where my home page url is like : http://127.0.0.1:8000/api/ but server is trying with http://localhost:56458/.
self.browser.get(('%s%s' % (self.live_server_url, '/api/')))
I am also reading the TDD Python book. I am using Django 2.1, instead of Django 1.11.
I faced the same problem as you described. I found out in setUpClass(), have to call super().setUpClass().
#classmethod
def setUpClass(cls):
super().setUpClass()
Similar in tearDownClass().
I am trying to use tor browser, and get a new IP address each URL I visit in python. I am able to open an instance of selenium running the tor browser, but how can I request a new IP every website I visit?
binary = '/Applications/TorBrowser.app/Contents/MacOS/firefox'
if os.path.exists(binary) is False:
raise ValueError("The binary path to Tor firefox does not exist.")
firefox_binary = FirefoxBinary(binary)
browser = None
def get_browser(binary=None):
browser = webdriver.Firefox(firefox_binary=binary)
return browser
if __name__ == "__main__":
browser = get_browser(binary=firefox_binary)
urls = (
('tor browser check', 'https://check.torproject.org/'),
('ip checker', 'http://icanhazip.com')
)
for url_name, url in urls:
print "getting", url_name, "at", url
browser.get(url)
To use Python to request a new IP for every request, you need to open a connection to the ControlPort and issue a NEWNYM signal.
You can use Stem to simplify the connection and commands:
from stem.control import Controller
from stem import Signal
if __name__ == '__main__':
with Controller.from_port(port = 9051) as controller:
controller.authenticate('password') # provide the password here if you set one
controller.signal(Signal.NEWNYM) # switch to clean circuits
Keep in mind Tor may rate limit NEWNYM requests so you may need to wait a short while (default 10 seconds) before issuing that command. Also, due to the limited number of exit nodes, your circuits might get the same exit node depending on how many requests you are issuing.
You need to issue this command every time you want to get a new IP (switch circuits).
I have a Django project for which I'm trying to write browser interaction tests with Selenium. My goal is to have the tests automated from Hudson/Jenkins. So far I'm able to get the test hitting the Django server, but from the server logs I see it's hitting the url /selenium-server/driver instead of the right path.
Here's my code (based on what was generated by the Selenium IDE plugin for Firefox:
from selenium import selenium
class AccountAdminPageTests(unittest.TestCase):
def setUp(self):
self.selenium = selenium("localhost",
8000,
"*chrome",
"http://localhost:8000/")
self.selenium.start()
self.selenium.open("/")
def test_ok(self):
self.assertTrue(self.selenium.is_text_present('OK'))
def tearDown(self):
self.selenium.stop()
if __name__ == "__main__":
unittest.main()
Any clues?
Never seen the exact error, but I think that Selenium is trying to connect to your app rather than the selenium Server ( a .jar file).
Port of the selenium server should be the first argument to selenium()
That should default to port 4444, you probably have to start it with
$ java -jar selenium-server.jar
FWIW here's how I got selenium tests running on a CI server...
from multiprocessing import Process
from django.test import TestCase
from selenium import selenium
class SeleniumFixtureCase(TestCase):
"""
Wrapper to multiprocess localhost server and selenium instance on one
test run.
"""
def setUp(self):
"Make the selenium connection"
TestCase.setUp(self)
self.server = Process(target=serve)
self.server.start()
self.verificationErrors = []
self.selenium = selenium("localhost", 4444, "*firefox",
"http://localhost:8000/")
self.selenium.start()
def tearDown(self):
"Kill processes"
TestCase.tearDown(self)
self.server.terminate()
self.selenium.stop()
self.assertEqual([], self.verificationErrors)
def _login(self):
"Login as Albert Camus"
self.selenium.open("http://localhost:8000/admin/")
self.selenium.wait_for_page_to_load("30000")
self.selenium.type("id_username", "albert")
self.selenium.type("id_password", "albert")
self.selenium.click("//input[#value='Log in']")
self.selenium.wait_for_page_to_load("30000")
a co-worker and myself created some automated selenium tests using django and selenium 2. It works without having to use the jar files. Here's a link to the code that shows our test cases.
We currently run django tests successfully from Jenkins using django-jenkins:
https://github.com/kmmbvnr/django-jenkins
FWIW nowadays django provides support for Selenium in the form of LiveServerTestCase:
https://docs.djangoproject.com/en/1.4/topics/testing/#django.test.LiveServerTestCase
LiveServerTestCase launches a django server which allows clients like Selenium to connect to it.
Furthermore, you can now use PhantomJs as a web driver for headless testing. This makes CI integration way easier.
The second argument of the selenium() call is supposed to be the Selenium server port number (as written in David's answer), not the tested application's port number. The default port number is 4444. I would replace the call with :
self.selenium = selenium("localhost", 4444, ....
For automating Selenium tests I would definitely use a CI solution like Jenkins. You can configure Jenkins to pull your code repository and trigger the Selenium tests from your server. I have been using Pytest for doing so from Jenkins.
You can find a step by step tutorial of configuring Jenkins with Github and Selenium here: http://www.6020peaks.com/2015/12/how-to-build-a-test-automation-solution-for-your-web-projects/