The following works when run in the context of a flask app executed in Jenkins:
A function is imported from one file to another:
def return_countries(db):
db.session.expire_on_commit = False
q = db.session.query(Country.countrycode).distinct().all()
q = [t[0] for t in q]
return q
Inside the file where it is imported I call it like this:
print(return_countries(db))
When this job is run in Jenkins, it works, no problems.
The thing is, I want one database call and then to use the returned list of country codes multiple times, not unnecessary keep querying the same thing. To keep it neat have tried to put it in a class. Yes, it could be done without the class but it may be needed in several parts of the code base, maybe with additional methods so would be cleaner if the class would work.
So I commented out that function definition and inserted the following class into the one file:
class checkCountries:
def __init__(self, db):
db.session.expire_on_commit = False
__q = db.session.query(Country.countrycode).distinct().all()
__q = [t[0] for t in __q]
print(__q)
def check(self, __q, countrycode):
if countrycode in __q:
return True
else:
return False
...updated the import statement in the other file and instantiated like this directly under the import statements:
cc = checkCountries(db)
and replaced the above print statement with this one:
print('check for DE: ', cc.check('DE'))
print('check for ES: ', cc.check('ES'))
print('check for GB: ', cc.check('GB'))
But it generated an error in the Jenkins log. The traceback is very long but below are what are probably the relevant excerpts.
Any ideas? Have a feeling something is wrong with what I have done with __q
/python3.6/site-packages/sqlalchemy/util/_collections.py", line 988, in __call__
15:26:01 return self.registry[key]
15:26:01 KeyError: 123456789123456
15:26:01
15:26:01 During handling of the above exception, another exception occurred:
15:26:01
15:26:01 Traceback (most recent call last):
/python3.6/site-packages/flask_sqlalchemy/__init__.py", line 922, in get_app
15:26:01 raise RuntimeError('application not registered on db '
15:26:01 RuntimeError: application not registered on db instance and no application bound to current context
15:26:02 Build step 'Execute shell' marked build as failure
15:26:02 Finished: FAILURE
I got it to work, so here is the class I am using which now works:
class checkCountries:
def __init__(self, db, app):
db.init_app(app)
with app.app_context():
db.session.expire_on_commit = False
self.__q = db.session.query(Country.countrycode).distinct().all()
self.__q = [t[0] for t in self.__q]
print(self.__q)
def check(self, countrycode):
if countrycode in self.__q:
return True
else:
return False
Then, the line which instantiates it now takes 2 arguments:
cc = checkCountries(db, app)
The rest is the same as in the question above.
This answer from #mark_hildreth helped, thanks!
Yes, there were more than one thing wrong with it. The class wasn't quite right but debugged that in a simplified app and got it working but this error message specific to sqlalchemy was still there.
Maybe it had something to do with the variables in scope in the class vs in the function that stopped it binding to the Flask application context
Related
I've got a weird behaviour when I raise an exception inside my Django application. Please loot at this snippet (I removed all the unnecessary code):
#csrf_exempt
def main(request):
ems_db = EmsDatabase()
# raise AssertionError
return HttpResponse('OK\n', content_type='text/plain')
this is the EmsDatabase class:
class EmsDatabase:
def __init__(self):
pass
def __del__(self):
print('>>>>>>>>>>>>>>>> DEL')
running this function (obviously through a proper http call) the EmsDatabase class is instantiated e properly garbage-collected; I see the print output in the Django server log.
But if I uncomment the raise AssertionError line I got no print output and the object is still alive; just modifying a source file to trigger the server reload makes the object to lose the reference to itself and be garbage-collected (the print line appears).
The same thing happens running Django through Lighttpd + Gunicorn.
Why is Django (v2.0.7, python 3.6, Linux) keeping a reference to my object or, more likely, to the frame of the main() function? What can I do?
I'm writing integration tests for an Alexa app.
Our application uses a controller-request-response pattern. The controller receives a request with a specified intent and session variables, routes the request to functions that do some computation with the session variables, and returns a response object with the results of that computation.
We get the right behavior from UnhandledIntentTestCase as far as test_for_smoke is concerned. However, test_returning_reprompt_text
never fires, because returns_reprompt_text is never overwritten.
Can someone explain how I can overwrite it in the parent class and/or
how the correct intent name is passed to the request object in setUpClass?
intent_base_case.py
import unittest
import mycity.intents.intent_constants as intent_constants
import mycity.mycity_controller as mcc
import mycity.mycity_request_data_model as req
import mycity.test.test_constants as test_constants
###############################################################################
# TestCase parent class for all intent TestCases, which are integration tests #
# to see if any changes in codebase have broken response-request model. #
# #
# NOTE: Assumes that address has already been set. #
###############################################################################
class IntentBaseCase(unittest.TestCase):
__test__ = False
intent_to_test = None
returns_reprompt_text = False
#classmethod
def setUpClass(cls):
cls.controller = mcc.MyCityController()
cls.request = req.MyCityRequestDataModel()
key = intent_constants.CURRENT_ADDRESS_KEY
cls.request._session_attributes[key] = "46 Everdean St"
cls.request.intent_name = cls.intent_to_test
cls.response = cls.controller.on_intent(cls.request)
#classmethod
def tearDownClass(cls):
cls.controller = None
cls.request = None
def test_for_smoke(self):
self.assertNotIn("Uh oh", self.response.output_speech)
self.assertNotIn("Error", self.response.output_speech)
def test_correct_intent_card_title(self):
self.assertEqual(self.intent_to_test, self.response.card_title)
#unittest.skipIf(not returns_reprompt_text,
"{} shouldn't return a reprompt text".format(intent_to_test))
def test_returning_reprompt_text(self):
self.assertIsNotNone(self.response.reprompt_text)
#unittest.skipIf(returns_reprompt_text,
"{} should return a reprompt text".format(intent_to_test))
def test_returning_no_reprompt_text(self):
self.assertIsNone(self.response.reprompt_text)
test_unhandled_intent.py
import mycity.test.intent_base_case as base_case
########################################
# TestCase class for unhandled intents #
########################################
class UnhandledIntentTestCase(base_case.IntentBaseCase):
__test__ = True
intent_to_test = "UnhandledIntent"
returns_reprompt_text = True
output
======================================================================
FAIL: test_correct_intent_card_title (mycity.test.test_unhandled_intent.UnhandledIntentTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/wdrew/projects/alexa_311/my_city/mycity/mycity/test/intent_base_case.py", line 44, in test_correct_intent_card_title
self.assertEqual(self.intent_to_test, self.response.card_title)
AssertionError: 'UnhandledIntent' != 'Unhandled intent'
- UnhandledIntent
? ^
+ Unhandled intent
? ^^
======================================================================
FAIL: test_returning_no_reprompt_text (mycity.test.test_unhandled_intent.UnhandledIntentTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/wdrew/projects/alexa_311/my_city/mycity/mycity/test/intent_base_case.py", line 56, in test_returning_no_reprompt_text
self.assertIsNone(self.response.reprompt_text)
AssertionError: 'So, what can I help you with today?' is not None
----------------------------------------------------------------------
This is because of execution order. The SkipIf decorators are executed once during the parsing of the IntentBaseCase class. They aren't re-executed for each class or for each call to the test function.
The decorator pattern for SkipIf is designed for use with fixed global variables such as versions of dependent modules, operating system or some other external resource who's availability can be calculated or known in the global context.
Skipping tests is also something that should be done for external reasons, not for internal ones such as the needs of a sub-class. A skip is still a kind of failing test which is indicated in the report so you can see your test suite isn't exercising the whole of the functional scope of the project.
You should redesign your base class structure so functions are only available to run if the sub-class and skip using Skip for this. My recommendation would be:
class IntentBaseCase(unittest.TestCase):
...
class RepromptBaseCase(IntentBaseCase):
def test_returning_reprompt_text(self):
self.assertIsNotNone(self.response.reprompt_text)
class NoRepromptBaseCase(IntentBaseCase):
def test_returning_no_reprompt_text(self):
self.assertIsNone(self.response.reprompt_text)
You should also consider moving the response portion out of the setUp and put it into a test_ function of it's own and change these test_returning functions into a simpler assertReprompt and assertNoReprompt functions. It's a good idea to set up the tests in setUp, but not a good idea to run the actual code there.
I want to save all the traceback of all the exceptions from every failed test I run to an external file.
Instead of using try and except in every test i would like to use the tearDown of unittest so it will be more generic.
something like:
import traceback
import unittest
class SomeTestCase(unittest.TestCase):
def setUp(self):
pass
def test_some_test(self):
self.assertTrue(False)
def tearDown(self):
with open(logger.txt, 'a') as doc:
doc.write(traceback.format_exc())
the problem is that the exception that you get in the test_some_test cant be called with traceback in the teardown (the tracebacke return None)
any suggestion?
So after I went over a big piece of the unittest code I found a solution!
The TestCase class has an _outcome attribute.
The _outcome has a list type attribute called errors.
errors[0][1] is a tuple that is exactly like the sys.exc_info() output that contains the traceback.
So now that I have this tuple that I can use in traceback.format_exception() just like traceback.format_exc() uses it and problem solved.
def tearDown(self):
try:
etype, value, tb = self._outcome.errors[0][1]
trace = ''.join(traceback.format_exception(etype=etype, value=value, tb=tb, limit=None))
date = '{date}\n'.format(date=str(datetime.datetime.now()))
name = '\n' + self._testMethodName + '-\n'
with open(self.logger, 'a') as doc:
doc.write(name + date + trace)
except:
pass
Below is just example code that provides the error I am hoping to get help on fixing, or getting an understanding of a better way to write this. I have a mysql "super" class called mysql_connection. In this class, the connection to the database is made. I also have a few methods within it. One that simply runs "select version()" to show that the connection/query works. I then have a "chktable" method which in this example instantiates a new subclass called "table" which inherits the super class. After instantiating the class, I then call a method within the subclass which attempts to use the the query method in the superclass to run "show tables like 'tbl name'". This is where I get an error.
import mysql.connector
from mysql.connector import errorcode
from mysql.connector.cursor import MySQLCursor
class mysql_connection(object):
def __init__(self, **kwargs):
self.connection_options = {}
self.connection_options['user'] = 'root'
self.connection_options['password'] = ''
self.connection_options['host'] = '192.168.33.10'
self.connection_options['port'] = '3306'
self.connection_options['database'] = "test"
self.connection_options['raise_on_warnings'] = True
self.connect()
def connect(self):
try:
self.cnx = mysql.connector.connect(**self.connection_options)
except mysql.connector.Error as err:
if err.errno == errorcode.ER_ACCESS_DENIED_ERROR:
print "Something is wrong with your user name or password"
elif err.errno == errorcode.ER_BAD_DB_ERROR:
print "Database does not exists"
else:
print err
def query(self, statement, data=''):
cursor = MySQLCursor(self.cnx)
cursor.execute(statement)
result = cursor.fetchall()
cursor.close
return result
def get_version(self):
print self.query("select version()")
def chktable(self, tb_name):
tab = table(name=tb_name)
tab.check_table()
class table(mysql_connection):
def __init__(self, **kwargs):
self.name = kwargs['name']
def check_table(self):
return super(table, self).query("show tables like '{}".format(self.name))
conn = mysql_connection()
conn.get_version()
conn.chktable("test")
The error that I get is:
$ python example.py
[(u'5.1.73',)]
Traceback (most recent call last):
File "example.py", line 50, in <module>
conn.chktable("test")
File "example.py", line 39, in chktable
tab.check_table()
File "example.py", line 46, in check_table
return super(table, self).query("show tables like '{}".format(self.name))
File "example.py", line 28, in query
cursor = MySQLCursor(self.cnx)
AttributeError: 'table' object has no attribute 'cnx'
I don't fully understand calling back to the super class and how it works with subclasses, so that is likely my issue. I'd also like to know if there is maybe a better way to accomplish this. I was thinking I could get rid of the subclass altogether, but I like the subclass I made so I feel there should be a way around it. A secondary thing I could try would be to put the subclass inside the master class, but I don't think that is correct.
As Jon points out, this is not an appropriate use of inheritance. That is for "is-a" relationships: ie Dog inherits from Animal, because a dog is an animal. But a table is not a connection: a table might use a connection, but that simply means you should assign an instance of the connection to an instance variable in table.
Also, in an inheritance relationship there is usually no good reason for a superclass to know about its subclasses, as you have in the chktable method.
(The actual bug you're seeing is because you haven't called the superclass method in table's __init__, but it's better to fix your structure.)
gives me an error message:
class logger:
session = web.ctx.session #this line
doesn't give me an error message:
class create:
def GET(self):
# loggedout()
session = web.ctx.session #this line
form = self.createform()
return render.create(form)
Why?
web.ctx can't be used in that scope. It's a thread-local object that web.py initializes before it calls GET/POST/etc. and gets discarded afterwards.
class logger:
print('Hi')
prints Hi. Statements under a class definition gets run at definition time.
A function definition like this one:
def GET(self):
# loggedout()
session = web.ctx.session #this line
form = self.createform()
return render.create(form)
is also a statement. It creates the function object which is named GET. But the code inside the function does not get run until the GET method is called.
This is why you get an error message in the first case, but not the second.