I am new to unnitest module. I have a file that has unittest in it. The file is something like ...
File1.py
class ABC (unittest.TestCase):
def setUp(self):
# Do some work here
def test_123(self, a,b,c):
# Do some work here
if __name__ == "__main__":
unittest.main()
*Now I am calling this file from another file by passing values to the function "test_123".* But python displays the following error. Could anybody please help!
Traceback (most recent call last):
File "caller_file.py", line 20, in <module>
r = file1.ABC()
File "/usr/lib/python2.7/unittest/case.py", line 191, in __init__
(self.__class__, methodName))
ValueError: no such test method in <class 'file1.ABC'>: runTest
You can run file1.ABC test case like this:
import unittest
import file1
suite = unittest.TestLoader().loadTestsFromTestCase(file1.ABC)
unittest.TextTestRunner(verbosity=2).run(suite)
Also you need to add the self argument to the setUp and test_123 methods and self should be the sole argument.
I run into similar problems with my unittests because missing entries in search path for modules.
I solved it by creating
my_env = os.environ.copy()
if not 'PYTHONPATH' in my_env:
my_env['PYTHONPATH'] = ''
my_env['PYTHONPATH'] += ';' + ';'.join(
[os.path.abspath('.'),
os.path.abspath('..'),
os.path.abspath('..\\..')])
and then the call of the file
_ = subprocess.check_output(filepath, shell=True, env=my_env)
I just added the current path environment because the calling-file is in the same directories. Maybe you have to adjust that.
Related
I need some help because I keep getting this erros:
Traceback (most recent call last):
File "/home/kostas_ubuntu/HemTools/bin/chromHMM.py", line 173, in <module>
main()
File "/home/kostas_ubuntu/HemTools/bin/chromHMM.py", line 100, in main
args = my_args()
File "/home/kostas_ubuntu/HemTools/bin/chromHMM.py", line 43, in my_args
input.add_argument('-bin',"--chromHMM_jar", help="chromHMM bin location",default=myPars['chromHMM_jar'])
NameError: global name 'myPars' is not defined
I run the script and get this error although in the script I import the utils.py module that sets the myPars variable. The below code snippet comes from the utils.py module.
p_dir = os.path.dirname(os.path.realpath(__file__)) + "/"
username = getpass.getuser()
myData = parse_config(p_dir+"../config/data.config")
myPars = parse_config(p_dir+"../config/parameters.config")
More information of the scripts you can find on the github links below:
utils.py -> https://github.com/YichaoOU/HemTools/blob/master/utils/utils.py
chromHMM.py ->https://github.com/YichaoOU/HemTools/blob/master/bin/chromHMM.py
Thanks in advance!
In Python 3 you should declare on every new scope the global variables it uses. In your case it'd be like the following:
def my_args():
global myPars, myData
# ... The remaining code
# ...
def main():
global myData, myPipelines
# ... The remaining code
I run the following file (called test_func.py) with py.test:
import findspark
findspark.init()
from pyspark.context import SparkContext
def filtering(data):
return data.map(lambda p: modif(p)).count()
def modif(row):
row.split(",")
class Test(object):
sc = SparkContext('local[1]')
def test_filtering(self):
data = self.sc.parallelize(['1','2', ''])
assert filtering(data) == 2
And, because the modif function is used inside the map transform, it fails with the following error:
org.apache.spark.api.python.PythonException: Traceback (most recent call last):
File "/home/osboxes/spark-1.5.2-bin-hadoop2.4/python/lib/pyspark.zip/pyspark/worker.py", line 98, in main
command = pickleSer._read_with_length(infile)
File "/home/osboxes/spark-1.5.2-bin-hadoop2.4/python/lib/pyspark.zip/pyspark/serializers.py", line 164, in _read_with_length
return self.loads(obj)
File "/home/osboxes/spark-1.5.2-bin-hadoop2.4/python/lib/pyspark.zip/pyspark/serializers.py", line 422, in loads
return pickle.loads(obj)
ImportError: No module named clustering.test_func
pyspark does not manage to find the modif function. Note that the file test_func.py is in the directory clustering and I run py.test from inside clustering directory.
What astonishes me is that if I use modif function outside a map, it works fine. For instance, if I do: modif(data.first())
Any idea why I get such importErrors and how I could fix it?
EDIT
I had already tested what has been proposed Avihoo Mamka's answer, i.e. to add test_func.py to the files copied to all workers. However, it had no effect. It is no surprise for me because I think the main file, from where the spark application is created, is always sent to all workers.
I think it can come from the fact that pyspark is looking for clustering.test_func and not test_func.
The key here is the Traceback you got.
PySpark is telling you that the worker process doesn't have access to clustering.test_func.py. When you initialize the SparkContext you can pass a list of files that should be copied to the worker:
sc = SparkContext("local[1]", "App Name", pyFiles=['MyFile.py', 'lib.zip', 'app.egg'])
More information: https://spark.apache.org/docs/1.5.2/programming-guide.html
I have two files,
main.py
options.py
main.py is my main program. I import options.py via:
from options import Options
whereas "options" is a class I defined in options.py
Options has a custom init method:
def __int__(self, afc="red", awc="orange", asc="gray", apn=1738):
If I try to make an object from within main.py like so:
options = Options("red","green","blue",1738)
Python tells me that there are unexpected arguments. How can I instantiate the object with my custom init method?
here is the respective code of my options.py file:
class Options:
anyBarFailureColor = ""
anyBarWarningColor = ""
anyBarScrubInProgressColor = ""
anyBarPortNumber = 0
def __int__(self, afc="red", awc="orange", asc="gray", apn=1738):
self.anyBarFailureColor = afc
self.anyBarWarningColor = awc
self.anyBarScrubInProgressColor = asc
self.anyBarPortNumber = apn
here is the Python error:
Traceback (most recent call last):
File "/pathToFile/Python/project/main", line 58, in <module>
start()
File "/pathToFile/Python/project/main.py", line 55, in start
options,listOfPools=mapArgs(args)
File "/pathToFile/Python/project/main.py", line 39, in mapArgs
options = Options(anyBarFailureColor,anyBarWarningColor,anyBarScrubInProgressColor,anyBarPortNumber)
TypeError: object() takes no parameters
One should neither type int nor index and wonder why it isn't working. Sigh. Typo, never was a init to begin with. Now everything works.
I want to split my Python 3.4 unit tests in separate modules and still be able to control which tests to run or skip from the command line, as if all tests were located in the same file. I'm having trouble doing so.
According to the docs, command line arguments can be used to select which tests to run. For example:
TestSeqFunc.py:
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import random
import unittest
class TestSequenceFunctions(unittest.TestCase):
def setUp(self):
self.seq = list(range(10))
def test_shuffle(self):
# make sure the shuffled sequence does not lose any elements
random.shuffle(self.seq)
self.seq.sort()
self.assertEqual(self.seq, list(range(10)))
# should raise an exception for an immutable sequence
self.assertRaises(TypeError, random.shuffle, (1,2,3))
def test_choice(self):
element = random.choice(self.seq)
self.assertTrue(element in self.seq)
def test_sample(self):
with self.assertRaises(ValueError):
random.sample(self.seq, 20)
for element in random.sample(self.seq, 5):
self.assertTrue(element in self.seq)
if __name__ == '__main__':
unittest.main()
can be controlled with either:
./TestSeqFunc.py
to run all tests in the file,
./TestSeqFunc.py TestSequenceFunctions
to run all tests defined in the TestSequenceFunctions class, and finally:
./TestSeqFunc.py TestSequenceFunctions.test_sample
to run the specific test_sample() method.
The problem I have is that I cannot find an organization of files that will allow me to:
Have multiple modules containing multiple classes and methods in separate files
Use a kind of wrapper script that will give the same kind of control over which tests (module/file, class, method) to run.
The problem I have is I cannot find a way to emulate the python3 -m unittest behaviour using a run_tests.py script. For example, I want to be able to do:
Run all the tests in the current directory
So ./run_tests.py -v should do do the same as python3 -m unittest -v
Run one module (file):
./run_tests.py -v TestSeqFunc being equivalent to python3 -m unittest -v TestSeqFunc
Run one class:
./run_tests.py -v TestSeqFunc.TestSequenceFunctions being equivalent to python3 -m unittest -v TestSeqFunc.TestSequenceFunctions
Run specific methods from a class:
./run_tests.py -v TestSeqFunc.TestSequenceFunctions.test_sample being equivalent to python3 -m unittest -v TestSeqFunc.TestSequenceFunctions.test_sample
Note that I want to:
be able to pass arguments to unittests, for example the verbose flag used previously;
allow running specific modules, classes and even methods.
As of now, I use a suite() function in my run_all.py script which loads manually the modules and add their class to a suite using addTest(unittest.makeSuite(obj)). Then, my main() is simple:
if __name__ == '__main__':
unittest.main(defaultTest='suite')
But using this I cannot run specific tests. In the end, I might just execute python3 -m unittest <sys.argv> from inside the run_all.py script, but that would be inelegant...
Any suggestions?!
Thanks!
Here's my final run_all.py:
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import unittest
import glob
test_pattern = 'validate_*.py'
if __name__ == '__main__':
# Find all files matching pattern
module_files = sorted(glob.glob(test_pattern))
module_names = [os.path.splitext(os.path.basename(module_file))[0] for module_file in module_files]
# Iterate over the found files
print('Importing:')
for module in module_names:
print(' ', module)
exec('import %s' % module)
print('Done!')
print()
unittest.main(defaultTest=module_names)
Notes:
I use exec() to simulate 'import modulename'. The issue is that using importlib (explained here for example) will import the module but will not create a namespace for the module content. When I type import os, an "os" namespace is created and I can then access os.path. By using importlib, I couldn't figure out a way to do create that namespace. Having such a namespace is required for unittest; you get these kind of errors:
Traceback (most recent call last):
File "./run_all.py", line 89, in <module>
unittest.main(argv=sys.argv)
File "~/usr/lib/python3.4/unittest/main.py", line 92, in __init__
self.parseArgs(argv)
File "~/usr/lib/python3.4/unittest/main.py", line 139, in parseArgs
self.createTests()
File "~/usr/lib/python3.4/unittest/main.py", line 146, in createTests
self.module)
File "~/usr/lib/python3.4/unittest/loader.py", line 146, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
File "~/usr/lib/python3.4/unittest/loader.py", line 146, in <listcomp>
suites = [self.loadTestsFromName(name, module) for name in names]
File "~/usr/lib/python3.4/unittest/loader.py", line 114, in loadTestsFromName
parent, obj = obj, getattr(obj, part)
AttributeError: 'module' object has no attribute 'validate_module1'
Hence the use of exec().
I have to add defaultTest=module_names or else main() defaults to all test classes inside the current file. Since there is no test class in run_all.py, nothing gets executed. So defaultTest must point to a list of all the modules name.
You can pass command-line arguments to unittest.main using the argv parameter:
The argv argument can be a list of options passed to the program, with
the first element being the program name. If not specified or None,
the values of sys.argv are used. (my emphasis)
So you should be able to use
if __name__ == '__main__':
unittest.main(defaultTest='suite')
without any change and be able to call your script with command-line arguments as desired.
The documentation is really vague about subclassing the CommandLineApp, only mentioning one example:
class YourApp(cli.app.CommandLineApp):
def main(self):
do_stuff()
So with the information I've found I've pieced together this code:
#!/usr/bin/env python
import os
import sys
from cli.app import CommandLineApp
# Append the parent folder to the python path
sys.path.append(os.path.join(os.path.dirname(__file__), '../'))
import tabulardata
from addrtools import extract_address
class SplitAddressApp(CommandLineApp):
def main(self):
"""
Split an address from one column to separate columns.
"""
table = tabulardata.from_file(self.params.file)
def for_each_row(i, item):
addr = extract_address(item['Address'])
print '%-3d %-75s %s' % (i, item['Address'], repr(addr))
table.each(for_each_row)
def setup(self):
self.add_param('file', metavar='FILE', help='The data file.')
self.add_param(
'cols', metavar='ADDRESS_COLUMN', nargs='+',
help='The name of the address column. If multiple names are ' + \
'passed, each column will be checked for an address in order'
)
if __name__ == '__main__':
SplitAddressApp().run()
Which seems correct to me. The documentation gives no examples on how to handle the setup method or running the application when using subclassing. I get the error:
Traceback (most recent call last):
File "bin/split_address_column", line 36, in
SplitAddressApp().run()
File "/Users/tomas/.pythonbrew/venvs/Python-2.7.3/address_cleaner/lib/python2.7/site-packages/cli/app.py", line 440, in __init__
Application.__init__(self, main, **kwargs)
File "/Users/tomas/.pythonbrew/venvs/Python-2.7.3/address_cleaner/lib/python2.7/site-packages/cli/app.py", line 129, in __init__
self.setup()
File "bin/split_address_column", line 28, in setup
self.add_param('file', metavar='FILE', help='The data file.')
File "/Users/tomas/.pythonbrew/venvs/Python-2.7.3/address_cleaner/lib/python2.7/site-packages/cli/app.py", line 385, in add_param
action = self.argparser.add_argument(*args, **kwargs)
AttributeError: 'SplitAddressApp' object has no attribute 'argparser'
So presumably I'm doing something wrong, but what?
I figured it out. Reading the source of pyCLI it turns out that the setup function is quite important for the functionality of the whole library, while I thought it was just a function where I could put my setup code. argparser is created in cli.app.CommandLineApp.setup which means I at least have to call
cli.app.CommandLineApp.setup(self)
inside the setup function for it to even work. And now the code works perfectly!