I have developed a program that contains many functions. I am now testings these functions. To do this, I have needed to use a monkeypath approach because the function tested calls input.
Because to test the functions I need to get the data of the previous tests, with the monkeypath function I have found difficulties.
data_to_test_1 = test_load_sample() # I take the data from the previous test here
# SECOND TEST
def test_take_sample(monkeypatch):
'''
take_sample() requests the name of the column
and take that input to split the data in new columns
This can be tested by checking some ofthe first values of that
columns (GT:AD:DP:GQ:PL)
monkeypatch simulate the input of the user
'''
monkeypatch.setattr('builtins.input', lambda _: "373978487") # The 9th sample in the file
data_to_test_2 = take_sample(data_to_test_1,NAME_FILE_1)
return data_to_test_2
assert data_to_test_2["GT"] == "0/1" # What I test
assert data_to_test_2["AD"] == "28,46"
assert data_to_test_2["DP"] == "74:99"
# Now, I want the output of the test_take_sample()
data_to_test_3 = test_take_sample()
def test_filter_1():
... # this function will use data_to_test_3
I have followed the same approach with previous functions to concatenated the data from one test to the following one but with this that involves the monkeypath thing I got
test_take_sample() missing 1 required positional argument: 'monkeypatch'
For the first case, I think function test_load_sample() did not need parameter. So you allocate 'data_to_test_1' about the value of the function 'test_load_sample()'
if you make
data_to_test_1 = test_load_sample()
data_to_test_1 receive return value of test_load_sample() not function itself
I think you think that you are allocating function to data_to_test_1 but it is not. you just allocate value of the function.
If you wanna allocate function to some variable, Use 'class' type
I have solve the issue with this
data_to_test_2,Name_sample = take_sample(data_to_test_1,NAME_FILE_1)
I have jumped the test test_take_sample() and I have just run the function take_sample. This has asked me the input and I have introduce the input adding -s when the test has been run like this: pythest test_1.py -s
Related
I have used the following suggestion to set up my code: Checking whether function has been called multiple times with different parameters
I would like to test how many times the function write_standalone_end_of_event is called.
#exchanges.impl.bats.mcformat.py
def delete_data(...):
exg = exchanges.impl.bate.parser.Exchange
securities = [one_item]
for security in securities:
exg.write_standalone_end_of_event
#unit_test_package
with patch('exchanges.impl.bate.parser.Exchange.write_standalone_end_of_event') as function:
bats.protocol.sec_id_in_batch = {1: 'dummy1'}
bats.protocol.transaction_end(exg, fields, values, msgtype, sequence)
assert function.gacs.call_count == 1
When I run the above, i get an asserTion error that 0 != 1.
I can't seem to figure out why the function is not picking up the one call to it, even though when I debug it, it does seem to call write_standalone_end_of_event one time.
Issue: I have 2 functions that both require the same nested functions to operate so they're currently copy-pasted into each function. These functions cannot be combined as the second function relies on calling the first function twice. Unnesting the functions would result in the addition of too many parameters.
Question: Is it better to run the nested functions in the first function and append their values to an object to be fed into the 2nd function, or is it better to copy and paste the nested functions?
Example:
def func_A(thing):
def sub_func_A(thing):
thing += 1
return sub_func_A(thing)
def func_B(thing):
def sub_func_B(thing):
thing += 1
val_A, val_B = func_A(5), func_A(5)
return sub_func_B(val_A), sub_func_B(val_B)
Imagine these functions couldn't be combined and the nested function relied on so many parameters that moving it outside and calling it would be too cluttered
The "better option" depends on a few factors -:
The type of optimization you want to achieve.
The time taken by the functions to execute.
If the type of optimization to be achieved here is based on the time taken to execute the second function in the two cases, then it depends on the time taken for the nested function to fully execute, if that time is less than the time taken to store it's output when it's first called by the first function then its better copy pasting them.
While, if the time taken by the nested function to execute is more than the time taken to store it's output, then its a better option to execute it first time and then store it's output for future use.
Further, As mentioned by #DarylG in the comments, a class based approach can also be used wherein the nested function(subfunction) can be a private function(only accessible by the class's inner components), while the two functions(func_A and func_B) can be public thus allowing them to be used and accessed widely from the outside as well. If implemented in code it might look something like this -:
class MyClass() :
def __init__(self, ...) :
...
return
def __subfunc(self, thing) :
# PRIVATE SUBFUNC
thing += 1
return thing
def func_A(self, thing):
# PUBLIC FUNC A
return self.__subfunc(thing)
def func_B(self, thing):
# PUBLIC FUNC B
val_A, val_B = self.func_A(5), self.func_A(5)
return self.__subfunc(val_A), self.__subfunc(val_B)
Assume I have a scenario similar to this:
Scenario Outline: Example scenario
Given the subprocess is running
When I generate the input
And I add <argument1> to the input
And I add <argument2> to the input
And this input is passed to the subprocess
Then the output should match the <output> for <argument1> and <argument2>
I'd very much like to reuse the 'when' step as, e.g. And I add <argument> to the input, but don't want to use an Examples table as I wish the fixtures to by dynamically generated in the step definition/conftest file. I'm currently using #pytest.mark.parametrize to parametrize the scenario outlines like so:
import pytest
from pytest_bdd import scenario
from functools import partial
from some_lib import test_data, utils
#pytest.fixture(scope='module')
def context():
return {}
scenario = partial(scenario, '../features/example.feature')
#pytest.mark.parametrize(
[argument1, argument2],
[(test_data.TEST_ARGUMENT[1], test_data.TEST_ARGUMENT[2]),],
)
#scenario('Example scenario')
def test_example_scenario(context, argument1, argument2):
pass
I would like to be able to reuse the same step definition in the same scenario with the different arguments somehow, e.g.
#when('I add <argument> to the input')
def add_argument(context, argument):
context['input'] = utils.add_argument(context['input'], argument)
rather than having to have two step definitions, e.g.
#when('I add <argument1> to the input')
def add_argument(context, argument1):
context['input'] = utils.add_argument(context['input'], argument1)
#when('I add <argument2> to the input')
def add_argument(context, argument2):
context['input'] = utils.add_argument(context['input'], argument2)
The pytest-bdd documentation seems to suggest this is possible, but I can't quite wrap my head around how I might accomplish this without using example tables.
Often it’s possible to reuse steps giving them a parameter(s). This allows to have single implementation and multiple use, so less code. Also opens the possibility to use same step twice in single scenario and with different arguments! [sic] (Emphasis my own)
Does anyone have any ideas on how I might accomplish this?
Thank you for your time as always!
I think the pytest-bdd documentation is rather suggesting re-usage of a step due to a variable in the step definition instead of a hard-coded value...so I think the documentation does not give you any solution for your problem.
Anyway, there is a solution that I use, which is getting the value of the step variable dynamically. Pytest-bdd will create a pytest-fixture for every variable you define in your steps and therefore you can obtain the value of a fixture by calling request.getfixturevalue(name_of_fixture), as long as you know the name of the fixture.
For your case I would use parsers.parse() for the step definitions, so that the variables argument1 and argument2 will hold the name of the fixtures instead of their value.
Example
#when(parsers.parse('I add {argument1} to the input'))
def add_argument(request, context, argument1):
# Remove angle brackets, because they are not part of the fixture name
argument1 = argument1.replace('<', '').replace('>', '')
argument_value = request.getfixturevalue(argument1)
context['input'] = utils.add_argument(context['input'], argument_value)
Assume for example that I am writing a function for the following code (in pandas module):
myData.sort_values(by=myVariable, ascending=False)
Now, I want to create a function whose variables are data, variable and ascending.
sort(data=mydata, variable=myvariable, ascending=False)
The below function gives an error, because whether ascending is true or false is not given in second line:
def sort(data,variable,ascending=False):
data.sort_values(by=variable, ascending)
Although it would work, I do not want to change variable name such as:
def sort(data,variable,asc=False):
data.sort_values(by=variable, ascending=asc)
One last approach would be to create a variable inside the function:
def sort(data,variable,ascending=False):
asc = ascending
data.sort_values(by=variable, ascending=asc)
but it looks somewhat confusing. Is there an alternative approach to use the same variable in this case?
Note: the question is not related with pandas module, it is just an example.
Edit: I have clearly stated my problem and showed what I have tried. I did not understand the reason for the downvote.
have you tried:
def sort(data,variable,ascending=False):
data.sort_values(by=variable, ascending=ascending)
The below function gives an error, because whether ascending is true or false is not given in second line:
def sort(data,variable,ascending=False):
data.sort_values(by=variable, ascending)
No; it gives an error because positional arguments cannot come after keyword arguments. Like the error message says: SyntaxError: positional argument follows keyword argument.
The way to fix this is as in the other answer, using ascending=ascending. The idea is that the arguments for the function call are in a separate namespace from the current local variables.
If it "didn't work", then you need to ask a new question and properly explain the expected and actual results, with a complete example that other users can test locally.
I'm currently trying to code a Python (3.4.4) GUI with tkinter which should allow to fit an arbitrary function to some datapoints. To start easy, I'd like to create some input-function and evaluate it. Later, I would like to plot and fit it using curve_fit from scipy.
In order to do so, I would like to create a dynamic (fitting) function from a user-input-string. I found and read about exec, but people say that (1) it is not safe to use and (2) there is always a better alternative (e.g. here and in many other places). So, I was wondering what would be the alternative in this case?
Here is some example code with two nested functions which works but it's not dynamic:
def buttonfit_press():
def f(x):
return x+1
return f
print(buttonfit_press()(4))
And here is some code that gives rise to NameError: name 'f' is not defined before I can even start to use xval:
def buttonfit_press2(xval):
actfitfunc = "f(x)=x+1"
execstr = "def {}:\n return {}\n".format(actfitfunc.split("=")[0], actfitfunc.split("=")[1])
exec(execstr)
return f
print(buttonfit_press2(4))
An alternative approach with types.FunctionType discussed here (10303248) wasn't successful either...
So, my question is: Is there a good alternative I could use for this scenario? Or if not, how can I make the code with exec run?
I hope it's understandable and not too vague. Thanks in advance for your ideas and input.
#Gábor Erdős:
Either I don't understand or I disagree. If I code the same segment in the mainloop, it recognizes f and I can execute the code segment from execstr:
actfitfunc = "f(x)=x+1"
execstr = "def {}:\n return {}\n".format(actfitfunc.split("=")[0], actfitfunc.split("=")[1])
exec(execstr)
print(f(4))
>>> 5
#Łukasz Rogalski:
Printing execstr seems fine to me:
def f(x):
return x+1
Indentation error is unlikely due to my editor, but I double-checked - it's fine.
Introducing my_locals, calling it in exec and printing in afterwards shows:
{'f': <function f at 0x000000000348D8C8>}
However, I still get NameError: name 'f' is not defined.
#user3691475:
Your example is very similar to my first example. But this is not "dynamic" in my understanding, i.e. one can not change the output of the function while the code is running.
#Dunes:
I think this is going in the right direction, thanks. However, I don't understand yet how I can evaluate and use this function in the next step? What I mean is: in order to be able to fit it, I have to extract fitting variables (i.e. a in f(x)=a*x+b) or evaluate the function at various x-values (i.e. print(f(3.14))).
The problem with exec/eval, is that they can execute arbitrary code. So to use exec or eval you need to either carefully parse the code fragment to ensure it doesn't contain malicious code (an incredibly hard task), or be sure that the source of the code can be trusted. If you're making a small program for personal use then that's fine. A big program that's responsible for sensitive data or money, definitely not. It would seem your use case counts as having a trusted source.
If all you want is to create an arbitrary function at runtime, then just use a combination of the lambda expression and eval. eg.
func_str = "lambda x: x + 1" # equates to f(x)=x+1
func = eval(func_str)
assert func(4) == 5
The reason why your attempt isn't working is that locals(), in the context of a function, creates a copy of the local namespace. Mutations to the resulting dictionary do not effect the current local namespace. You would need to do something like:
def g():
src = """
def f(x):
return x + 1
"""
exec_namespace = {} # exec will place the function f in this dictionary
exec(src, exec_namespace)
return exec_namespace['f'] # retrieve f
I'm not sure what exactly are you trying to do, i.e. what functions are allowed, what operations are permitted, etc.
Here is an example of a function generator with one dynamic parameter:
>>> def generator(n):
def f(x):
return x+n
return f
>>> plus_one=generator(1)
>>> print(plus_one(4))
5