I have a function that uses the input function as follows:
def get_valid_input(input_str: str, valid_options: tuple):
while True:
response = input(input_str)
if response in valid_options:
return response
raise InvalidOptionException('Invalid option')
I need to test that the exception InvalidOptionException, which simple extends Exception, is raised when a response is not a valid option. I have the following test:
class UtilTest(unittest.TestCase):
#patch('builtins.input', lambda: 'a')
def test_when_input_is_invalid_then_exception_is_raised(self):
self.assertRaises(
InvalidOptionException,
get_valid_input,
'', ('y', 'n')
)
But, when I ran the test, I am getting the following error:
Error
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/mock.py", line 1179, in patched
return func(*args, **keywargs)
File "/Users/user/mathsistor/chumme/test/test_util.py", line 13, in test_when_input_is_invalid_then_exception_is_raised
'', ('y', 'n')
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/case.py", line 728, in assertRaises
return context.handle('assertRaises', args, kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/unittest/case.py", line 177, in handle
callable_obj(*args, **kwargs)
File "/Users/user/mathsistor/chumme/util.py", line 14, in get_valid_input
response = input(input_str)
TypeError: <lambda>() takes 0 positional arguments but 1 was given
How can I solve this?
The input function takes one argument (the prompt string). To mock it you must use a function/lambda with one argument:
#patch('builtins.input', lambda _: 'a')
It's customary to use the underscore when you don't plan on using the argument.
Related
Current Implementation which needs optimization
import subprocess
childprocess = subprocess.Popen(
['python',
'/full_path_to_directory/called_script.py',
'arg1',
'arg2'],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
returnVal = childprocess.communicate()[0]
print(retVal)
Is this a correct way to call another script(called_script.py) within the current working directory?
Is there a better way to call the other script? I used import script but it gives me below error
called_script.py
def func(arg1, arg2, arg3):
#doSomething
#sys.out.write(returnVal)
if __name__ == "__main__":
func(arg1, arg2, arg3)
Implementation 2 (throws exception and errored out)
caller_script.py
Both of them are under the same path (i.e. /home/bin)
import called_script
returnVal = called_script.func(arg1,arg2,arg3)
print(returnVal)
Output:
nullNone
Traceback (most recent call last):
File "/path_to_caller/caller_script.py", line 89, in <module>
l.simple_bind_s(binddn, pw)
File "/usr/lib64/python2.6/site-packages/ldap/ldapobject.py", line 206, in simple_bind_s
msgid = self.simple_bind(who,cred,serverctrls,clientctrls)
File "/usr/lib64/python2.6/site-packages/ldap/ldapobject.py", line 200, in simple_bind
return self._ldap_call(self._l.simple_bind,who,cred,EncodeControlTuples(serverctrls),EncodeControlTuples(clientctrls))
File "/usr/lib64/python2.6/site-packages/ldap/ldapobject.py", line 96, in _ldap_call
result = func(*args,**kwargs)
TypeError: argument 2 must be string or read-only buffer, not None
Another alternative I used and gave me an error is
Implementation 3(throws exception and errors out)
caller_script.py
import ldap
returnVal = subprocess.call(['python','called_script.py','arg1','arg2'])
print(returnVal)
l = ldap.initialize(cp.get('some_config_ref','some_url'))
try:
l.protocol_version = ldap.VERSION3
l.simple_bind_s(binddn, returnVal)
except ldap.INVALID_CREDENTIALS:
sys.stderr.write("Your username or password is incorrect.")
sys.exit(1)
except ldap.LDAPError, e:
if type(e.message) == dict and e.message.has_key('xyz'):
sys.stderr.write(e.message['xyz'])
else:
sys.stderr.write(e)
sys.exit(1)
Output:
returnVal0Traceback (most recent call last):
File "./path_to_script/caller_script.py", line 88, in <module>
l.simple_bind_s(binddn, pw)
File "/usr/lib64/python2.6/site-packages/ldap/ldapobject.py", line 206, in simple_bind_s
msgid = self.simple_bind(who,cred,serverctrls,clientctrls)
File "/usr/lib64/python2.6/site-packages/ldap/ldapobject.py", line 200, in simple_bind
return self._ldap_call(self._l.simple_bind,who,cred,EncodeControlTuples(serverctrls),EncodeControlTuples(clientctrls))
File "/usr/lib64/python2.6/site-packages/ldap/ldapobject.py", line 96, in _ldap_call
result = func(*args,**kwargs)
TypeError: argument 2 must be string or read-only buffer, not int
Here is an example where you are calling a function from another file, you pass one value, a list, which can have an arbitrary amount of numbers, and you get the sum. Make sure they are in the same directory or you will need the path. The function in your example "script.py" does not allow you to pass a value.
called_script.py
def add_many(list_add):
the_sum = sum(list_add)
return the_sum
caller_script.py
import called_script
a_list = [1, 2, 3, 4]
the_sum = called_script.add_many(a_list)
print(the_sum)
I have a function that takes a keyword-only argument and want to run it in a process pool. How do I pass my entries from an iterable to the function in the process as a keyword argument?
import multiprocessing
greetees = ('Foo', 'Bar')
def greet(*, greetee):
return f'Hello, {greetee}!'
I tried using multiprocessing.map:
greetings = multiprocessing.Pool(2).map(greet, greetees)
for greeting in greetings:
print(greeting)
But that raises an exception, as expected:
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/usr/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
TypeError: greet() takes 0 positional arguments but 1 was given
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/bengt/Projekte/gitlab.com/PFASDR/PFASDR.Code.Main/pfasdr/neural/multi_pool_kwargs.py", line 10, in <module>
greetings = multiprocessing.Pool(2).map(greet, greetees)
File "/usr/lib/python3.6/multiprocessing/pool.py", line 266, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/usr/lib/python3.6/multiprocessing/pool.py", line 644, in get
raise self._value
TypeError: greet() takes 0 positional arguments but 1 was given
It works fine if I remove the asterisk to not require the arguments to be keyword-only:
[...]
def greet(greetee):
return f'Hello, {greetee}!'
[...]
Output:
Hello, Foo!
Hello, Bar!
A solution here is to use Pool.apply or Pool.apply_async:
greetings = list(
multiprocessing.Pool(2).apply(greet, kwds={'greetee': greetees[i]})
for i in range(len(greetees))
)
for greeting in greetings:
print(greeting)
Output:
Hello, Foo!
Hello, Bar!
Courtesy of Mad Physicist and this QnA, one can use functools.partial to inject keyword-only arguments into a function:
from functools import partial
greetings = []
for i in range(len(greetees)):
kwargs = {'greetee': greetees[i]}
greet_partial = partial(greet, **kwargs)
greetings.append(multiprocessing.Pool(2).apply(greet_partial))
Or with less variable bleed:
from functools import partial
greetings = [
multiprocessing.Pool(2).apply(
partial(greet, **{'greetee': greetees[i]})
)
for i in range(len(greetees))
]
For the purpose of a unit test, I made a class whose instance is an iterable that would yield a certain sequence and then raise an exception:
class Iter:
def __init__(self, seq):
self.seq = seq
self.pos = 0
def __next__(self):
if self.pos == len(self.seq):
raise Exception
value = self.seq[self.pos]
self.pos += 1
return value
def __iter__(self):
return self
so that:
for value in Iter((1, 2, 3)):
print(value)
would output:
1
2
3
Traceback (most recent call last):
File "test.py", line 25, in <module>
for value in mocked_iterable:
File "test.py", line 11, in __next__
raise Exception
Exception
But why reinvent the wheel when MagicMock already has a side_effect attribute that should do the same? Per the documentation, the side_effect attribute can be an iterable that yields either a value to be returned from the call to the mock, or an exception to raise, so it suits the purpose of mimicking the aforementioned class perfectly. I therefore created a MagicMock object and made its __iter__ method return the object itself, and made its __next__ method to have a side effect of the desired sequence and the exception:
from unittest.mock import MagicMock
mocked_iterable = MagicMock()
mocked_iterable.__iter__.return_value = mocked_iterable
mocked_iterable.__next__.side_effect = [1, 2, 3, Exception]
for value in mocked_iterable:
print(value)
However, this outputs:
...
File "C:\Program Files (x86)\Python36-32\lib\unittest\mock.py", line 1005, in _mock_call
ret_val = effect(*args, **kwargs)
File "C:\Program Files (x86)\Python36-32\lib\unittest\mock.py", line 1793, in __iter__
return iter(ret_val)
File "C:\Program Files (x86)\Python36-32\lib\unittest\mock.py", line 939, in __call__
return _mock_self._mock_call(*args, **kwargs)
File "C:\Program Files (x86)\Python36-32\lib\unittest\mock.py", line 944, in _mock_call
self.called = True
RecursionError: maximum recursion depth exceeded
But the question is, why is there any recursion?
I found that I can work around this "bug" by putting the self reference in __iter__'s side_effect attribute instead:
mocked_iterable = MagicMock()
mocked_iterable.__iter__.side_effect = [mocked_iterable]
mocked_iterable.__next__.side_effect = [1, 2, 3, Exception]
for value in mocked_iterable:
print(value)
This correctly outputs:
1
2
3
Traceback (most recent call last):
File "test.py", line 6, in <module>
for value in mocked_iterable:
File "C:\Program Files (x86)\Python36-32\lib\unittest\mock.py", line 939, in __call__
return _mock_self._mock_call(*args, **kwargs)
File "C:\Program Files (x86)\Python36-32\lib\unittest\mock.py", line 1000, in _mock_call
raise result
Exception
But is the recursion error indeed a bug, or a feature of mock with an unintended consequence?
I agree that this is indeed a bug. Although this is an edge case.
As we can see in the source code. mock module expects that iter(ret_val) will return the unchanged iterator if ret_val has already been an iterator.
Well, it actually does but still needs to call ret_val's __iter__ method.
I am running into an error calling add_job with kwargs for a function:
I created a function to create scheduled jobs:
def initial_data_pull():
q.enqueue(initial_pull, timout='3h')
def initial_data_load():
dates_list = date_between_list(start=date(2010, 1, 1), stop=date.today())
naics = ['541611', '541618', '541613',
'541511', '541512', '541513',
'541519', '518210', '541612']
fundings = ['3600', '97DH', '7504', '7505',
'7522', '7523', '7524', '7526',
'7527', '7528', '7529', '7530',
'7570']
for date_item in dates_list:
for fund in fundings:
for naic in naics:
sched.add_job(initial_data_pull,
kwargs={'naics_code': naic,
'funding_agency_id': fund,
'date_signed': date_item})
The function initial_pull looks like:
def initial_pull(naics_code=None, funding_agency_id=None, date_signed=None):
print('Gathering Contracts...')
df = fpds_generic(naics_code=naics_code,
funding_agency_id=funding_agency_id,
date_signed=date_signed)
# Code to Process and load data
The function fpds_generic is a function that goes to fpds and gather's data. I am getting the following error when I run the inital_data_load function.
Traceback (most recent call last):
File "clock.py", line 55, in <module>
initial_data_load()
File "clock.py", line 52, in initial_data_load
sched.add_job(initil_data_pull, kwargs=kw)
File "/home/spitfiredd/anaconda3/envs/g2x_flask/lib/python3.6/site-packages/apscheduler/schedulers/base.py", line 425, in add_job
job = Job(self, **job_kwargs)
File "/home/spitfiredd/anaconda3/envs/g2x_flask/lib/python3.6/site-packages/apscheduler/job.py", line 44, in __init__
self._modify(id=id or uuid4().hex, **kwargs)
File "/home/spitfiredd/anaconda3/envs/g2x_flask/lib/python3.6/site-packages/apscheduler/job.py", line 175, in _modify
check_callable_args(func, args, kwargs)
File "/home/spitfiredd/anaconda3/envs/g2x_flask/lib/python3.6/site-packages/apscheduler/util.py", line 385, in check_callable_args
', '.join(unmatched_kwargs))
ValueError: The target callable does not accept the following keyword arguments: naics_code, funding_agency_id, date_signed
Why is it saying that the function does not accept those keyword args when it does, and how do I fix this?
I have a command as follows:
class AddChatMessages(Command):
arguments = [
('messages', AmpList([('message', Unicode()), ('type', Integer())]))]
And I have a responder for it in a controller:
def add_chat_messages(self, messages):
for i, m in enumerate(messages):
messages[i] = (m['message'], m['type'])
self.main.add_chat_messages(messages)
return {}
commands.AddChatMessages.responder(add_chat_messages)
I am writing a unit test for it. This is my code:
class AddChatMessagesTest(ProtocolTestMixin, unittest.TestCase):
command = commands.AddChatMessages
data = {'messages': [{'message': 'hi', 'type': 'None'}]}
def assert_callback(self, unused):
pass
Where ProtocolMixin is as follows:
class ProtocolTestMixin(object):
def setUp(self):
self.protocol = client.CommandProtocol()
def assert_callback(self, unused):
raise NotImplementedError("Has to be implemented!")
def test_responder(self):
responder = self.protocol.lookupFunction(
self.command.commandName)
d = responder(self.data)
d.addCallback(self.assert_callback)
return d
It works if AmpList is not involved, but when it is - I get following error:
======================================================================
ERROR: test_responder
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/<username>/Projects/space/env/lib/python2.7/site-packages/twisted/internet/defer.py", line 139, in maybeDeferred
result = f(*args, **kw)
File "/Users/<username>/Projects/space/env/lib/python2.7/site-packages/twisted/internet/utils.py", line 203, in runWithWarningsSuppressed
reraise(exc_info[1], exc_info[2])
File "/Users/<username>/Projects/space/env/lib/python2.7/site-packages/twisted/internet/utils.py", line 199, in runWithWarningsSuppressed
result = f(*a, **kw)
File "/Users/<username>/Projects/space/tests/client_test.py", line 32, in test_responder
d = responder(self.data)
File "/Users/<username>/Projects/space/env/lib/python2.7/site-packages/twisted/protocols/amp.py", line 1016, in doit
kw = command.parseArguments(box, self)
File "/Users/<username>/Projects/space/env/lib/python2.7/site-packages/twisted/protocols/amp.py", line 1717, in parseArguments
return _stringsToObjects(box, cls.arguments, protocol)
File "/Users/<username>/Projects/space/env/lib/python2.7/site-packages/twisted/protocols/amp.py", line 2510, in _stringsToObjects
argparser.fromBox(argname, myStrings, objects, proto)
File "/Users/<username>/Projects/space/env/lib/python2.7/site-packages/twisted/protocols/amp.py", line 1209, in fromBox
objects[nk] = self.fromStringProto(st, proto)
File "/Users/<username>/Projects/space/env/lib/python2.7/site-packages/twisted/protocols/amp.py", line 1465, in fromStringProto
boxes = parseString(inString)
File "/Users/<username>/Projects/space/env/lib/python2.7/site-packages/twisted/protocols/amp.py", line 2485, in parseString
return cls.parse(StringIO(data))
TypeError: must be string or buffer, not list
Which makes sense, but how do I serialize a list in AddChatMessagesTest.data?
The responder expects to be called with a serialized box. It will then deserialize it, dispatch the objects to application code, take the object the application code returns, serialize it, and then return that serialized form.
For a few AMP types. most notably String, the serialized form is the same as the deserialized form, so it's easy to overlook this.
I think that you'll want to pass your data through Command.makeArguments in order to produce an object suitable to pass to a responder.
For example:
>>> from twisted.protocols.amp import Command, Integer
>>> class Foo(Command):
... arguments = [("bar", Integer())]
...
>>> Foo.makeArguments({"bar": 17}, None)
AmpBox({'bar': '17'})
>>>
If you do this with a Command that uses AmpList I think you'll find makeArguments returns an encoded string for the value of that argument and that the responder is happy to accept and parse that kind of string.