I need to write simple Python script for validating some system ENV variables. And I can do it with simplest code like a:
#test.py
import os
import sys
validation_errors = []
min_resources_count = os.environ.get('RESOURCES_MIN')
max_resources_count = os.environ.get('RESOURCES_MAX')
if int(min_resources_count) > int(max_resources_count):
validation_errors.append('Maximum value must be greater than or equal to minimum value')
if int(max_resources_count) == 0:
validation_errors.append('Maximum value cannot be zero')
...
# And so on...
...
if validation_errors:
sys.stderr.write(validation_errors) # will add some formatting
But, something inside me says that it can be written better. Maybe it can be moved inside some class, something like this:
class SystemEnvValidation:
def __init__(self):
self.min_resources_count = os.environ.get('RESOURCES_MIN')
self.max_resources_count = os.environ.get('RESOURCES_MAX')
self.validation_errors = []
def max_resources_count(self):
# place IF conditions here
if int(self.max_resources_count) == 0:
self.validation_errors.append('Maximum value cannot be zero')
...
print(SystemEnvValidation().validation_errors)
So, my question is - what is the best way to implement it? Should I place it in class? Or maybe I can leave it as is? In the case of class - I am not familiar with Python classes - could you show me simple working code structure?
Related
I'm writing some tooling for online programming contexts.
Part of it is a test case checker which actually based on a set of pairs of (input, output) files are gonna check whether the solution method is actually working.
Basically, the solution method is expected to be defined as follow:
def solution(Nexter: inputs):
# blahblah some code here and there
n = inputs.next_int()
sub_process(inputs)
# simulating a print something
yield str(n)
can be then translated (once the AST modifications) as:
def solution():
# blahblah some code here and there
n = int(input())
sub_process()
print(str(n))
Note: Nexter is a class defined to be whether a generator of user input() calls or carry out the expected inputs + some other goodies.
I'm aware of the issues related to converting back to source code from the AST (requires to rely on 3rd party stuff). I also know that there is a NodeTransformer class:
http://greentreesnakes.readthedocs.io/en/latest/manipulating.html
https://docs.python.org/3/library/ast.html#ast.NodeTransformer
But its use remains unclear to me I don't know if I'm better off checking calls, expr, etc.
Here is below what I've ended up with:
signature = inspect.signature(iterative_greedy_solution)
if len(signature.parameters) == 1 and "inputs" in signature.parameters:
parameter = signature.parameters["inputs"]
annotation = parameter.annotation
if Nexter == annotation:
source = inspect.getsource(iterative_greedy_solution)
tree = ast.parse(source)
NexterInputsRewriter().generic_visit(tree)
class NexterInputsRewriter(ast.NodeTransformer):
def visit(self, node):
#???
This is definitely not the best design ever. Next time, I would probably go for the other way around (i.e. having a definition with simple user defined input() (and output, i.e. print(...)) and replacing them with test case inputs) when passing to a tester class asserting whether actual outputs are matching expecting ones.
To sum up this what I would like to achieve and I don't really know exactly how (apart of subclassing the NodeTransformer class):
Get rid of the solution function arguments
Modifiy the inputs calls in method body (as well as in the sub calls of methods also leveraging Nexter: inputs) in order to replace them with their actual user input() implementation, e.g. inputs.next_int() = int(input())
EDIT
Found that tool (https://python-ast-explorer.com/) that helps a lot to visualize what kind of ast.AST derivatives are used for a given function.
You can probably use NodeTransformer + ast.unparse() though it wouldn't be as effective as checking out some other 3rd party solutions considering it won't preserve any of your comments.
Here is an example transformation done by refactor (I'm the author), which is a wrapper layer around ast.unparse for doing easy source-to-source transformations through AST;
import ast
import refactor
from refactor import ReplacementAction
class ReplaceNexts(refactor.Rule):
def match(self, node):
# We need a call
assert isinstance(node, ast.Call)
# on an attribute (inputs.xxx)
assert isinstance(node.func, ast.Attribute)
# where the name for attribute is `inputs`
assert isinstance(node.func.value, ast.Name)
assert node.func.value.id == "inputs"
target_func_name = node.func.attr.removeprefix("next_")
# make a call to target_func_name (e.g int) with input()
target_func = ast.Call(
ast.Name(target_func_name),
args=[
ast.Call(ast.Name("input"), args=[], keywords=[]),
],
keywords=[],
)
return ReplacementAction(node, target_func)
session = refactor.Session([ReplaceNexts])
source = """\
def solution(Nexter: inputs):
# blahblah some code here and there
n = inputs.next_int()
sub_process(inputs)
st = inputs.next_str()
sub_process(st)
"""
print(session.run(source))
$ python t.py
def solution(Nexter: inputs):
# blahblah some code here and there
n = int(input())
sub_process(inputs)
st = str(input())
sub_process(st)
What I would like to do is this...
x = MagicMock()
x.iter_values = [1, 2, 3]
for i in x:
i.method()
I am trying to write a unit test for this function but I am unsure about how to go about mocking all of the methods called without calling some external resource...
def wiktionary_lookup(self):
"""Looks up the word in wiktionary with urllib2, only to be used for inputting data"""
wiktionary_page = urllib2.urlopen(
"http://%s.wiktionary.org/wiki/%s" % (self.language.wiktionary_prefix, self.name))
wiktionary_page = fromstring(wiktionary_page.read())
definitions = wiktionary_page.xpath("//h3/following-sibling::ol/li")
print definitions.text_content()
defs_list = []
for i in definitions:
print i
i = i.text_content()
i = i.split('\n')
for j in i:
# Takes out an annoying "[quotations]" in the end of the string, sometimes.
j = re.sub(ur'\u2003\[quotations \u25bc\]', '', j)
if len(j) > 0:
defs_list.append(j)
return defs_list
EDIT:
I may be misusing mocks, I am not sure. I am trying to unit-test this wiktionary_lookup method without calling external services...so I mock urlopen..I mock fromstring.xpath() but as far as I can see I need to also iterate through the return value of xpath() and call a method "text_contents()" so that is what I am trying to do here.
If I have totally misunderstood how to unittest this method then please tell me where I have gone wrong...
EDIT (adding current unittest code)
#patch("lang_api.models.urllib2.urlopen")
#patch("lang_api.models.fromstring")
def test_wiktionary_lookup_2(self, fromstring, urlopen):
"""Looking up a real word in wiktionary, should return a list"""
fromstring().xpath.return_value = MagicMock(
content=["test", "test"], return_value='test\ntest2')
# A real word should give an output of definitions
output = self.things.model['word'].wiktionary_lookup()
self.assertEqual(len(output), 2)
What you actually want to do is not return a Mock with a return_value=[]. You actually want to return a list of Mock objects. Here is a snippet of your test code with the correct components and a small example to show how to test one of the iterations in your loop:
#patch('d.fromstring')
#patch('d.urlopen')
def test_wiktionary(self, urlopen_mock, fromstring_mock):
urlopen_mock.return_value = Mock()
urlopen_mock.return_value.read.return_value = "some_string_of_stuff"
mocked_xpath_results = [Mock()]
fromstring_mock.return_value.xpath.return_value = mocked_xpath_results
mocked_xpath_results[0].text_content.return_value = "some string"
So, to dissect the above code to explain what was done to correct your problem:
The first thing to help us with testing the code in the for loop is to create a list of mock objects per:
mocked_xpath_results = [Mock()]
Then, as you can see from
fromstring_mock.return_value.xpath.return_value = mocked_xpath_results
We are setting the return_value of the xpath call to our list of mocks per mocked_xpath_results.
As an example of how to do things inside your list, I added how to mock within the loop, which is shown with:
mocked_xpath_results[0].text_content.return_value = "some string"
In unittests (this might be a matter of opinion) I like to be explicit, so I'm accessing the list item explicitly and determining what should happen.
Hope this helps.
I have a class in my code to represent a root. A simplified version looks like this:
class Root:
simple = []
def __init__(self, square):
if square == 0:
self = 0
simple = simplify(square) # Finds prime factors, simplifies
What i'd like this to do is set it so the instance will become a plain integer (so if I call x = Root(0) x will just be a 0). I know the self = 0 wont work because I'm just setting self to be 0 like any other variable. Is there any way to reassign the class? I think this requires pointers, a feature I don't believe is available in python.
I have a program that compares two classes in a series of tests.
The main program (called initial.py) assigns both values to a dictionary
import testcheck
values = {}
valueChange = False
if __name__ == "__main__":
values['valueOne'] = testcheck.assignValue() #see note 1
values['valueTwo'] = testcheck.assignValueTwo()
testcheck.checkValues() #see note 2
while valueChange is True :
values['valueTwo'] = testcheck.assignValueTwo()
testcheck.checkValues()
Note 1: both of these return the same class but with different values
Note 2: compares the two classes. after a series of tests, valueChange is set to True, and one value is to be deleted using this code
import initial
...
if initial.valueChange is True:
del initial.values['valueTwo']
...
This returns the error
del initial.values['valueTwo']
KeyError: 'valueTwo'
I thought it was because adding valueOne and valueTwo would be adding it in the local scope, but even with global values it doesn't fix it. How would I go about solving this?
This appears to be a design issue. You seem to be setting up circular imports, which should be avoided if possible. If what you are after is to share a global state across modules of your package, you might want to make use of storing the state within your testcheck module, and not in a global variable of your initial.py
testcheck.py
# module globals
_TEST_VALUES = {}
valueChanged = False
...
def getTestValue(name):
return _TEST_VALUES.get('name', None)
def assignValue():
# do stuff
result = 'foo'
_TEST_VALUES['valueOne'] = result
return result
def assignValueTwo():
# do stuff
result = 'bar'
_TEST_VALUES['valueOne'] = result
return result
...
initial.py
testcheck.assignValue()
testcheck.assignValueTwo()
testcheck.checkValues()
while testcheck.valueChange:
testcheck.assignValueTwo()
testcheck.checkValues()
otherModule.py
import testcheck
...
if testcheck.valueChange:
try:
del initial.values['valueTwo']
except KeyError:
pass
...
I have no idea where this whole thing is going in terms of real usage. But maybe this will give you an idea of where to start looking. There is no longer a circular import of other modules importing your intial.py entry point. You are storing all the globals within the testcheck module. This example is very quick and dirty. Its only to illustrate.
No module should ever try to be accessing data of another module which handles the data within an if __name__ == "__main__" block. Because now you are making the assumption that it will always be used as the entry point (never imported by something else) and you start putting restrictions on your code.
Lets suppose this example: Two siblings classes where one loads the other class as a new attribute and then i wish to use this attribute from the main class inside the sibling.
a = 2
class AN(object):
def __init__(self,a):
self.aplus = a + 2
self.BECls = BE(a)
class BE(object):
def __init__(self,a):
print a
def get_aplus(self):
????
c = AN(a)
and i'd like to do:
c.BECls.get_aplus()
and this shall return something like self.self.aplus (metaphorically), that would be 4
Resuming: get aplus attribute from AN inside BE class, without declaring as arguments, but doing a "Reverse introspection", if it possible, considering the 'a' variable must be already loaded trough AN.
Sorry if I not made myself clear but I've tried to simplify what is happening with my real code.
I guess the problem may be the technique i'm using on the classes. But not sure what or how make it better.
Thanks
OP's question:
get aplus attribute from AN inside BE class, without declaring as
arguments, but doing a "Reverse introspection", if it possible,
considering the 'a' variable must be already loaded trough AN.
The closest thing we have to "reverse introspection" is a search through gc.getreferrers().
That said, it would be better to simply make the relationship explicit
class AN(object):
def __init__(self,a):
self.aplus = a + 2
self.BECls = BE(self, a)
class BE(object):
def __init__(self, an_obj, a):
self.an_obj = an_obj
print a
def get_aplus(self):
return self.an_obj.aplus
if __name__ == '__main__':
a = 2
c = AN(a)
print c.BECls.get_aplus() # this returns 4