We Keep Coding

Is there a way to grab list attributes that have been initialized using self and append data to them in Python?

I have a class in Python that initializes the attributes of an environment. I am attempting to grab the topographyRegistry attribute list of my Environment class in a separate function, which when called, should take in the parameters of 'self' and the topography to be added. When this function is called, it should simply take an argument such as addTopographyToEnvironment(self, "Mountains") and append it to the topographyRegistry of the Environment class.
When implementing what I mentioned above, I ran into an error regarding the 'self' method not being defined. Hence, whenever I call the above line, it gives me:
print (Environment.addTopographyToEnvironment(self, "Mountains"))
^^^^
NameError: name 'self' is not defined
This leads me to believe that I am unaware of and missing a step in my implementation, but I am unsure of what that is exactly.
Here is the relevant code:
class EnvironmentInfo:
def __init__(self, perceivableFood, perceivableCreatures, regionTopography, lightVisibility):
self.perceivableFood = perceivableFood
self.perceivableCreatures = perceivableCreatures
self.regionTopography = regionTopography
self.lightVisibility = lightVisibility
class Environment:
def __init__(self, creatureRegistry, foodRegistry, topographyRegistery, lightVisibility):
logging.info("Creating new environment")
self.creatureRegistry = []
self.foodRegistry = []
self.topographyRegistery = []
self.lightVisibility = True
def displayEnvironment():
creatureRegistry = []
foodRegistry = []
topographyRegistery = ['Grasslands']
lightVisibility = True
print (f"Creatures: {creatureRegistry} Food Available: {foodRegistry} Topography: {topographyRegistery} Contains Light: {lightVisibility}")
def addTopographyToEnvironment(self, topographyRegistery):
logging.info(
f"Registering {topographyRegistery} as a region in the Environment")
self.topographyRegistery.append(topographyRegistery)
def getRegisteredEnvironment(self):
return self.topographyRegistry
if __name__ == "__main__":
print (Environment.displayEnvironment()) #Display hardcoded attributes
print (Environment.addTopographyToEnvironment(self, "Mountains"))#NameError
print (Environment.getRegisteredEnvironment(self)) #NameError
What am I doing wrong or not understanding when using 'self'?
Edit: In regard to omitting 'self' from the print statement, it still gives me an error indicating a TypeError:
print (Environment.addTopographyToEnvironment("Mountains"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Environment.addTopographyToEnvironment() missing 1 required positional argument: 'topographyRegistery'

Comments
Despite having def getRegisteredEnvironment(self): it wasn't indented, so it's not recognized as a class method.
self is a keyword used in conjunction with classes (class methods or attributes) - not functions. self is implied to be the instantiated object (eg a = Environment(...) -> self would refer to a) or the module's (I can't think of the proper term) class.
You didn't have your addTopographyToEnvironment class method defined.
In terms of your Environment class, you aren't using the variables you are passing to the class, so I made that change as well - I don't know if that was intentional or not.
As per your comment from the other answer, if you had def my_class_method(self) and you try to invoke it through an object with additional parameters, like so a = my_object(); a.my_class_method("Mountains"), you should get an error of the sorts, "2 positional arguments passed, expected 1.".
Your main problem is that you are doing Environment.class_method() and not creating an object from the class. Do a = Environment(whatever arguments here) to create an object from the class, then do a.addTopographyToEnvironment("Mountains") to do what you were going to do with "Mountains" and that object. What you have currently may be right, its just is missing the proper implementation, but the below article does a great job explaining the differences between all of them (Class Methods vs Static Methods vs Instance Methods), and is definitely worth the read.
class EnvironmentInfo:
def __init__(self, perceivableFood, perceivableCreatures, regionTopography, lightVisibility):
self.perceivableFood = perceivableFood
self.perceivableCreatures = perceivableCreatures
self.regionTopography = regionTopography
self.lightVisibility = lightVisibility
class Environment:
def __init__(self, creatureRegistry, foodRegistry, topographyRegistery, lightVisibility):
logging.info("Creating new environment")
self.creatureRegistry = creatureRegistry
self.foodRegistry = foodRegistry
self.topographyRegistery = topographyRegistery
self.lightVisibility = lightVisibility
def displayEnvironment(self):
creatureRegistry = []
foodRegistry = []
topographyRegistery = ['Grasslands']
lightVisibility = True
print (f"Creatures: {creatureRegistry} Food Available: {foodRegistry} Topography: {topographyRegistery} Contains Light: {lightVisibility}")
def addTopographyToEnvironment(self, environment):
return "Whatever this is supposed to return." + environment
def getRegisteredEnvironment(self):
return self.topographyRegistry
if __name__ == "__main__":
print (Environment.displayEnvironment()) #Display hardcoded attributes
print (Environment.addTopographyToEnvironment("Mountains"))#NameError
print (Environment.getRegisteredEnvironment()) #NameError
Object Instantiation In Python
With all that out of the way, I will answer the question as is posed, "Is there a way to grab list attributes that have been initialized using self and append data to them in Python?". I am assuming you mean the contents of the list and not the attributes of it, the attributes would be "got" or at least printed with dir()
As a simple example:
class MyClass:
def __init__(self, my_list):
self.my_list = my_list
if __name__ == "__main__":
a = MyClass([1, 2, 3, 4, 5])
print(a.my_list)
# will print [1, 2, 3, 4, 5]
a.my_list.append(6)
print(a.my_list)
# will print [1, 2, 3, 4, 5, 6]
print(dir(a.my_list))
# will print all object methods and object attributes for the list associated with object "a".
Sub Classing In Python
Given what you have above, it looks like you should be using method sub classing - this is done with the keyword super. From what I can guess, it would look like you'd implement that kind of like this:
class EnvironmentInfo:
def __init__(self, perceivableFood, perceivableCreatures, regionTopography, lightVisibility):
self.perceivableFood = perceivableFood
self.perceivableCreatures = perceivableCreatures
self.regionTopography = regionTopography
self.lightVisibility = lightVisibility
class Environment(EnvironmentInfo):
def __init__(self, creatureRegistry, foodRegistry, topographyRegistery, lightVisibility, someOtherThingAvailableToEnvironmentButNotEnvironmentInfo):
logging.info("Creating new environment")
super.__init__(foodRegistry, creatureRegistry, topographyRegistery, lightVisibility)
self.my_var1 = someOtherThingAvailableToEnvironmentButNotEnvironmentInfo
def displayEnvironment(self):
creatureRegistry = []
foodRegistry = []
topographyRegistery = ['Grasslands']
lightVisibility = True
print (f"Creatures: {creatureRegistry} Food Available: {foodRegistry} Topography: {topographyRegistery} Contains Light: {lightVisibility}")
def addTopographyToEnvironment(self, environment):
return "Whatever this is supposed to return." + environment
def getRegisteredEnvironment(self):
return self.topographyRegistry
def methodAvailableToSubClassButNotSuper(self)
return self.my_var1
if __name__ == "__main__":
a = Environment([], [], [], True, "Only accessible to the sub class")
print(a.methodAvailableToSubClassButNotSuper())
as the article describes when talking about super(), methods and attributes from the super class are available to the sub class.
Extra Resources
Class Methods vs Static Methods vs Instance Methods - "Difference #2: Method Defination" gives an example that would be helpful I think.
What is sub classing in Python? - Just glanced at it; probably an okay read.

Self represents the instance of the class and you don't have access to it outside of the class, by the way when you are calling object methods of a class you don't need to pass self cause it automatically be passed to the method you just need to pass the parameters after self so if you want to call an object method like addTopographyToEnvironment(self, newVal) you should do it like:
Environment.addTopographyToEnvironment("Mountains")
and it should work fine

Elegant way in Python to map string content to method calls

Given this example code where we have a series of log processors, I can't help feeling there ought to be a more pythonic/efficient way of deciding which log processor to use to process some data:
class Component1ErrorLogProcessor:
def process(logToProcess):
# Do something with the logs
pass
class Component2ErrorLogProcessor:
def process(logToProcess):
# Do something with the logs
pass
class LogProcessor:
def __init__(self):
self.component1 = Component1ErrorLogProcessor()
self.component2 = Component2ErrorLogProcessor()
def process_line(self, line, component):
if component == "Component1Log-" or component == "[Component1]":
self.component1.process_errors(line)
elif component == "Component2Log-" or component == "[Component2]":
self.component2.process_errors(line)

I'd personally use the idea of registry, so you map each class to component names.
There are a bunch of different ways to go about this, here's a quick example by using a base class:
class ComponentLogProcessor(object):
_Mapping = {}
#classmethod
def register(cls, *component_names):
for name in component_names:
cls._Mapping[name] = cls
#classmethod
def cls_from_component(cls, component):
return cls._Mapping[component]
class Component1ErrorLogProcessor(ComponentLogProcessor):
def process(logToProcess):
# Do something with the logs
pass
Component1ErrorLogProcessor.register('Component1Log-', '[Component1]')
class Component2ErrorLogProcessor(ComponentLogProcessor):
def process(logToProcess):
# Do something with the logs
pass
Component2ErrorLogProcessor.register('Component2Log-', '[Component2]')
class LogProcessor:
def process_line(self, line, component):
ComponentLogProcessor.cls_from_component(component).process_errors(line)

Python: Construct class (and variable names) through a function?

I recently started to work with Python's classes, since I need to work with it through the use of OTree, a Python framework used for online experiment.
In one file, I define the pages that I want to be created, using classes. So essentially, in the OTree system, each class corresponds to a new page. The thing is, all pages (so classes) are basically the same, at the exception to some two parameters, as shown in the following code:
class Task1(Page):
form_model = 'player'
form_fields = ['Envie_WordsList_Toy']
def is_displayed(self):
return self.round_number == self.participant.vars['task_rounds'][1]
def vars_for_template(player):
WordsList_Toy= Constants.WordsList_Toy.copy()
random.shuffle(WordsList_Toy)
return dict(
WordsList_Toy=WordsList_Toy
)
#staticmethod
def live_method(player, data):
player.WTP_WordsList_Toy = int(data)
def before_next_page(self):
self.participant.vars['Envie_WordsList_Toy'] = self.player.Envie_WordsList_Toy
self.participant.vars['WTP_WordsList_Toy'] = self.player.WTP_WordsList_Toy
So here, the only thing that would change would be the name of the class, as well as the suffix of the variable WordsList_ used throughout this code, which is Toy.
Naively, what I tried to do is to define a function that would take those two parameters, such as this:
def page_creation(Task_Number,name_type):
class Task+str(Task_Number)(Page):
form_model = 'player'
form_fields = ['Envie_WordsList_'+str(name_type)]
def is_displayed(self):
return self.round_number == self.participant.vars['task_rounds'][1]
def vars_for_template(player):
WordsList_+str(name_type) = Constants.WordsList+str(name_type).copy()
random.shuffle(WordsList_+str(name_type))
return dict(
WordsList_+str(name_type)=WordsList_+str(name_type)
)
#staticmethod
def live_method(player, data):
player.WTP_WordsList_+str(name_type) = int(data)
def before_next_page(self):
self.participant.vars['Envie_WordsList_+str(name_type)'] = self.player.Envie_WordsList_+str(name_type)
self.participant.vars['WTP_WordsList_+str(name_type)'] = self.player.WTP_WordsList_+str(name_type)
Obviously, it does not work since I have the feeling that it is not possible to construct variables (or classes identifier) this way. I just started to really work on Python some weeks ago, so some of its aspects might escape me still. Could you help me on this issue? Thank you.

You can generate dynamic classes using the type constructor:
MyClass = type("MyClass", (BaseClass1, BaseClass2), {"attr1": "value1", ...})
Thus, according to your case, that would be:
cls = type(f"Task{TaskNumber}", (Page, ), {"form_fields": [f"Envive_WordList_{name_type}"], ...})
Note that you still have to construct your common methods like __init__, is_displayed and so on, as inner functions of the class factory:
def class_factory(*args, **kwargs):
...
def is_displayed(self):
return self.round_number == self.participant.vars['task_rounds']
def vars_for_template(player):
...
# Classmethod wrapping is done below
def live_method(player, data):
...
cls = type(..., {
"is_displayed": is_displayed,
"vars_for_template": vars_for_template,
"live_method": classmethod(live_method),
...,
}
#classmethod could be used as a function - {"live_method": classmethod(my_method)}

Using Python unittesting libraries (unittest, mock), how to assert if a method of class B was called within a method of class A?

Assuming the following setup:
class A:
def __init__(self, nodes):
self.nodes=nodes
def update(self, bool_a=True):
if bool_a:
for n in self.nodes:
if hasattr(self.nodes[n], 'update'):
self.nodes[n].update()
class B:
def __init__(self, int_attr=5):
self.int_attr=int_attr
def update(self):
self.int_attr = 0
Let us assume that the list of nodes in class A is in fact a list of instances of class B.
How do I write a unit test for the update method of class A to check whether the update method of each class B node contained in self.nodes of class A was called?
In a more general setup, let us assume that there are multiple classes implementing update method and can be nodes within self.nodes of class A. How do I check that all update methods of self.nodes members were called?
I have tried the following, unsuccessfully:
mock_obj = MagicMock()
#patch('module.A.update', return_value=mock_obj)
def test_update(self, mock_obj):
nodes = {}
nodes['first'] = B(int_attr=1)
nodes['second'] = B(int_attr=2)
test_A = module.A(nodes=nodes)
test_A.update(bool_A=True)
self.assertTrue(mock_obj.called)
as suggested in mocking a function within a class method.
Edit: If we assume this particular case:
import unittest
import mock
from unittest import TestCase
class A:
def __init__(self, nodes):
self.nodes=nodes
def update(self, bool_a=True):
if bool_a:
to_update = [n for n in self.nodes]
while len(to_update) > 0:
if hasattr(self.nodes[to_update[-1]], 'update'):
self.nodes[to_update[-1]].update()
print('Update called.')
if self.nodes[to_update[-1]].is_updated:
to_update.pop()
class B:
def __init__(self, int_attr=5):
self.int_attr=int_attr
self.is_updated = False
def update(self):
self.int_attr = 0
self.is_updated = True
class TestEnsemble(TestCase):
def setUp(self):
self.b1 = B(1)
self.b2 = B(2)
self.b3 = B(3)
self.nodes = {}
self.nodes['1'] = self.b1
self.nodes['2'] = self.b2
self.nodes['3'] = self.b3
self.a = A(self.nodes)
#mock.patch('module.B.update')
def test_update(self, mock_update):
mock_update.return_value = None
self.a.update()
with self.subTest():
self.assertEqual(mock_update.call_count, 3)
Running unittest for this case results in an endless loop, since is_updated attribute never gets set to True because the update method of class B is mocked. How to measure the amount of time B.update was called within A.update in this case?
Update:
Tried this:
#mock.patch('dummy_script.B')
def test_update(self, mock_B):
self.a.update()
with self.subTest():
self.assertEqual(mock_B.update.call_count, 3)
The update function indeed runs 3 times now (I see it in the console output, as "Update called." is printed out three times), but the call_count of update method stays zero. Am I inspecting the wrong attribute / object?

How do I write a unit test for TestA.test_update() to see if B.update() was called?
This is just to give some ideas.
import mock
import unittest
import A
import B
class TestB(unittest.TestCase):
# only mock away update method of class B, this is python2 syntax
#mock.patch.object(B, 'update')
def test_update(self, mockb_update):
# B.update() does not return anything
mockb_update.return_value = None
nodes = {}
nodes['first'] = B(int_attr=1)
nodes['second'] = B(int_attr=2)
test_A = A(nodes)
test_A.update(bool_A=True)
self.assertTrue(mockb_update.called)
How do I check all B.update() were called for all A.nodes?
# same everthing except this
self.assertEqual(mockb_update.call_count, 2)
Running into an endless loop when B.is_udpated is not mocked after OP updated the code
mock a B.is_updated inside __init__ or mock class __init__ is a topic more complicated that the original post
Here is a few thoughts, B.is_updated cannot be just mock.patch, it is only available after the class B is initiated. so the option is to
a) mock B.__init__, or a class constructor
b) mock the whole class B, which is easier in your case, set is_updated to be True, will end the endless loop.

Combining answers from #Gang and #jonrsharpe, the following code snippets solve the questions I have asked above:
How do I write a unit test for TestA.test_update() to see if B.update() was called?
See #Gangs' answer.
How do I check all B.update() were called for all A.nodes?
See #Gangs' answer.
Running into an endless loop when B.is_udpated is not mocked after OP updated the code
As #jonrsharpe suggested, here the solution is to create one mock object per B instance in nodes and check for function calls separately:
class TestA(TestCase):
#mock.patch('module.B')
#mock.patch('module.B')
#mock.patch('module.B')
def test_update(self, mock_B1, mock_B2, mock_B3):
nodes = {}
nodes['1'] = mock_B1
nodes['2'] = mock_B2
nodes['3'] = mock_B3
a = A(nodes)
a.update()
with self.subTest():
self.assertEqual(mock_B1.update.call_count, 1)
with self.subTest():
self.assertEqual(mock_B2.update.call_count, 1)
with self.subTest():
self.assertEqual(mock_B3.update.call_count, 1)
Additionally, if you want for some reason to execute mocked functions (in case they set some flags or variables which affect runtime), one can write a test like this:
def test_fit_skip_ancestors_all(self):
nodes = {}
nodes['1'] = mock_B1
nodes['2'] = mock_B2
nodes['3'] = mock_B3
a = A(nodes)
with mock.patch.object(A.nodes['1'],'update',wraps=A.nodes['1'].update) as mock_B1, \
mock.patch.object(A.nodes['2'], 'update', wraps=A.nodes['2'].update) as mock_B2, \
mock.patch.object(A.nodes['3'], 'update', wraps=A.nodes['3'].update) as mock_B3:
a.update()
with self.subTest():
self.assertEqual(mock_B1.call_count, 1)
with self.subTest():
self.assertEqual(mock_B2.call_count, 1)
with self.subTest():
self.assertEqual(mock_B3.call_count, 1)

Python Unit Test Mocking

If you have seen my other question you will know I am having a very hard time at the moment with unit tests in Python. Two days of trying and I've made no progress.
In my method which is part of a Class there is several calls to a DAL.
car_registration = self.dal.cars.get_by_registration('121D121')
This DAL is configured in the base class. I want to completely overrid/mock these calls when running my unit tests and instead return predefined responses so I can continue with the method and ensure everything works as expected.
The method starts of with:
def change_registration(self):
body = json.loads(self.request.body)
registration = body['registration']
car = self.dal.cars.get_by_registration(registration)
My Python test file at the moment is:
class CarTestCase(unittest.TestCase):
def setUp(self):
self.car_controller = CarController()
def test_change_registrations(self):
self.car_controller.dal.cars.get_by_registration = MagicMock(return_value=3)
response = self.car_controller.change_registration()
I am expecting to get the response 3. However, an error is being thrown.
AttributeError: 'CarController' object has no attribute '_py_object'
It appears the mocking isn't working and it still trying to use the main DAL which isn't fully set up when using the unit tests. How do I prevent it for looking for the actual DAL but instead mocks?

I think you are not showing us the code that triggers the error because there is nothing wrong with your strategy. Using some imagination to mimic code we don't have I can write this and it runs with no problem:
import unittest
from unittest.mock import MagicMock
class CarList():
def get_by_registration(self, registration):
pass
class Dal:
def __init__(self):
self.cars = CarList()
pass
class CarController:
def __init__(self):
self.dal = Dal()
def change_registration(self):
registration = None
car = self.dal.cars.get_by_registration(registration)
return car
class CarTestCase(unittest.TestCase):
def setUp(self):
self.car_controller = CarController()
def test_change_registrations(self):
self.car_controller.dal.cars.get_by_registration =\
MagicMock(return_value=3)
result = self.car_controller.change_registration()
self.assertEqual(result, 3)
unittest.main()

Here my example:
# test_tool_file.py
import unittest
from unittest.mock import patch, Mock, call
import test_tools_file
class MyObject():
def __init__(self, data):
self.data = data
def get_data(self):
return self.data
def count(self):
return len(self.get_data())
class TestFile(unittest.TestCase):
""" Cas de tests.
"""
#patch("test_tools_file.MyObject.get_data")
def test_1(self, mock_get):
""" test_1
"""
mock_get.return_value = [1,2,3,4,5,6]
obj = MyObject(["12", "13"])
result = obj.count()
self.assertEqual(result, 6)

Personal opinion: I would suggest you to start simple. Use anything 'magic' once you understand what it provides you over the non-magic way. Using a non-magic solution tends to be easier to understand.
I think you have multiple simple solutions at hand. For what you tried to achieve, instead of:
self.car_controller.dal.cars.get_by_registration = MagicMock(return_value=3)
You could try:
self.car_controller.dal.cars.get_by_registration = lambda: 3
But you mentioned you want to replace all methods. In fact I would consider a 'simple' dependency injection. If you find it difficult to test, it might be a sign that the another design would be better (that's the idea of TDD - Test Driven Design). A simple dependency injection is one where you for example just pass dal to the constructor of CarController.
Here is a complete example with some variations of the test:
from unittest import TestCase
from unittest.mock import Mock
class SomeDal(object):
def get_something(self):
return 'something'
class SomeController(object):
def __init__(self, dal):
self.dal = dal
def upper_something(self):
return self.dal.get_something().upper()
class TestSomeController(TestCase):
def test_using_simple_patch(self):
controller = SomeController(SomeDal())
controller.dal.get_something = lambda: 'anything'
assert controller.upper_something() == 'ANYTHING'
def test_using_simple_patch_and_injection(self):
dal = SomeDal()
dal.get_something = lambda: 'anything'
controller = SomeController(dal)
assert controller.upper_something() == 'ANYTHING'
def test_using_simple_mock_class(self):
class MockDal(object):
def get_something(self):
return 'anything'
controller = SomeController(MockDal())
assert controller.upper_something() == 'ANYTHING'
def test_using_semi_magic_mock(self):
mock_dal = Mock(spec=SomeDal)
mock_dal.get_something.return_value = 'anything'
controller = SomeController(mock_dal)
assert controller.upper_something() == 'ANYTHING'