Ok so I got this code :
class ApiCall(object):
def __init__(self, url):
self.url = url
def call(self):
call = requests.get(self.url)
response = call.content.decode('utf-8')
result = json.loads(response)
return result
class IncomeSources(object):
def __init__(self, result):
self.result = result
def all(self):
#This is the dict comprehension
#return {(slot['accountLabelType'], slot['totalPrice']) for slot in self.result}
for slot in self.result:
return (slot['accountLabelType'], slot['totalPrice'])
def main():
url = ('https://datafeed/api/')
api_result = ApiCall(url).call()
target = IncomeSources(api_result).all()
print(target)
main()
The result with a regular for on a function, returns this wich is not desired, as it only returns the pair of the first object :
('Transport', 888)
But with the dict comprehension, it returns all slot pairs of all the json objects on that json response ( that is cool ) Why the dict comprehension grabs all the pairs and the regular for is not ?
Why the dict comprehension grabs all the pairs and the regular for is not ?
What happens when you loop over something and have a return statement in the loop is that as soon as a return statement is encountered, that value (and only that value) is returned.
The dict comprehension first constructs the entire dictionary which then gets returned as a whole to the caller.
This has less to do with the comprehension and more with the return statement. Compare:
>>> def foo():
... for i in range(5):
... return i
...
>>> foo()
0
With:
>>> def foo():
... return list(range(5))
...
>>> foo()
[0, 1, 2, 3, 4]
Related
I have several string processing functions like:
def func1(s):
return re.sub(r'\s', "", s)
def func2(s):
return f"[{s}]"
...
I want to combine them into one pipeline function: my_pipeline(), so that I can use it as an argument, for example:
class Record:
def __init__(self, s):
self.name = s
def apply_func(self, func):
return func(self.name)
rec = Record(" hell o")
output = rec.apply_func(my_pipeline)
# output = "[hello]"
The goal is to use my_pipeline as an argument, otherwise I need to call these functions one by one.
Thank you.
You can write a simple factory function or class to build a pipeline function:
>>> def pipeline(*functions):
... def _pipeline(arg):
... result = arg
... for func in functions:
... result = func(result)
... return result
... return _pipeline
...
>>> rec = Record(" hell o")
>>> rec.apply_func(pipeline(func1, func2))
'[hello]'
This is a more refined version written with reference to this using functools.reduce:
>>> from functools import reduce
>>> def pipeline(*functions):
... return lambda initial: reduce(lambda arg, func: func(arg), functions, initial)
I didn't test it, but according to my intuition, each loop will call the function one more time at the python level, so the performance may not be as good as the loop implementation.
You can just create a function which calls these functions:
def my_pipeline(s):
return func1(func2(s))
Using a list of functions (so you can assemble these elsewhere):
def func1(s):
return re.sub(r'\s', "", s)
def func2(s):
return f"[{s}]"
def func3(s):
return s + 'tada '
def callfuncs(s, pipeline):
f0 = s
pipeline.reverse()
for f in pipeline:
f0 = f(f0)
return f0
class Record:
def __init__(self, s):
self.name = s
def apply_func(self, pipeline):
return callfuncs(s.name, pipeline)
# calling order func1(func2(func3(s)))
my_pipeline = [func1, func2, func3]
rec = Record(" hell o")
output = rec.apply_func(my_pipeline)
I have the following simplified code:
class States:
def __init__(self):
pass
def state1(self):
a = 2*10
return a
def state2(self):
a = 50/10
return a
class Results:
def __init__(self):
pass
def result(self):
states = States()
x = []
for i in [state1,state2]:
state_result = states.i()
x.append(state_result)
return x
I want to loop through every function in the class "States". Of course
for i in [state1,state2]
will return "name 'state1' is not defined", but I hope it gives an idea what I try to achieve.
You can use dir() to get the name of the functions of a class. You can then use getattr() to call the function.
class States:
def __init__(self):
pass
def state1(self):
a = 2*10
return a
def state2(self):
a = 50/10
return a
state = States()
for func in dir(States):
if func.startswith('__'):
continue
print(func)
print(getattr(state, func)())
Will output
state1
20
state2
5.0
You can do this tho:
def result(self):
states = States()
x = []
for i in [states.state1,states.state2]: # changed state1 to states.state1 and so on
state_result = i()
x.append(state_result)
return x
I think you can use lambda. Here, i made a simple example for you.
def foo(text):
print(text)
a = [lambda: foo("hey"), lambda: foo("boo")]
for i in a:
i()
Result:
hey
boo
In your case, you should come over with this:
for i in [lambda: state1(), lambda:state2()]:
state_result = i()
x.append(state_result)
But if you ask my opinion, it's important to inform you that calling functions through a list is not a healthy way. A software languge usually has a solution for many cases; but in this case, i think your point of view is wrong. Doing work by messing with built-in techniques and trying to find some secret tricks is is not a suggested thing.
The clean way to do this is to "register" your state methods. SOmething like this:
class States():
states = []
def register_state(cache):
def inner(fn):
cache.append(fn)
return inner
#register_state(states)
def state1(self):
a = 2*10
return a
#register_state(states)
def state2(self):
a = 50/10
return a
Then your Results class can do
class Results:
def __init__(self):
pass
def result(self):
states = States()
x = []
for i in states.states:
state_result = i(states)
x.append(state_result)
return x
You can get the members of class States via the class' dict as:
States.__dict__
Which'll give you all the attributes and function of your class as:
{'__module__': '__main__', '__init__': <function States.__init__ at 0x00000183066F0A60>, 'state1': <function States.state1 at 0x00000183066F0AF0>, 'state2': <function States.state2 at 0x000001830 ...
You can filter this into a list comprehension dict to not include dunders as:
[funcname for funcname in States.__dict__ if not (str.startswith('__') and str.endswith('__'))]
This will return you a list of member functions as:
['state1', 'state2']
Then create an object of States as:
states = States()
get the whole calculation done as:
for funcname in [funcname for funcname in States.__dict__ if not (funcname.startswith('__') and funcname.endswith('__'))]:
x.append(States.__dict__[funcname](states))
Better yet, make it a comprehension as:
[States.__dict__[funcname](states) for funcname in States.__dict__ if not (funcname.startswith('__') and funcname.endswith('__'))]
Your answer after applying this approach is: [20, 5.0]
or get the dict of functionName and returnValues as a comprehension:
{funcname: States.__dict__[funcname](states) for funcname in States.__dict__ if not (funcname.startswith('__') and funcname.endswith('__'))}
Which'll give you an output as:
{'state1': 20, 'state2': 5.0}
What is a good design pattern to implement templated object generation (not sure that's the name) in python?
By that, I mean having a function such as:
from typing import TypeVar
T = TypeVar('T')
def mk_templated_obj_factory(template: T) -> Callable[..., T]:
"""Returns a f(**kwargs) function that returns an object of type T created by a template of the same type."""
Python has templated strings. Something like `"this {is} a {template}".format' would be how one could achieve the above. If we want to get a "proper" function that has a signature (useful for a user so they know what arguments they need to provide!), we could do this:
from inspect import signature, Signature, Parameter
from operator import itemgetter
from typing import Callable
f = "hello {name} how are you {verb}?".format
def templated_string_func(template: str) -> Callable:
"""A function making templated strings. Like template.format, but with a signature"""
f = partial(str.format, template)
names = filter(None, map(itemgetter(1), string.Formatter().parse(template)))
params = [Parameter(name=name, kind=Parameter.KEYWORD_ONLY) for name in names]
f.__signature__ = Signature(params)
return f
f = templated_string_func("hello {name} how are you {verb}?")
assert f(name='Christian', verb='doing') == 'hello Christian how are you doing?'
assert str(signature(f)) == '(*, name, verb)'
But would if we want to make dict factories? Something having this behavior:
g = templated_dict_func(template={'hello': '$name', 'how are you': ['$verb', 2]})
assert g(name='Christian', verb='doing') == {'hello': '$name', 'how are you': ['doing', 2]}
What about other types of objects?
It seems like something that would have a solid design pattern...
I would recommend using decorators to register your template function generating functions in a dictionary that maps from types to the functions that handle them. The dictionary is needed in order to be able to template objects of any type in an extensible way, without writing all the templating logic in a single big function, but instead adding handling logic for new types as needed.
The core code is in the Templater class, just grouped here for organisation:
class Templater:
templater_registry: dict[type, Callable[[Any], TemplateFunc]] = {}
#classmethod
def register(cls, handles_type: type):
def decorator(f):
cls.templater_registry[handles_type] = f
return f
return decorator
...
Where TemplateFunc is defined as Generator[str, None, Callable[..., T]], a generator that yields strs and returns a function that returns some type T. This is chosen so that the template handlers can yield the names of their keyword arguments and then return their template function. The Templater.template_func method uses a something of type TemplateFunc to generate a function with the correct signature.
The register decorator presented above is written such that:
#Templater.register(dict)
def templated_dict_func(template: dict[K, V]):
pass
is equivalent to:
def templated_dict_func(template: dict[K, V]):
pass
Templater.templater_registry[dict] = templated_dict_func
The code for templating any type is fairly self-explainatory:
class Templater:
...
#classmethod
def template_func_generator(cls, template: T) -> TemplateFunc[T]:
# if it is a type that can be a template
if type(template) in cls.templater_registry:
# then return the template handler
template_factory = cls.templater_registry[type(template)]
return template_factory(template)
else:
# else: an empty generator that returns a function that returns the template unchanged,
# since we don't know how to handle it
def just_return():
return lambda: template
yield # this yield is needed to tell python that this is a generator
return just_return()
The code for templating strings is fairly unchanged, except that the argument names are yielded instead of put in the function signature:
#Templater.register(str)
def templated_string_func(template: str) -> TemplateFunc[str]:
"""A function making templated strings. Like template.format, but with a signature"""
f = partial(str.format, template)
yield from filter(None, map(itemgetter(1), string.Formatter().parse(template)))
return f
The list template function could look like this:
#Templater.register(list)
def templated_list_func(template: list[T]) -> TemplateFunc[list[T]]:
entries = []
for item in template:
item_template_func = yield from Templater.template_func_generator(item)
entries.append(item_template_func)
def template_func(**kwargs):
return [
item_template_func(**kwargs)
for item_template_func in entries
]
return template_func
Although, if you cannot guarantee that every template function can handle extra arguments, you need to track which arguments belong to which elements and only pass the necessary ones. I use the get_generator_return utility function (defined later on) to capture both the yielded values and the return value of the recursive calls.
#Templater.register(list)
def templated_list_func(template: list[T]) -> TemplateFunc[list[T]]:
entries = []
for item in template:
params, item_template_func = get_generator_return(Templater.template_func_generator(item))
params = tuple(params)
yield from params
entries.append((item_template_func, params))
def template_func(**kwargs):
return [
item_template_func(**{arg: kwargs[arg] for arg in args})
for item_template_func, args in entries
]
return template_func
The dict handler is implemented similarly. This system could be extended to support all kinds of different objects, including arbitrary dataclasses and more, but I leave that as an exercise for the reader!
Here is the entire working example:
import string
from functools import partial
from inspect import Signature, Parameter
from operator import itemgetter
from typing import Callable, Any, TypeVar, Generator, Tuple, Dict, List
from collections import namedtuple
T = TypeVar('T')
U = TypeVar('U')
def get_generator_return(gen: Generator[T, Any, U]) -> Tuple[Generator[T, Any, U], U]:
return_value = None
def inner():
nonlocal return_value
return_value = yield from gen
gen_items = list(inner())
def new_gen():
yield from gen_items
return return_value
return new_gen(), return_value
# TemplateFunc: TypeAlias = Generator[str, None, Callable[..., T]]
TemplateFunc = Generator[str, None, Callable[..., T]]
class Templater:
templater_registry: Dict[type, Callable[[Any], TemplateFunc]] = {}
#classmethod
def register(cls, handles_type: type):
def decorator(f):
cls.templater_registry[handles_type] = f
return f
return decorator
#classmethod
def template_func_generator(cls, template: T) -> TemplateFunc[T]:
if type(template) in cls.templater_registry:
template_factory = cls.templater_registry[type(template)]
return template_factory(template)
else:
# an empty generator that returns a function that returns the template unchanged,
# since we don't know how to handle it
def just_return():
return lambda: template
yield # this yield is needed to tell python that this is a generator
return just_return()
#classmethod
def template_func(cls, template: T) -> Callable[..., T]:
gen = cls.template_func_generator(template)
params, f = get_generator_return(gen)
f.__signature__ = Signature(Parameter(name=param, kind=Parameter.KEYWORD_ONLY) for param in params)
return f
#Templater.register(str)
def templated_string_func(template: str) -> TemplateFunc[str]:
"""A function making templated strings. Like template.format, but with a signature"""
f = partial(str.format, template)
yield from filter(None, map(itemgetter(1), string.Formatter().parse(template)))
return f
K = TypeVar('K')
V = TypeVar('V')
#Templater.register(dict)
def templated_dict_func(template: Dict[K, V]) -> TemplateFunc[Dict[K, V]]:
DictEntryInfo = namedtuple('DictEntryInfo', ['key_func', 'value_func', 'key_args', 'value_args'])
entries: list[DictEntryInfo] = []
for key, value in template.items():
key_params, key_template_func = get_generator_return(Templater.template_func_generator(key))
value_params, value_template_func = get_generator_return(Templater.template_func_generator(value))
key_params = tuple(key_params)
value_params = tuple(value_params)
yield from key_params
yield from value_params
entries.append(DictEntryInfo(key_template_func, value_template_func, key_params, value_params))
def template_func(**kwargs):
return {
entry_info.key_func(**{arg: kwargs[arg] for arg in entry_info.key_args}):
entry_info.value_func(**{arg: kwargs[arg] for arg in entry_info.value_args})
for entry_info in entries
}
return template_func
#Templater.register(list)
def templated_list_func(template: List[T]) -> TemplateFunc[List[T]]:
entries = []
for item in template:
params, item_template_func = get_generator_return(Templater.template_func_generator(item))
params = tuple(params)
yield from params
entries.append((item_template_func, params))
def template_func(**kwargs):
return [
item_template_func(**{arg: kwargs[arg] for arg in args})
for item_template_func, args in entries
]
return template_func
g = Templater.template_func(template={'hello': '{name}', 'how are you': ['{verb}', 2]})
assert g(name='Christian', verb='doing') == {'hello': 'Christian', 'how are you': ['doing', 2]}
print(g.__signature__)
If this is my code:
x = 1
x = 2
x = 3
How can I “log” the things x has been and print them? If my explanation was dumb, then here’s what I expect:
>>> # Code to print the things x has been
1, 2, 3
>>>
How can I achieve this?
Since assignment overwrites the value of the object (in your example 'x'), it is not possible to do exactly what you want. However, you could create an object, of which the value can be changed and its history remembered. For example like this:
#!/usr/bin/env/python3
class ValueWithHistory():
def __init__(self):
self.history = []
self._value = None
#property
def value(self):
return self._value
#value.setter
def value(self, new_value):
self.history.append(new_value)
self._value = new_value
def get_history(self):
return self.history
def clear_history(self):
self.history.clear()
def main():
test = ValueWithHistory()
test.value = 1
print(test.value)
test.value = 2
print(test.value)
test.value = 3
print(test.value)
print(test.get_history())
if __name__ == '__main__':
main()
This prints:
1
2
3
[1, 2, 3]
Of course, you could also use a set instead of a list to only remember each unique value once, for example.
You can order a second thread to observe the string and print the changes:
from threading import Thread
def string_watcher():
global my_string
global log
temp = ''
while True:
if my_string != temp:
log.append(my_string)
temp = my_string
t = Thread(target=string_watcher, daemon=True)
t.start()
This checks weather the string „my_string“ was manipulated and appends it to the list „log“, if it has been changed. With this you should be able to perform
Print(log)
At any moment of the runtime
When using the following code only for demonstrative purposes:
from uuid import uuid4
class router(object):
def route(self):
res = response(jid=str(uuid4()))
worker = resource()
worker.dispatch(res)
print '[jid: %s, status: %s]' % (res.jid, res.status)
class response(object):
def __init__(self, jid):
self.jid = jid
self.status = 0
class resource(object):
def __init__(self):
self.status = 200
def dispatch(self, res):
res.status = self.status
rs = 'ok'
#return rs
yield rs
app = router()
app.route()
If using return rs instead of yield rs I can update the value of status within the dispatch(self, res) method of the resource class, out put is something like:
[jid: 575fb301-1aa9-40e7-a077-87887c8be284, status: 200]
But if using yield rs I can't update the value of status, I am getting always the original value, example:
[jid: 575fb301-1aa9-40e7-a077-87887c8be284, status: 0]
Therefore I would like to know, how to update the object variables of an object passed as a reference, when using yield.
You need to iterate the generator. Otherwise the generator is not executed.
>>> def gen():
... print(1)
... yield 'blah'
... print(2)
...
>>> g = gen() # No print (not executed)
>>> next(g) # print 1, yield `blah`. execution suspended.
1
'blah'
Replace following line:
worker.dispatch(res)
with:
for rs in worker.dispatch(res):
pass
or
next(worker.dispatch(res))
By using yield you are telling python that your dispatch() method is a generator.
So when you call worker.dispatch(res), nothing actually happens (try to call print worker.dispatch(res), you'll see just the object reference).
You have to iterate over it as mentioned by falsetru.
See also the Python yield keyword explained