This question is posed as a general Python question, but will be exemplified using Kubeflow Pipelines SDK, kfp==1.8.12. In this module, I want to create a helper class around the class dsl.ContainerOp to simplify a lot of my work. As a minimal example, below I use this class to create a component as such:
from kfp import dsl
name = 'My name'
image = 'My image'
docker_entrypoint = "/main.py"
docker_args = [
'--arg1', 'some arg',
'--arg2', 'some other arg'
]
component = dsl.ContainerOp(
name=name,
image=image,
arguments=[docker_entrypoint] + docker_args
)
Then, I would like to set one of its attributes that relates to caching as such;
use_caching = False
if use_caching:
staleness = "P30D"
else:
staleness = "P0D"
component.execution_options.caching_strategy.max_cache_staleness = staleness
which works fine, as expected. Now, I would like to create a ContainerOpHelper class, to simplify a lot of my argument passing (the "real" code has a lot of parameters). Problem: I need to access the attribute execution_options.caching_strategy.max_cache_staleness from the class, but I can't figure out how! Here is the helper class, and my attempt to access the attribute;
class ContainerOpHelper(dsl.ContainerOp):
def __init__(
self,
name: str,
image: str,
docker_entrypoint: str = None,
docker_args: list = None,
use_caching: bool = None
):
super().__init__(
name=name,
image=image,
arguments=([docker_entrypoint] if docker_entrypoint else []) + (docker_args if docker_args else [])
)
if use_caching:
staleness = "P30D"
else:
staleness = "P0D"
# Tried to be creative; but doesnt work
super.__setattr__("execution_options.caching_strategy.max_cache_staleness", staleness)
This helper class can then be used as such;
component = ContainerOpHelper(
name='My name',
image='My image',
docker_entrypoint="/main.py",
docker_args=[
'--arg1', 'some arg',
'--arg2', 'some other arg'
],
use_caching=False
)
Since the attribute execution_options.caching_strategy.max_cache_staleness is "many levels deep", I'm not sure how I can set it in my helper class. Any ideas?
The solution was fairly simple, as provided by the comment of #quamrana.
Simply set it straight in the child class constructor as such;
self.execution_options.caching_strategy.max_cache_staleness = staleness
Related
I recently started to work with Python's classes, since I need to work with it through the use of OTree, a Python framework used for online experiment.
In one file, I define the pages that I want to be created, using classes. So essentially, in the OTree system, each class corresponds to a new page. The thing is, all pages (so classes) are basically the same, at the exception to some two parameters, as shown in the following code:
class Task1(Page):
form_model = 'player'
form_fields = ['Envie_WordsList_Toy']
def is_displayed(self):
return self.round_number == self.participant.vars['task_rounds'][1]
def vars_for_template(player):
WordsList_Toy= Constants.WordsList_Toy.copy()
random.shuffle(WordsList_Toy)
return dict(
WordsList_Toy=WordsList_Toy
)
#staticmethod
def live_method(player, data):
player.WTP_WordsList_Toy = int(data)
def before_next_page(self):
self.participant.vars['Envie_WordsList_Toy'] = self.player.Envie_WordsList_Toy
self.participant.vars['WTP_WordsList_Toy'] = self.player.WTP_WordsList_Toy
So here, the only thing that would change would be the name of the class, as well as the suffix of the variable WordsList_ used throughout this code, which is Toy.
Naively, what I tried to do is to define a function that would take those two parameters, such as this:
def page_creation(Task_Number,name_type):
class Task+str(Task_Number)(Page):
form_model = 'player'
form_fields = ['Envie_WordsList_'+str(name_type)]
def is_displayed(self):
return self.round_number == self.participant.vars['task_rounds'][1]
def vars_for_template(player):
WordsList_+str(name_type) = Constants.WordsList+str(name_type).copy()
random.shuffle(WordsList_+str(name_type))
return dict(
WordsList_+str(name_type)=WordsList_+str(name_type)
)
#staticmethod
def live_method(player, data):
player.WTP_WordsList_+str(name_type) = int(data)
def before_next_page(self):
self.participant.vars['Envie_WordsList_+str(name_type)'] = self.player.Envie_WordsList_+str(name_type)
self.participant.vars['WTP_WordsList_+str(name_type)'] = self.player.WTP_WordsList_+str(name_type)
Obviously, it does not work since I have the feeling that it is not possible to construct variables (or classes identifier) this way. I just started to really work on Python some weeks ago, so some of its aspects might escape me still. Could you help me on this issue? Thank you.
You can generate dynamic classes using the type constructor:
MyClass = type("MyClass", (BaseClass1, BaseClass2), {"attr1": "value1", ...})
Thus, according to your case, that would be:
cls = type(f"Task{TaskNumber}", (Page, ), {"form_fields": [f"Envive_WordList_{name_type}"], ...})
Note that you still have to construct your common methods like __init__, is_displayed and so on, as inner functions of the class factory:
def class_factory(*args, **kwargs):
...
def is_displayed(self):
return self.round_number == self.participant.vars['task_rounds']
def vars_for_template(player):
...
# Classmethod wrapping is done below
def live_method(player, data):
...
cls = type(..., {
"is_displayed": is_displayed,
"vars_for_template": vars_for_template,
"live_method": classmethod(live_method),
...,
}
#classmethod could be used as a function - {"live_method": classmethod(my_method)}
I'm super new to Python (I started about 3 weeks ago) and I'm trying to make a script that scrapes web pages for information. After it's retrieved the information it runs through a function to format it and then passes it to a class that takes 17 variables as parameters. The class uses this information to calculate some other variables and currently has a method to construct a dictionary. The code works as intended but a plugin I'm using with Pycharm called SonarLint highlights that 17 variables is too many to use as parameters?
I've had a look for alternate ways to pass the information to the class, such as in a tuple or a list but couldn't find much information that seemed relevant. What's the best practice for passing many variables to a class as parameters? Or shouldn't I be using a class for this kind of thing at all?
I've reduced the amount of variables and code for legibility but here is the class;
Class GenericEvent:
def __init__(self, type, date_scraped, date_of_event, time, link,
blurb):
countdown_delta = date_of_event - date_scraped
countdown = countdown_delta.days
if countdown < 0:
has_passed = True
else:
has_passed = False
self.type = type
self.date_scraped = date_scraped
self.date_of_event = date_of_event
self.time = time
self.link = link
self.countdown = countdown
self.has_passed = has_passed
self.blurb = blurb
def get_dictionary(self):
event_dict = {}
event_dict['type'] = self.type
event_dict['scraped'] = self.date_scraped
event_dict['date'] = self.date_of_event
event_dict['time'] = self.time
event_dict['url'] = self.link
event_dict['countdown'] = self.countdown
event_dict['blurb'] = self.blurb
event_dict['has_passed'] = self.has_passed
return event_dict
I've been passing the variables as key:value pairs to the class after I've cleaned up the data the following way:
event_info = GenericEvent(type="Lunar"
date_scraped=30/01/19
date_of_event=28/07/19
time=12:00
link="www.someurl.com"
blurb="Some string.")
and retrieving a dictionary by calling:
event_info.get_dictionary()
I intend to add other methods to the class to be able to perform other operations too (not just to create 1 dictionary) but would like to resolve this before I extend the functionality of the class.
Any help or links would be much appreciated!
One option is a named tuple:
from typing import Any, NamedTuple
class GenericEvent(NamedTuple):
type: Any
date_scraped: Any
date_of_event: Any
time: Any
link: str
countdown: Any
blurb: str
#property
def countdown(self):
countdown_delta = date_of_event - date_scraped
return countdown_delta.days
#property
def has_passed(self):
return self.countdown < 0
def get_dictionary(self):
return {
**self._asdict(),
'countdown': self.countdown,
'has_passed': self.has_passed,
}
(Replace the Anys with the fields’ actual types, e.g. datetime.datetime.)
Or, if you want it to be mutable, a data class.
I don't think there's anything wrong with what you're doing. You could, however, take your parameters in as a single dict object, and then deal with them by iterating over the dict or doing something explicitly with each one. Seems like that would, in your case, make your code messier.
Since all of your parameters to your constructor are named parameters, you could just do this:
def __init__(self, **params):
This would give you a dict named params that you could then process. The keys would be your parameter names, and the values the parameter values.
If you aligned your param names with what you want the keys to be in your get_dictionary method's return value, saving off this parameter as a whole could make that method trivial to write.
Here's an abbreviated version of your code (with a few syntax errors fixed) that illustrates this idea:
from pprint import pprint
class GenericEvent:
def __init__(self, **params):
pprint(params)
event_info = GenericEvent(type="Lunar",
date_scraped="30/01/19",
date_of_event="28/07/19",
time="12:00",
link="www.someurl.com",
blurb="Some string.")
Result:
{'blurb': 'Some string.',
'date_of_event': '28/07/19',
'date_scraped': '30/01/19',
'link': 'www.someurl.com',
'time': '12:00',
'type': 'Lunar'}
Here's the class:
class MinerNotFullAction:
def __init__(self, entity, image_store):
self.entity = entity
self.image_store = image_store
def miner_to_ore(self, world, ore):
entity_pt = entities.get_position(self.entity)
if not ore:
return ([entity_pt], False)
ore_pt = entities.get_position(ore)
obj = point.Point(0, 0)
if obj.adjacent(entity_pt, ore_pt):
entities.set_resource_count(self.entity,
1 + entities.get_resource_count(self.entity))
remove_entity(world, ore)
return ([ore_pt], True)
else:
new_pt = next_position(world, entity_pt, ore_pt)
return (worldmodel.move_entity(world, entity, new_pt), False)
And here's the function that's in the same file, but it's outside of the class:
def miner_not_full_action(world, action, ticks):
entity = action.entity
entity_pt = entities.get_position(entity)
ore = find_nearest(world, entity_pt, entities.Ore)
(tiles, found) = MinerNotFullAction.miner_to_ore(world, entity, ore)
if found:
entity = try_transform_miner(world, entity, try_transform_miner_not_full)
schedule_action(world, entity,
create_miner_action(entity, action.image_store),
ticks + entities.get_rate(entity))
return tiles
If you look at the function, def miner_not_full_action, you'll see the line: (tiles, found) = miner_to_ore(world, entity, ore). Notice that inside this function, it is calling the method, miner_to_ore (from the class that I've provided above).
My question is, what is the correct way to rewrite this line of code so that the function can use this method from the class (even though the function itself is outside of the class)? Thanks!
You can only call methods of a class from outside if you either have an object of this class or can construct one. Except, if it is a static method or a class method.
In your example, you want to use something from
class MinerNotFullAction:
def __init__(self, entity, image_store):
self.entity = entity
self.image_store = image_store
def miner_to_ore(self, world, ore):
...
in
def miner_not_full_action(world, action, ticks):
entity = action.entity
entity_pt = entities.get_position(entity)
ore = find_nearest(world, entity_pt, entities.Ore)
(tiles, found) = MinerNotFullAction.miner_to_ore(world, entity, ore)
...
So, the question is: Do you have an appropriate ection object here?
If so (action sounds like an appropriate candidate), you can do
(tiles, found) = action.miner_to_ore(world, ore)
in order to perform your task.
See it from the other side: If you call this method, you must have something to be seen as self. This should be an instance of this class. If you don't have anything like that, you have to create one. Otherwise, self would make no sense in that method.
Easy example:
def miner_not_full_action(world, action, ticks):
# 1. call it via the class. That may or may not work, but you have to provide an action object as the first argument.
try1 = MinerNotFullAction.miner_to_ore(action, world, ore)
# 2. call it regularly (as already mentioned), equivalent to 1.
try2 = action.miner_to_ore(world, ore)
# If we don't have such an object? Then we create one:
ac = MinerNotFullAction()
# and use it to call:
try3 = ac.miner_to_ore(world, ore)
I wanted to know if there is a way of populating a option menu with choices and then each of the different choice gives the build button a different function
for eg:
Type = cmds.optionMenu('type',w = 300 ,label = 'Type of crowd:')
cmds.menuItem( label='Walking' )
cmds.menuItem( label='Running' )
cmds.menuItem( label='Cheering' )
cmds.button('Build',command = bld)
def walk(*args):
print (walking)
def run(*args)
print (running)
def cheer(*args)
print (cheer)
so if the menu item selected would be walking the button command would execute the command wak
and if the menu item selected would be running then the button command would execute the command run and so on....
is this even possible in maya python...????
There's three parts to the problem.
First, you want your options to be callables. I like functools.partial for that, so that you could give the same command different parameters and have it be treated as two different actions:
from functools import partial
bigcircle = functools.partial ( cmds.circle, radius = 10)
littleCircle = functools.partial (cmds.circle, radius = 1)
the second problem is that menuItems in OptionMenus don't fire their commands directly. They trigger the -cc change command on the owning optionMenu. So we need something that will turn the label back into a callable object. A little class will do:
class menuMgr(object):
'''call the function associated with a key in the **callables dictionary'''
def __init__(self, **callables):
self.Callables = callables
def __call__(self, *args):
self.Callables[args[-1]]()
The third part is to match these with a label. You can do this elegantly with the **kwargs syntax, where you can either pass in a whole dictionary or named keywords:
def menu_of_functions(**callables):
mmgr = menuMgr(**callables)
Main = cmds.optionMenu('type3',w = 300 ,label = 'Type of crowd:', cc = mmgr)
for key, partial in callables.items():
cmds.menuItem(label = key)
cmds.setParent("..")
Heres the whole thing in working form to inspect:
import maya.cmds as cmds
import functools
bigCircle = functools.partial ( cmds.circle, radius = 10)
littleCircle = functools.partial (cmds.circle, radius = 1)
class menuMgr(object):
def __init__(self, **callables):
self.Callables = callables
def __call__(self, *args):
self.Callables[args[-1]]()
def menu_of_functions(**callables):
mmgr = menuMgr(**callables)
Main = cmds.optionMenu('type3',w = 300 ,label = 'Type of crowd:', cc = mmgr)
for key, partial in callables.items():
cmds.menuItem(label = key)
cmds.setParent("..")
q = cmds.window()
cmds.columnLayout()
menu_of_functions(big = bigCircle, small = littleCircle)
cmds.showWindow(q)
Sure you can, since functions are first class objects in python.
Let's say you have a list of functions.
fns = [walk, run, cheer]
1) You'll need a mapping from a string key to the python function.
Let's use a dictionary comprehension.
options = dict((fn.__name__, fn) for fn in fns)
Alternatively you can build the dictionary with arbitrary keys.
options = {"Walking": walk, "Running": run, "Cheering": cheer}
2) Get a reference to the function by accessing the dictionary item with the functions name.
options['run'](*args)
3) ???
4) Profit
Had a chance to do a small bit of research on this today, but I don't actually use Maya Python so this may not be viable code -- at least it should be close!
def walk(*args):
print ("walking")
def run(*args)
print ("running")
def cheer(*args)
print ("cheer")
fncdict = {"Walking":walk,"Running":run,"Cheering":cheer}
def dofunc(funcname):
try: fncdict[funcname]()
except KeyError: print("adsmith doesn't really know how this crap works,
and gave you some really shoddy advice. Go downvote
his answer.")
Type = cmds.optionMenu('type',w = 300 ,label = 'Type of crowd:')
# Please don't name things this way! naming conventions in PEP 8
# make this look like a class not an instance of optionMenu
cmds.menuItem( label='Walking', parent = Type )
cmds.menuItem( label='Running', parent = Type )
cmds.menuItem( label='Cheering', parent = Type )
cmds.button('Build',command = lambda x: dofunc(Type.value))
# I'm assuming this is the button you want to use to say "GO", right?
From the little I've read, it looks like optionMenu.value refers to the text in the active menuItem, but I can't say for sure -- it may just be the text for that optionMenu in which case the button will call dofunc('Type of crowd:') which will return the exception I built to shame myself.
Here's an alternative that I KNOW will work, but it's ugly and unnecessary.
# all
# those
# functions
# go
# here
# including
# the dict and dofunc
activeMenuItem = None
Type = cmds.optionMenu('type',w = 300 ,label = 'Type of crowd:',
changeCommand = lambda x: activeMenuItem = x)
# this fails because you can't do assignments in a lambda -- OOPS!!
cmds.menuItem( label='Walking', parent = Type )
cmds.menuItem( label='Running', parent = Type )
cmds.menuItem( label='Cheering', parent = Type )
cmds.button('Build',command = lambda x: dofunc(activeMenuItem))
The changeCommand option in optionMenu gets called every time you change items. I've assigned it to a lambda that updates the variable activeMenuItem to the value in the newly-active menuItem, then had the button reference the variable instead of querying the optionMenu for its currently selected button, but let's be honest -- this is what menus are MADE to do. There is DEFINITELY a way to do it without storing every single selection.
EDIT: THIS LAST ONE WON'T WORK BECAUSE YOU CAN'T DO ASSIGNMENTS WITHIN LAMBDA EXPRESSIONS. MY FAULT!
I have kind of figured out an easier way to this problem
cmds.optionMenu('Greetings',Label = 'Greet')
cmds.menuItem(label= hi,parent = 'Greetings)
cmds.menuItem(label = Hello,parent = 'Greetings')
def Greet(*args)
menuItems = cmds.optionMenu('Greetings',q=True,v=True)
if menuItems = hi
print "hi"
menuItemsnew = cmds.optionMenu('Greetings',q=True,v=True)
if menuItemsnew = hello
print "hello"
This should work,it worked for me
I need to construct a tool that will be used to create field mappings (between tables) in the most automated manner possible.
Here is the deal: imagine a table being appended to other. (lets ignore field type, just for a second...)
CREATE OR REPLACE TABLE fooA(
id,
name,
type,
foo)
CREATE OR REPLACE TABLE otherFooTable(
idFoo,
nameFoo,
spam)
I am thinking to create a structure like this:
fieldMap = {'otherFooTable': [('idFoo','id'),('nameFoo','name'),('spam','foo')]}
I would be able to access this using (for example)
print fieldMap['tabelax'][0][1]
It´s not a very complex structure, but i can run into some problems using it? Is there any suggestions of how to handle this sort of issue? I need to store (for now) at least inputTable (i don´t want to repeat it for each field mapped), inputField,outputField. There is no reason to store outputTable, because that is always known beforehand.
Suggestions and past experiences are deeply appreciated.
PS: perhaps a formal structure (like a class) would be better?
Thanks
I'd honestly just take hints from (or use) SQLAlchemy or Django Models. These are tried and true data representation methods.
Here is a little wrapper class for FooB's to mimic FooA's, but still retain their FooB-ishness.
from collections import namedtuple
# use namedtuple to define some simple classes (requires Py2.6 or later)
FooA = namedtuple('FooA', 'id name type foo')
FooB = namedtuple('FooB', 'idfoo namefoo spam')
# create a wrapper class for FooB's to look like a FooA
class FooAMimic(object):
attrMap = dict(zip(FooA._fields, FooB._fields))
# or if the fields aren't nicely ordered, declare this mapping explicitly
#~ attrMap = { 'id' : 'idfoo', 'name' : 'namefoo', 'foo' : 'spam' }
def __init__(self, obj):
self.obj = obj
def __getattr__(self, aname):
ob = self.obj
if aname in self.attrMap:
return getattr(ob, self.attrMap[aname])
elif hasattr(ob, aname):
return getattr(ob, aname)
else:
raise AttributeError("no such attribute " + aname)
def __dir__(self):
return sorted(set(dir(super(FooAMimic,self))
+ dir(self.obj)
+ list(FooA._fields)))
Use it like this:
# make some objects, some FooA, some FooB
fa = FooA('a', 'b', 'c','d')
fb = FooB('xx', 'yy', 'zz')
fc = FooA('e', 'f', 'g','h')
# create list of items that are FooA's, or FooA lookalikes
coll = [fa, FooAMimic(fb), fc]
# access objects like FooA's, but notice that the wrapped FooB
# attributes are still available too
for f in sorted(coll, key=lambda k : k.id):
print f.id, '=',
try:
print f.namefoo, "(really a namefoo)"
except AttributeError:
print f.name
Prints:
a = b
e = f
xx = yy (really a namefoo)
Think about this
class Column( object ):
def __init__( self, name, type_information=None ):
self.name = name
self.type_information = type_information
self.pk = None
self.fk_ref = None
def fk( self, column ):
self.fk_ref = column
class Table( object ):
def __init__( self, name, *columns ):
self.name = name
self.columns = dict( (c.name, c) for c in columns )
def column( self, name ):
return self.columns[ name ]
Table( "FOOA", Column( "id" ), Column( "name" ), Column( "type" ), Column( "foo" ) )
Table( "otherFooTable", Column( "idFoo" ), Column( "nameFoo" ), Column( "spam" ) )
It's not clear at all what you're tying to do or why, so this is as good as anything, since it seems to represent the information you actually have.
Try to avoid accessing your data through fixed numerical indexes as in fieldMap['tabelax'][0][1]. After a year of not looking at your code, it may take you (or others) a while to figure out what it all means in human terms (e.g. "the value of idFoo in table tabelax"). Also, if you ever need to change your data structure (e.g. add another field) then some/all your numerical indexes may need fixing. Your code becomes ossified when the risk of breaking the logic prevents you from modifying the data structure.
It is much better to use a class and use class (accessor) methods to access the data structure. That way, the code outside of your class can be preserved even if you need to change your data structure (inside the class) at some future date.