We Keep Coding

Set parent class attribute in child class constructor

This question is posed as a general Python question, but will be exemplified using Kubeflow Pipelines SDK, kfp==1.8.12. In this module, I want to create a helper class around the class dsl.ContainerOp to simplify a lot of my work. As a minimal example, below I use this class to create a component as such:
from kfp import dsl
name = 'My name'
image = 'My image'
docker_entrypoint = "/main.py"
docker_args = [
'--arg1', 'some arg',
'--arg2', 'some other arg'
]
component = dsl.ContainerOp(
name=name,
image=image,
arguments=[docker_entrypoint] + docker_args
)
Then, I would like to set one of its attributes that relates to caching as such;
use_caching = False
if use_caching:
staleness = "P30D"
else:
staleness = "P0D"
component.execution_options.caching_strategy.max_cache_staleness = staleness
which works fine, as expected. Now, I would like to create a ContainerOpHelper class, to simplify a lot of my argument passing (the "real" code has a lot of parameters). Problem: I need to access the attribute execution_options.caching_strategy.max_cache_staleness from the class, but I can't figure out how! Here is the helper class, and my attempt to access the attribute;
class ContainerOpHelper(dsl.ContainerOp):
def __init__(
self,
name: str,
image: str,
docker_entrypoint: str = None,
docker_args: list = None,
use_caching: bool = None
):
super().__init__(
name=name,
image=image,
arguments=([docker_entrypoint] if docker_entrypoint else []) + (docker_args if docker_args else [])
)
if use_caching:
staleness = "P30D"
else:
staleness = "P0D"
# Tried to be creative; but doesnt work
super.__setattr__("execution_options.caching_strategy.max_cache_staleness", staleness)
This helper class can then be used as such;
component = ContainerOpHelper(
name='My name',
image='My image',
docker_entrypoint="/main.py",
docker_args=[
'--arg1', 'some arg',
'--arg2', 'some other arg'
],
use_caching=False
)
Since the attribute execution_options.caching_strategy.max_cache_staleness is "many levels deep", I'm not sure how I can set it in my helper class. Any ideas?

The solution was fairly simple, as provided by the comment of #quamrana.
Simply set it straight in the child class constructor as such;
self.execution_options.caching_strategy.max_cache_staleness = staleness

How do I use variables with classes for python?

There is probably an stupidly obvious solution to this but I'm new to python and can't find it. I'm working out a few of the systems for a practice project I'm working on and I can't seem to get this to work:
class Item:
def __init__(self, name, description, type, mindamage, maxdamage):
self.name = name
self.desc = description
self.type = type
self.mindmg = mindamage
self.maxdmg = maxdamage
woodsman = Item("'Woodsman'", "An automatic chambered in .22lr", "gun", 4, 10)
inspect = input("inspect:").lower()
print(inspect.name)
print(inspect.desc)
print(inspect.type)
I can't find a solution to this for some reason.

Use dataclasses and items dict:
from dataclasses import dataclass
#dataclass
class Item:
name: str
description: str
item_type: str # don't use 'type' for variables name, it's reserved name
min_damage: int
max_damage: int
woodsman = Item(
name="'Woodsman'",
description="An automatic chambered in .22lr",
item_type="gun",
min_damage=4,
max_damage=10
)
# other items...
items = {
"woodsman": woodsman,
# other items...
}
inspect = items.get(input("inspect:").lower())
print(inspect.name)
print(inspect.description)
print(inspect.item_type)

This might be closer to what you're trying to do:
inventory = {
"woodsman": Item("'Woodsman'","An automatic chambered in .22lr","gun",4,10)
}
inspect = inventory[input("inspect:").lower()]
print(inspect.name)
print(inspect.desc)
print(inspect.type)
Note that you will probably want to have some kind of error handling in case the user enters an item that doesn't exist in the inventory.

I was fiddling around and found another solution that works for me:
inspect = input("inspect:").lower()
exec("print(" + inspect + ".name)")

Method __init__ has too many parameters

I'm super new to Python (I started about 3 weeks ago) and I'm trying to make a script that scrapes web pages for information. After it's retrieved the information it runs through a function to format it and then passes it to a class that takes 17 variables as parameters. The class uses this information to calculate some other variables and currently has a method to construct a dictionary. The code works as intended but a plugin I'm using with Pycharm called SonarLint highlights that 17 variables is too many to use as parameters?
I've had a look for alternate ways to pass the information to the class, such as in a tuple or a list but couldn't find much information that seemed relevant. What's the best practice for passing many variables to a class as parameters? Or shouldn't I be using a class for this kind of thing at all?
I've reduced the amount of variables and code for legibility but here is the class;
Class GenericEvent:
def __init__(self, type, date_scraped, date_of_event, time, link,
blurb):
countdown_delta = date_of_event - date_scraped
countdown = countdown_delta.days
if countdown < 0:
has_passed = True
else:
has_passed = False
self.type = type
self.date_scraped = date_scraped
self.date_of_event = date_of_event
self.time = time
self.link = link
self.countdown = countdown
self.has_passed = has_passed
self.blurb = blurb
def get_dictionary(self):
event_dict = {}
event_dict['type'] = self.type
event_dict['scraped'] = self.date_scraped
event_dict['date'] = self.date_of_event
event_dict['time'] = self.time
event_dict['url'] = self.link
event_dict['countdown'] = self.countdown
event_dict['blurb'] = self.blurb
event_dict['has_passed'] = self.has_passed
return event_dict
I've been passing the variables as key:value pairs to the class after I've cleaned up the data the following way:
event_info = GenericEvent(type="Lunar"
date_scraped=30/01/19
date_of_event=28/07/19
time=12:00
link="www.someurl.com"
blurb="Some string.")
and retrieving a dictionary by calling:
event_info.get_dictionary()
I intend to add other methods to the class to be able to perform other operations too (not just to create 1 dictionary) but would like to resolve this before I extend the functionality of the class.
Any help or links would be much appreciated!

One option is a named tuple:
from typing import Any, NamedTuple
class GenericEvent(NamedTuple):
type: Any
date_scraped: Any
date_of_event: Any
time: Any
link: str
countdown: Any
blurb: str
#property
def countdown(self):
countdown_delta = date_of_event - date_scraped
return countdown_delta.days
#property
def has_passed(self):
return self.countdown < 0
def get_dictionary(self):
return {
**self._asdict(),
'countdown': self.countdown,
'has_passed': self.has_passed,
}
(Replace the Anys with the fields’ actual types, e.g. datetime.datetime.)
Or, if you want it to be mutable, a data class.

I don't think there's anything wrong with what you're doing. You could, however, take your parameters in as a single dict object, and then deal with them by iterating over the dict or doing something explicitly with each one. Seems like that would, in your case, make your code messier.
Since all of your parameters to your constructor are named parameters, you could just do this:
def __init__(self, **params):
This would give you a dict named params that you could then process. The keys would be your parameter names, and the values the parameter values.
If you aligned your param names with what you want the keys to be in your get_dictionary method's return value, saving off this parameter as a whole could make that method trivial to write.
Here's an abbreviated version of your code (with a few syntax errors fixed) that illustrates this idea:
from pprint import pprint
class GenericEvent:
def __init__(self, **params):
pprint(params)
event_info = GenericEvent(type="Lunar",
date_scraped="30/01/19",
date_of_event="28/07/19",
time="12:00",
link="www.someurl.com",
blurb="Some string.")
Result:
{'blurb': 'Some string.',
'date_of_event': '28/07/19',
'date_scraped': '30/01/19',
'link': 'www.someurl.com',
'time': '12:00',
'type': 'Lunar'}

Is there I can use ANIMAL_CHOICES.RABBIT to get value in Python?

I have a requirement to use the data like bellow:
I am in assumption:
ANIMAL_CHOICES = {
"RABBIT":"RABBIT",
"CAMEL":"CAMEL",
}
print(ANIMAL_CHOICES["RABBIT"]) # there I can get the "RABBIT", and can see the RABBIT easily.
You see, I can put the ANIMAL_CHOICES to a single file , when I use it I can use ANIMAL_CHOICES["RABBIT"] to do it. I can easily know the value by its key in other files, but there is a defect, and I will go to ANIMAL_CHOICES source to see the key.
So, whether there is a more tidy method to get this effect?
I mean whether I can use ANIMAL_CHOICES.RABBIT to get the "RABBIT"?
EDIT
I mean is there other python data type we can use to get the effect? you see the Dictionary can not reach the requirement.

Just use a class as a namespace:
class AnimalChoices(object):
RABBIT = "RABBIT"
CAMEL = "CAMEL"
print(AnimalChoices.RABBIT)

class My_dict(dict):
def __getattribute__(self, name):
return self[name]
d = My_dict({
"RABBIT": "RABBIT",
"CAMEL": "CAMEL",
})
print(d.RABBIT)

You could define your own animal_choices class that generates it's properties based on the keys of your data structure.
class animal_properties:
def __init__(self, dictionary):
for k, v in dictionary.items():
setattr(self, k, v)
animal = animal_properties({"RABBIT":"RABBIT", "CAMEL":"CAMEL"})
animal.RABBIT

With a class it would work:
class AnimalChoices:
def __init__(self):
self.RABBIT = 'RABBIT'
self.CAMEL = 'CAMEL'
animal_choices = AnimalChoices()
print(animal_choices.RABBIT)
animal_choices.DOG = 'DOG'
print(animal_choices.DOG)

Best way to construct a "complex" data structure in Python

I need to construct a tool that will be used to create field mappings (between tables) in the most automated manner possible.
Here is the deal: imagine a table being appended to other. (lets ignore field type, just for a second...)
CREATE OR REPLACE TABLE fooA(
id,
name,
type,
foo)
CREATE OR REPLACE TABLE otherFooTable(
idFoo,
nameFoo,
spam)
I am thinking to create a structure like this:
fieldMap = {'otherFooTable': [('idFoo','id'),('nameFoo','name'),('spam','foo')]}
I would be able to access this using (for example)
print fieldMap['tabelax'][0][1]
It´s not a very complex structure, but i can run into some problems using it? Is there any suggestions of how to handle this sort of issue? I need to store (for now) at least inputTable (i don´t want to repeat it for each field mapped), inputField,outputField. There is no reason to store outputTable, because that is always known beforehand.
Suggestions and past experiences are deeply appreciated.
PS: perhaps a formal structure (like a class) would be better?
Thanks

I'd honestly just take hints from (or use) SQLAlchemy or Django Models. These are tried and true data representation methods.

Here is a little wrapper class for FooB's to mimic FooA's, but still retain their FooB-ishness.
from collections import namedtuple
# use namedtuple to define some simple classes (requires Py2.6 or later)
FooA = namedtuple('FooA', 'id name type foo')
FooB = namedtuple('FooB', 'idfoo namefoo spam')
# create a wrapper class for FooB's to look like a FooA
class FooAMimic(object):
attrMap = dict(zip(FooA._fields, FooB._fields))
# or if the fields aren't nicely ordered, declare this mapping explicitly
#~ attrMap = { 'id' : 'idfoo', 'name' : 'namefoo', 'foo' : 'spam' }
def __init__(self, obj):
self.obj = obj
def __getattr__(self, aname):
ob = self.obj
if aname in self.attrMap:
return getattr(ob, self.attrMap[aname])
elif hasattr(ob, aname):
return getattr(ob, aname)
else:
raise AttributeError("no such attribute " + aname)
def __dir__(self):
return sorted(set(dir(super(FooAMimic,self))
+ dir(self.obj)
+ list(FooA._fields)))
Use it like this:
# make some objects, some FooA, some FooB
fa = FooA('a', 'b', 'c','d')
fb = FooB('xx', 'yy', 'zz')
fc = FooA('e', 'f', 'g','h')
# create list of items that are FooA's, or FooA lookalikes
coll = [fa, FooAMimic(fb), fc]
# access objects like FooA's, but notice that the wrapped FooB
# attributes are still available too
for f in sorted(coll, key=lambda k : k.id):
print f.id, '=',
try:
print f.namefoo, "(really a namefoo)"
except AttributeError:
print f.name
Prints:
a = b
e = f
xx = yy (really a namefoo)

Think about this
class Column( object ):
def __init__( self, name, type_information=None ):
self.name = name
self.type_information = type_information
self.pk = None
self.fk_ref = None
def fk( self, column ):
self.fk_ref = column
class Table( object ):
def __init__( self, name, *columns ):
self.name = name
self.columns = dict( (c.name, c) for c in columns )
def column( self, name ):
return self.columns[ name ]
Table( "FOOA", Column( "id" ), Column( "name" ), Column( "type" ), Column( "foo" ) )
Table( "otherFooTable", Column( "idFoo" ), Column( "nameFoo" ), Column( "spam" ) )
It's not clear at all what you're tying to do or why, so this is as good as anything, since it seems to represent the information you actually have.

Try to avoid accessing your data through fixed numerical indexes as in fieldMap['tabelax'][0][1]. After a year of not looking at your code, it may take you (or others) a while to figure out what it all means in human terms (e.g. "the value of idFoo in table tabelax"). Also, if you ever need to change your data structure (e.g. add another field) then some/all your numerical indexes may need fixing. Your code becomes ossified when the risk of breaking the logic prevents you from modifying the data structure.
It is much better to use a class and use class (accessor) methods to access the data structure. That way, the code outside of your class can be preserved even if you need to change your data structure (inside the class) at some future date.