This question already has answers here:
Can modules have properties the same way that objects can?
(8 answers)
Closed 6 years ago.
This probably already looks like a duplicate; here's the scenario:
default_config.py:
unit_id = -1 # a serial number; int. -1 is a test unit, for example
def um():
return unit_id % 60 # stagger uploads
upload_hour = 2 #am
upload_minute = property( um ) # <- something that works needed here...
config.py
from default_config import *
# Override defaults here, if necessary
unit_id = 12 # ACTUAL serial number...
some_file.py
import config as cfg
do_something(cfg.upload_hour, cfg.upload_minute)
print cfg.upload_minute * 5 # should be an int...?
So, the goals are:
A specific config file can override the defaults, which works fine
Some values which are calculated can be accessed - after the overrides are applied - but in a "transparent" way (ie. without the prop() brackets)
This seemed to be simple for python properties, but after various combinations, doesn't work. I guess its something to do with the function being defined on a module, not an object, and unbound first variables, etc...
Either I get a "property object" back, and can't then use operators on it, etc, or I can't get the value the property should calculate and return, or after many iterations I can't remember, some other error...
I guess its something to do with the function being defined on a
module, not an object...
Why not use an object then? :)
default_config.py
_DEFAULT_UNIT_ID = -1
_DEFAULT_UPLOAD_HOUR = 2
_MINUTES_PER_HOUR = 60
class BaseConfig(object):
def __init__(self, unit_id=_DEFAULT_UNIT_ID, upload_hour=_DEFAULT_UPLOAD_HOUR):
self.unit_id = unit_id
self.upload_hour = upload_hour
#property
def upload_minute(self):
return self.unit_id % _MINUTES_PER_HOUR
config.py
from default_config import BaseConfig
# organized place to put non-default parameters
config_dict = {
'unit_id': 12,
'upload_hour': 3,
}
CONFIG = BaseConfig(**config_dict)
some_file.py
from config import CONFIG
print CONFIG.upload_hour, CONFIG.upload_minute # "3 12"
I would also consider just combining default_config.py and config.py if there's no need to separate them, since it would be easier to see what keyword arguments the BaseConfig takes.
You can't define special methods on modules, but entries in sys.modules don't have to be module objects, they can also be class instance objects. This means you can take advantage of their attribute-access special methods like this:
default_config.py
class DefaultConfig(object):
unit_id = -1 # a serial number
upload_hour = 2 # am
#property
def upload_minute(self):
return self.unit_id % 60 # stagger uploads
config.py
import sys
from default_config import DefaultConfig
# override defaults
DefaultConfig.unit_id = 12
# see http://stackoverflow.com/questions/5365562/why-is-the-value-of-name-changing-after-assignment-to-sys-modules-name
# as to why the _ref is necessary
_ref, sys.modules[__name__] = sys.modules[__name__], DefaultConfig()
# clean up this module's namespace
del sys, DefaultConfig
some_file.py
from __future__ import print_function
import config as cfg
def do_something(hour, minute):
print('do_something({}, {}) called'.format(hour, minute))
do_something(cfg.upload_hour, cfg.upload_minute)
print(cfg.upload_minute * 5)
Output from running some_file.py:
do_something(2, 12) called
60
Related
I'm super new to Python (I started about 3 weeks ago) and I'm trying to make a script that scrapes web pages for information. After it's retrieved the information it runs through a function to format it and then passes it to a class that takes 17 variables as parameters. The class uses this information to calculate some other variables and currently has a method to construct a dictionary. The code works as intended but a plugin I'm using with Pycharm called SonarLint highlights that 17 variables is too many to use as parameters?
I've had a look for alternate ways to pass the information to the class, such as in a tuple or a list but couldn't find much information that seemed relevant. What's the best practice for passing many variables to a class as parameters? Or shouldn't I be using a class for this kind of thing at all?
I've reduced the amount of variables and code for legibility but here is the class;
Class GenericEvent:
def __init__(self, type, date_scraped, date_of_event, time, link,
blurb):
countdown_delta = date_of_event - date_scraped
countdown = countdown_delta.days
if countdown < 0:
has_passed = True
else:
has_passed = False
self.type = type
self.date_scraped = date_scraped
self.date_of_event = date_of_event
self.time = time
self.link = link
self.countdown = countdown
self.has_passed = has_passed
self.blurb = blurb
def get_dictionary(self):
event_dict = {}
event_dict['type'] = self.type
event_dict['scraped'] = self.date_scraped
event_dict['date'] = self.date_of_event
event_dict['time'] = self.time
event_dict['url'] = self.link
event_dict['countdown'] = self.countdown
event_dict['blurb'] = self.blurb
event_dict['has_passed'] = self.has_passed
return event_dict
I've been passing the variables as key:value pairs to the class after I've cleaned up the data the following way:
event_info = GenericEvent(type="Lunar"
date_scraped=30/01/19
date_of_event=28/07/19
time=12:00
link="www.someurl.com"
blurb="Some string.")
and retrieving a dictionary by calling:
event_info.get_dictionary()
I intend to add other methods to the class to be able to perform other operations too (not just to create 1 dictionary) but would like to resolve this before I extend the functionality of the class.
Any help or links would be much appreciated!
One option is a named tuple:
from typing import Any, NamedTuple
class GenericEvent(NamedTuple):
type: Any
date_scraped: Any
date_of_event: Any
time: Any
link: str
countdown: Any
blurb: str
#property
def countdown(self):
countdown_delta = date_of_event - date_scraped
return countdown_delta.days
#property
def has_passed(self):
return self.countdown < 0
def get_dictionary(self):
return {
**self._asdict(),
'countdown': self.countdown,
'has_passed': self.has_passed,
}
(Replace the Anys with the fields’ actual types, e.g. datetime.datetime.)
Or, if you want it to be mutable, a data class.
I don't think there's anything wrong with what you're doing. You could, however, take your parameters in as a single dict object, and then deal with them by iterating over the dict or doing something explicitly with each one. Seems like that would, in your case, make your code messier.
Since all of your parameters to your constructor are named parameters, you could just do this:
def __init__(self, **params):
This would give you a dict named params that you could then process. The keys would be your parameter names, and the values the parameter values.
If you aligned your param names with what you want the keys to be in your get_dictionary method's return value, saving off this parameter as a whole could make that method trivial to write.
Here's an abbreviated version of your code (with a few syntax errors fixed) that illustrates this idea:
from pprint import pprint
class GenericEvent:
def __init__(self, **params):
pprint(params)
event_info = GenericEvent(type="Lunar",
date_scraped="30/01/19",
date_of_event="28/07/19",
time="12:00",
link="www.someurl.com",
blurb="Some string.")
Result:
{'blurb': 'Some string.',
'date_of_event': '28/07/19',
'date_scraped': '30/01/19',
'link': 'www.someurl.com',
'time': '12:00',
'type': 'Lunar'}
I am migrating a project I have from being littered with globals variables to actually have a structure defined by classes defined in a separate module. This is my first time really using OOP so want to understand if it is safe to re-define an instance of a Class or if my code is missing something.
At the top of my code, I import my module -
import NHLGameEvents
config = configparser.ConfigParser()
config.read('config.ini')
TEAM_BOT = config['DEFAULT']['TEAM_NAME']
I then build two Team objects (defined in my NHLGameEvents module).
game_today, game_info = is_game_today(get_team(TEAM_BOT))
awayteam_info = game_info["teams"]["away"]["team"]
awayteamobj_name = awayteam_info["name"]
awayteamobj_shortname = awayteam_info["teamName"]
awayteamobj_tri = awayteam_info["abbreviation"]
away_team_obj = NHLGameEvents.Team(
awayteamobj_name, awayteamobj_shortname, awayteamobj_tri, "away")
game_obj.register_team(away_team_obj, "away")
hometeam_info = game_info["teams"]["home"]["team"]
hometeamobj_name = hometeam_info["name"]
hometeamobj_shortname = hometeam_info["teamName"]
hometeamobj_tri = hometeam_info["abbreviation"]
home_team_obj = NHLGameEvents.Team(
hometeamobj_name, hometeamobj_shortname, hometeamobj_tri, "home")
game_obj.register_team(home_team_obj, "home")
home_team_obj.preferred = bool(home_team_obj.team_name == TEAM_BOT)
away_team_obj.preferred = bool(away_team_obj.team_name == TEAM_BOT)
In some instances, I want to reference these Team objects as preferred and other as opposed to home / away so I use a method defined in my Game class to retrieve that. Since my Game object knows about both of my Teams, the method in my Game class that returns this Tuple is -
def register_team(self, team, key):
"""Registers a team to the instance of the Game."""
if key not in ('home', 'away'):
raise AttributeError(
"Key '{}' is not valid - Team key can only be home or away.".format(key))
if len(self.teams) > 1:
raise ValueError(
"Too many teams! Cannot register {} for {}".format(team, self))
self.teams[key] = team
team.game = self
team.tv_channel = self.broadcasts[key]
def get_preferred_team(self):
"""Returns a Tuple of team objects of the preferred & other teams."""
if self.teams["home"].preferred is True:
return (self.teams["home"], self.teams["away"])
return (self.teams["away"], self.teams["home"])
I can then retrieve that information from anywhere in my script.
preferred_team_obj, other_team_obj = game_obj.get_preferred_team()
Is it safe to redefine these class instances (ex - home_team_obj also known as preferred_team_obj) or should I just use an if statement whenever I want to reference these, such as -
if home_team_obj.preferred:
# Do something with home_team_obj
else:
# Do something with away_team_obj
Just as a follow up to this question, it seems that is totally safe to refer to assign an object to another name for use later in the code with no issues (as per the example below).
preferred_team = game.preferred_team
preferred_homeaway = preferred_team.home_away
on_ice = json_feed["liveData"]["boxscore"]["teams"][preferred_homeaway]["onIce"]
players = json_feed["gameData"]["players"]
if recent_event(play):
get_lineup(game, event_period, on_ice, players)
I am using the amazing attrs library to define a lot of object attributes in a very elegant way and it has been working like a charm so far.
The only problem that I am currently having is that I sometimes want to define default values by referencing other attr.ib() attributes. Here is some code that would run if the default for name were a static string:
import attr
from attr.validators import instance_of
import datetime
#attr.s
class Something:
some_date = attr.ib(validator=instance_of(datetime.date))
some_number = attr.ib(convert=float)
name = attr.ib(validator=instance_of(str),
default="Generic Name {0} - {1}%".format(
some_date.strftime("%d-%b-%Y"),
some_number * 100)
)
something_instance = Something(some_date=datetime.date.today(), some_number=0.375)
The problem is that name doesn't see a float and a date, but a _CountingAttr object, hence I get an AttributeError (and a TypeError for some_number * 100). Since I can't reference self either, how do I do this?
So this seems not possible with the default keyword at the moment. However, to achieve the same effect, it's possible to use the __attrs_post_init__ method, which can used to execute arbitrary calculations after instance initialization: http://attrs.readthedocs.io/en/stable/examples.html?highlight=attrs_post_init#other-goodies
In my example it would basically come down to adding
def __attrs_post_init__(self):
if self.name is None:
self.name = "Generic Name {0} - {1}%".format(
self.some_date.strftime("%d-%b-%Y"),
self.some_number * 100)
Credit goes to the attrs github issue tracker for pointing me in the right direction.
You can also do it without __attrs_post_init__.
Just use default = attr.Factory(lambda self: ..., takes_self=True)
import attr
from attr.validators import instance_of
import datetime
#attr.s
class Something:
some_date = attr.ib(validator=instance_of(datetime.date))
some_number = attr.ib(convert=float)
name = attr.ib(validator=instance_of(str),
default=attr.Factory(lambda self: "Generic Name {0} - {1}%".format(
self.some_date.strftime("%d-%b-%Y"),
self.some_number * 100)
), takes_self=True)
something_instance = Something(some_date=datetime.date.today(), some_number=0.375)
If I import a class and rename it by subclassing, it's fairly simple to find the new class name:
>>> from timeit import Timer
>>> class Test(Timer):
... pass
...
>>> test = Test()
>>> test.__class__.__name__
'Test'
However, if I alias the class as I import it, it retains the name from its host module:
>>> from timeit import Timer as Test2
>>> test2 = Test2()
>>> test2.__class__.__name__
'Timer'
Later, I want to provide user-facing output which is aware of the name they've given the class in their namespace. Consider:
def report_stats(timer):
print("Runtime statistics for %s:" % timer.__class__.__name__)
...
Is there a way to get a string reading "Test2", short of iterating over variables in the namespace to test for an exact match?
There's a really terrible answer to my own question; I won't be accepting this since it's probably pretty fragile (I only tested for a limited set of call circumstances). I mostly just hunted this down for the challenge; I will most likely be using something more durable for my actual use case.
This assumes we have access to the init function of the class we're trying to import as blah, and some sort of persistent external data store, at least for more complicated edge cases:
import inspect, dis
class Idiom(object):
description = None
alias = None
def __init__(self, desc):
global data_ob
self.description = desc
if self.__class__.__name__ == 'Idiom':
#cheat like hell to figure out who called us
self.alias = data_ob.name_idiom(inspect.currentframe().f_back)
else:
self.alias = self.__class__.__name__
class DataOb(object):
code = None
locations = {}
LOAD_NAME = 101
codelen = None
def name_idiom(self, frame):
if not self.code:
self.code = frame.f_code
self.codelen = len(self.code.co_code)
self.locations = {y:x for x, y in dis.findlinestarts(self.code)}
target_line = frame.f_lineno
addr_index = self.locations[target_line]+1
name_index = self.code.co_code[addr_index]
# there's a chance we'll get called again this line,
# so we want to seek to the next LOAD_NAME instance(101)
addr_index += 1
while addr_index < self.codelen:
if self.code.co_code[addr_index] == self.LOAD_NAME:
self.locations[target_line] = addr_index
break
addr_index += 1
return self.code.co_names[name_index]
The short explanation of how this works is:
we look up the previous frame from the init function
obtain the code object
find bytecode locations for the start of every line in the code
use the line-number from the frame to grab the bytecode location for the start of that line
locate a LOAD_NAME indicator in the bytecode for this line (I don't really follow this; my code assumes it'll be there)
look in the next bytecode position for an index which indicates which position in the code.co_names tuple contains the "name" of the LOAD_NAME call
From here we can do something like:
>>> from rabbit_hole import Idiom as timer_bob
>>> with timer_bob("down the rabbit hole"):
... waste_some_time = list(range(50000))
...
timer_bob: down the rabbit hole
runtime: 0:00:00.001909, children: 0:00:00, overhead: 0:00:00.001909
I have objects from various classes that work together to perform a certain task. The task requires a lot of parameters, provided by the user (through a configuration file). The parameters are used deep inside the system.
I have a choice of having the controller object read the configuration file, and then allocate the parameters as appropriate to the next layer of objects, and so on in each layer. But the only objects themselves know which parameters they need, so the controller object would need to learn a lot of detail about every other object.
The other choice is to bundle all the parameters into a collection, and pass the whole collection into every function call (equivalently, create a global object that stores them, and is accessible to everyone). This looks and feels ugly, and would cause a variety of minor technical issues (e.g., I can't allow two objects to use parameters with the same name; etc.)
What to do?
I have used the "global collection" alternative in the past.
If you are concerned with naming: how would you handle this in your config file? The way I see it, your global collection is a datastructure representing the same information you have in your config file, so if you have a way of resolving or avoiding name clashes in your cfg-file, you can do the same in your global collection.
I hope you don't feel like I'm thread-jacking you - what you're asking about is similar to what I was thinking about in terms of property aggregation to avoid the models you want to avoid.
I also nicked a bit of the declarative vibe that Elixir has turned me onto.
I'd be curious what the Python gurus of stack overflow think of it and what better alternatives there might be. I don't like big kwargs and if I can avoid big constructors I prefer to.
#!/usr/bin/python
import inspect
from itertools import chain, ifilter
from pprint import pprint
from abc import ABCMeta
class Property(object):
def __init__(self, value=None):
self._x = value
def __repr__(self):
return str(self._x)
def getx(self):
return self._x
def setx(self, value):
self._x = value
def delx(self):
del self._x
value = property(getx, setx, delx, "I'm the property.")
class BaseClass(object):
unique_baseclass_thing = Property()
def get_prop_tree(self):
mro = self.__class__.__mro__
r = []
for i in xrange( 0, len(mro) - 1 ):
child_prop_names = set(dir(mro[i]))
parent_prop_names = set(dir(mro[i+1]))
l_k = list( chain( child_prop_names - parent_prop_names ) )
l_n = [ (x, getattr(mro[i],x,None)) for x in l_k ]
l_p = list( ifilter(lambda y: y[1].__class__ == Property, l_n))
r.append(
(mro[i],
(dict
( l_p )
)
)
)
return r
def get_prop_list(self):
return list( chain(* [ x[1].items() for x in reversed( self.get_prop_tree() ) ] ) )
class SubClass(BaseClass):
unique_subclass_thing = Property(1)
class SubSubClass(SubClass):
unique_subsubclass_thing_one = Property("blah")
unique_subsubclass_thing_two = Property("foo")
if __name__ == '__main__':
a = SubSubClass()
for b in a.get_prop_tree():
print '---------------'
print b[0].__name__
for prop in b[1].keys():
print "\t", prop, "=", b[1][prop].value
print
for prop in a.get_prop_list():
When you run it..
SubSubClass
unique_subsubclass_thing_one = blah
unique_subsubclass_thing_two = foo
---------------
SubClass
unique_subclass_thing = 1
---------------
BaseClass
unique_baseclass_thing = None
unique_baseclass_thing None
unique_subclass_thing 1
unique_subsubclass_thing_one blah
unique_subsubclass_thing_two foo