I'm creating a python library that acts as library and not an app. It's a wrapper around a REST API that has complex responses and I want to abstract the parsing. The library can then be used by apps.
The REST API is around a SaaS offering so one configuration part would be the domain and another the api key. I'm aware I can create an ini file and use configparser. But that means I still need to pass the path of the config file to the library. A dict would however be preferable as that way the app can decide how to store the config. I also want to initialize the library exactly once and not have the options as method or constructor arguments (except for an initialization method).
How can I achieve that? Is it sensible to have the app require to call specific method like config(options: dict) before the library can be used?
EDIT:
Best analogy I can come up with is ORM for REST. Let's say there are Rooms, shelves and Books each backed by a set of API end points (GET, PUT etc). So I will want to create a class that wraps these calls for each item and each class like Book, Room etc needs access to the config.
An important thing is that these items are configurable in the system, eg. admins can add whatever properties to them they want and a GET call for example returns a list of properties with metadata about them (id, name, type etc). Hence the need for the wrapper.
I usually make my configuration options as constructor params where the args are optional. If the value isn't overridden then I fetch the values from environment variables. This allows developers to easily override the configuration without any complex setup.
class Api:
def __init__(self, url=None, key=None):
self.url = url or os.environ.get("MYAPP_URL", "safe default")
self.key = key or os.environ.get("MYAPP_KEY", "safe default")
If you end up getting a large amount of config items that you need to pass around, I'd follow the same pattern, but have a config class:
class ApiConfig:
def __init__(self, url=None, key=None, user=None, region=None):
self.url = url or os.environ.get("MYAPP_URL", "safe default")
self.key = key or os.environ.get("MYAPP_KEY", "safe default")
self.user = user or os.environ.get("MYAPP_USER", "safe default")
self.region = region or os.environ.get("MYAPP_REGION", "safe default")
class Api:
def __init__(self, config=None):
self.config = config or ApiConfig()
Well, here's an idea. How about writing your own config parser that uses dictionaries instead? You could write your config in JSON format and load it as a dictionary using json.loads. This way you can retrieve the configuration parameters from anywhere, including the web since JSON is web-friendly.
Related
I have developed two separate modules which ultimately yield an object which can be passed to cherrypy's quickstart function like chp.quickstart(the_app_object, '/', some_config). The dispatcher I use in some_config is chp.dispatch.MethodDispatcher()
Now I need to develop a module which wraps both.
As I put a reference to each of the first two apps in the third one, I get to a situation like this:
import cherrypy
#cherrypy.expose
class HandlerApp1(object):
def POST(self):
return "output_app_1"
#If mounted, HandlerApp1 may be accessed at hostname:port/path_app_1
class App1(object):
path_app_1 = HandlerApp1()
############################################################
#cherrypy.expose
class HandlerApp2(object):
def POST(self):
return "output_app_2"
#If mounted, HandlerApp2 may be accessed at hostname:port/path_app_2
class App2(object):
path_app_2 = HandlerApp2()
############################################################
#If mounted, HandlerApp1 may be accessed at hostname:port/wrapped_1/path_app_1
#and HandlerApp2 at hostname:port/wrapped_2/path_app_2
class App3(object):
wrapped_1 = App1()
wrapped_2 = App2()
What I would like to invoke both POST methods in a single call, having something like this (continues from above):
#cherrypy.expose
class HandlerApp3(object):
app_1 = App1()
app_2 = App2()
def POST(self):
return app_1.POST() + app_2.POST()
##If mounted, HandlerApp3 may be accessed at hostname:port/path_app_3
class App3(object):
path_app_3 = HandlerApp3()
However, having App3 as in the second part, the individual old applications may still be accessed at hostname:port/path_app_3/app_1 and hostname:port/path_app_3/app_2, which is what I would like to avoid.
As I understand, this is because the chosen dispatcher (chp.dispatch.MethodDispatcher()) maps url path segments to attributes of the application object along the application mount point (in this case '/'), and then invokes the attribute object method corresponding to the request method (GET, POST, etc.) if the object instance as an exposed attribute (which is set by #cherrypy.expose), which means a POST to hostname:port/path_app_3/app_1 would effectively invoke <base_app_object>.path_app_3.app_1.POST().
I was wondering if it could be possible to somehow "unexpose" exposed methods down the hierarchy.
Simply renaming attributes in the third app won't do, obviously. I am not sure whether using dictionaries or another collection would help, as it feels like some hack relying on the library not properly accessing nested collections instead of attributes (assuming this would be enough to disrupt the path matching process.
I've been reading through some Google App Engine SDK source code, and I've noticed Google often writes a private class method (prepending the method's name with an _), but immediately after finishing the method code block, they create a public variable with the same name, and assign the private method to the variable.
Why do they do this?
Example code:
#classmethod
#utils.positional(3)
def _get_by_id(cls, id, parent=None, **ctx_options):
"""Returns an instance of Model class by ID.
This is really just a shorthand for Key(cls, id, ...).get().
Args:
id: A string or integer key ID.
parent: Optional parent key of the model to get.
namespace: Optional namespace.
app: Optional app ID.
**ctx_options: Context options.
Returns:
A model instance or None if not found.
"""
return cls._get_by_id_async(id, parent=parent, **ctx_options).get_result()
get_by_id = _get_by_id
This is almost certainly a hold-over from a time when Python had no decorator syntax yet. Decorators where introduced in Python 2.4 (see PEP 318), and before this time you had to manually apply a decorator function to an existing, already defined function object.
The code would originally have been written like this:
def _get_by_id(cls, id, parent=None, **ctx_options):
"""Returns an instance of Model class by ID.
This is really just a shorthand for Key(cls, id, ...).get().
Args:
id: A string or integer key ID.
parent: Optional parent key of the model to get.
namespace: Optional namespace.
app: Optional app ID.
**ctx_options: Context options.
Returns:
A model instance or None if not found.
"""
return cls._get_by_id_async(id, parent=parent, **ctx_options).get_result()
get_by_id = classmethod(utils.positional(3)(_get_by_id))
This manually applied the classmethod and utils.positional(3) decorator functions.
Generally, the undecorated underscore name was retained for easier testing. This doesn't make that much sense with a classmethod, but it could have been a pattern in the codebase that was followed anywhere a decorator was used.
This is not the only hold-over; the Google Python styleguide on Properties is similarly archaic, see Google Style Guide properties for getters and setters.
Is it bad practice to use a ConfigParser within class methods? Doing this would mean the class is then tied to the config and not as easily re-usable, but means less input arguments in methods, which I would find messy especially if arguments had to be passed down multiple layers.
Are there good alternatives (apart from just passing config values as method arguments)? Or a particular pattern people find works well for this?
For example
# get shared config parser configured by main script
from utils.config_utils import config_parser
class FooClass(object):
def foo_method(self):
self._foo_bar_method()
def _foo_bar_method(self):
some_property = config_parser.get("root", "FooProperty")
....
If you need a lot of arguments in a single class that might be a symptom that you are trying to do too much with that class (see SRP)
If there is indeed a real need for some configuration options that are too many to provide for a simple class as arguments I would advice to abstract the configuration as a separate class and use that as an argument:
class Configuration(object):
def __init__(self, config_parser):
self.optionA = config_parser.get("root", "AProperty")
self.optionB = config_parser.get("root", "BProperty")
self.optionX = config_parser.get("root", "XProperty")
#property
def optionY(self):
return self.optionX == 'something' and self.optionA > 10
class FooClass(object):
def __init__(self, config):
self._config = config
def _foo_bar_method(self):
some_property = self._config.optionY
....
config = Configuration(config_parser)
foo = FooClass(config)
In this way you can reuse your configuration abstraction or even build different configuration abstraction for different purposes from the same config parser.
You can even improve the configuration class to have a more declarative way to map configuration properties to instance attributes (but that's more advanced topic).
I came across this python code base where there is a custom ORM code that maps to MongoDb. I was asked to write new Entities following the existing pattern of code. I have only been writing python code for a week now. But, i find this mix of getter style method name marked as property etc confusing. I know I am not giving a lot of context here. But, I want to discuss how this api looks from a good python programming api pov.
Its python 2.7.
class Entity(object):
def save(self):
...
class Person(Entity):
...
#property
def get_config(self):
return getattr(self, 'config', None)
def set_config(self, config):
self.set('config', config)
...
It doesn't look that good. There is no reason to use getters/setters in Python just to get and set static values. See this question. You should only use them if you actually need to programmatically compute the attribute value on every access.
Also, instead of using getattr to return None if the attribute is undefined, you should just initialize it to None. Then you can later read its value unconditionally.
So:
class Person(Entity):
def __init__(self):
# initialize config to None
self.config = None
Later, if other code needs to read the config attribute of a Person instance called somePerson, just do:
somePerson.config
If you need to set it, just do:
somePerson.config = newConfig
In the code below the User class needs to access a function get_user inside an instance of WebService class, as that contains other functions required for authentication with the web server (last.fm). Actual code is here.
class WebService:
def __init__(self, key):
self.apikey = key
def get_user(self, name):
pass # Omitted
class User:
def __init__(self, name, webservice):
self.name = name
self.ws = webservice
def fill_profile(self):
data = self.ws.GetUser(self.name)
# Omitted
The problem is that a reference needs to be held inside every ´User´. Is there another way of doing this? Or is it just me overcomplicating things, and this is how it actually works in the real world?
As requested:
As to handling things like get_top_albums and get_friends, that depends on how you want to model the system. If you don't want to cache the data locally, I'd say just call the service each time with a user ID. If you do want to cache the data locally, you could pass a User object to the method in WebService, then have the method populate the members of the User. You do have to make a design decision though to either have a WebService and a User (what would probably be best), or just a UserWebService.
You can certainly make the reference a static variable, if the web service object is the same for all users.
The syntax is:
class User:
webservice = ...
...
You will then even be able to access it from User instances, but not to assign to it that way, that would require User.webservice syntax.
You are also getting good design alternatives suggested in the comments.