I need to write simple Python script for validating some system ENV variables. And I can do it with simplest code like a:
#test.py
import os
import sys
validation_errors = []
min_resources_count = os.environ.get('RESOURCES_MIN')
max_resources_count = os.environ.get('RESOURCES_MAX')
if int(min_resources_count) > int(max_resources_count):
validation_errors.append('Maximum value must be greater than or equal to minimum value')
if int(max_resources_count) == 0:
validation_errors.append('Maximum value cannot be zero')
...
# And so on...
...
if validation_errors:
sys.stderr.write(validation_errors) # will add some formatting
But, something inside me says that it can be written better. Maybe it can be moved inside some class, something like this:
class SystemEnvValidation:
def __init__(self):
self.min_resources_count = os.environ.get('RESOURCES_MIN')
self.max_resources_count = os.environ.get('RESOURCES_MAX')
self.validation_errors = []
def max_resources_count(self):
# place IF conditions here
if int(self.max_resources_count) == 0:
self.validation_errors.append('Maximum value cannot be zero')
...
print(SystemEnvValidation().validation_errors)
So, my question is - what is the best way to implement it? Should I place it in class? Or maybe I can leave it as is? In the case of class - I am not familiar with Python classes - could you show me simple working code structure?
I'm writing some tooling for online programming contexts.
Part of it is a test case checker which actually based on a set of pairs of (input, output) files are gonna check whether the solution method is actually working.
Basically, the solution method is expected to be defined as follow:
def solution(Nexter: inputs):
# blahblah some code here and there
n = inputs.next_int()
sub_process(inputs)
# simulating a print something
yield str(n)
can be then translated (once the AST modifications) as:
def solution():
# blahblah some code here and there
n = int(input())
sub_process()
print(str(n))
Note: Nexter is a class defined to be whether a generator of user input() calls or carry out the expected inputs + some other goodies.
I'm aware of the issues related to converting back to source code from the AST (requires to rely on 3rd party stuff). I also know that there is a NodeTransformer class:
http://greentreesnakes.readthedocs.io/en/latest/manipulating.html
https://docs.python.org/3/library/ast.html#ast.NodeTransformer
But its use remains unclear to me I don't know if I'm better off checking calls, expr, etc.
Here is below what I've ended up with:
signature = inspect.signature(iterative_greedy_solution)
if len(signature.parameters) == 1 and "inputs" in signature.parameters:
parameter = signature.parameters["inputs"]
annotation = parameter.annotation
if Nexter == annotation:
source = inspect.getsource(iterative_greedy_solution)
tree = ast.parse(source)
NexterInputsRewriter().generic_visit(tree)
class NexterInputsRewriter(ast.NodeTransformer):
def visit(self, node):
#???
This is definitely not the best design ever. Next time, I would probably go for the other way around (i.e. having a definition with simple user defined input() (and output, i.e. print(...)) and replacing them with test case inputs) when passing to a tester class asserting whether actual outputs are matching expecting ones.
To sum up this what I would like to achieve and I don't really know exactly how (apart of subclassing the NodeTransformer class):
Get rid of the solution function arguments
Modifiy the inputs calls in method body (as well as in the sub calls of methods also leveraging Nexter: inputs) in order to replace them with their actual user input() implementation, e.g. inputs.next_int() = int(input())
EDIT
Found that tool (https://python-ast-explorer.com/) that helps a lot to visualize what kind of ast.AST derivatives are used for a given function.
You can probably use NodeTransformer + ast.unparse() though it wouldn't be as effective as checking out some other 3rd party solutions considering it won't preserve any of your comments.
Here is an example transformation done by refactor (I'm the author), which is a wrapper layer around ast.unparse for doing easy source-to-source transformations through AST;
import ast
import refactor
from refactor import ReplacementAction
class ReplaceNexts(refactor.Rule):
def match(self, node):
# We need a call
assert isinstance(node, ast.Call)
# on an attribute (inputs.xxx)
assert isinstance(node.func, ast.Attribute)
# where the name for attribute is `inputs`
assert isinstance(node.func.value, ast.Name)
assert node.func.value.id == "inputs"
target_func_name = node.func.attr.removeprefix("next_")
# make a call to target_func_name (e.g int) with input()
target_func = ast.Call(
ast.Name(target_func_name),
args=[
ast.Call(ast.Name("input"), args=[], keywords=[]),
],
keywords=[],
)
return ReplacementAction(node, target_func)
session = refactor.Session([ReplaceNexts])
source = """\
def solution(Nexter: inputs):
# blahblah some code here and there
n = inputs.next_int()
sub_process(inputs)
st = inputs.next_str()
sub_process(st)
"""
print(session.run(source))
$ python t.py
def solution(Nexter: inputs):
# blahblah some code here and there
n = int(input())
sub_process(inputs)
st = str(input())
sub_process(st)
I wrote the following code in python and it works fine until I try to make it a function, can anyone help?
import random
def club():
members=int(input("members"))
print (random.randint(1, members))
You have to use 4 spaces ( or tab) per indentation level. And call the function of course.
import random
def club():
members=int(input("members"))
print (random.randint(1, members))
club()
members is local to the club function, thus not visible to your print call
members is considered to be a local variable within the club function. If you want to use it, you can return it. You can do something like this:
import random
def club():
members = int(input("Members: "))
return members
print(random.randint(1, club()))
Also, since Python is an Object-Oriented Language, you can set the value that club returns to a variable (or anything for that matter):
a = club() # Then a might be 5
b = random.randint(1, a)
And then use that variable elsewhere:
print(b)
I'm making a game in pygame and I have made an 'abstract' class that's sole job is to store the sprites for a given level (with the intent of having these level objects in a list to facilitate the player being moved from one level to another)
Alright, so to the question. If I can do the equivalent of this in Python(code curtesy of Java):
Object object = new Object (){
public void overriddenFunction(){
//new functionality
};
};
Than when I build the levels in the game I would simply have to override the constructor (or a class/instance method that is responsible for building the level) with the information on where the sprites go, because making a new class for every level in the game isn't that elegant of an answer. Alternatively I would have to make methods within the level class that would then build the level once a level object is instantiated, placing the sprites as needed.
So, before one of the more stanch developers goes on about how anti-python this might be (I've read enough of this site to get that vibe from Python experts) just tell me if its doable.
Yes, you can!
class Foo:
def do_other(self):
print('other!')
def do_foo(self):
print('foo!')
def do_baz():
print('baz!')
def do_bar(self):
print('bar!')
# Class-wide impact
Foo.do_foo = do_bar
f = Foo()
g = Foo()
# Instance-wide impact
g.do_other = do_baz
f.do_foo() # prints "bar!"
f.do_other() # prints "other!"
g.do_foo() # prints "bar!"
g.do_other() # prints "baz!"
So, before one of the more stanch developers goes on about how anti-python this might be
Overwriting functions in this fashion (if you have a good reason to do so) seems reasonably pythonic to me. An example of one reason/way for which you might have to do this would be if you had a dynamic feature for which static inheritance didn't or couldn't apply.
The case against might be found in the Zen of Python:
Beautiful is better than ugly.
Readability counts.
If the implementation is hard to explain, it's a bad idea.
Yes, it's doable. Here, I use functools.partial to get the implied self argument into a regular (non-class-method) function:
import functools
class WackyCount(object):
"it's a counter, but it has one wacky method"
def __init__(self, name, value):
self.name = name
self.value = value
def __str__(self):
return '%s = %d' % (self.name, self.value)
def incr(self):
self.value += 1
def decr(self):
self.value -= 1
def wacky_incr(self):
self.value += random.randint(5, 9)
# although x is a regular wacky counter...
x = WackyCount('spam', 1)
# it increments like crazy:
def spam_incr(self):
self.value *= 2
x.incr = functools.partial(spam_incr, x)
print (x)
x.incr()
print (x)
x.incr()
print (x)
x.incr()
print (x)
and:
$ python2.7 wacky.py
spam = 1
spam = 2
spam = 4
spam = 8
$ python3.2 wacky.py
spam = 1
spam = 2
spam = 4
spam = 8
Edit to add note: this is a per-instance override. It takes advantage of Python's attribute look-up sequence: if x is an instance of class K, then x.attrname starts by looking at x's dictionary to find the attribute. If not found, the next lookup is in K. All the normal class functions are actually K.func. So if you want to replace the class function dynamically, use #Brian Cane's answer instead.
I'd suggest using a different class, via inheritance, for each level.
But you might get some mileage out of copy.deepcopy() and monkey patching, if you're really married to treating Python like Java.
I have a class that contains only fields and no methods, like this:
class Request(object):
def __init__(self, environ):
self.environ = environ
self.request_method = environ.get('REQUEST_METHOD', None)
self.url_scheme = environ.get('wsgi.url_scheme', None)
self.request_uri = wsgiref.util.request_uri(environ)
self.path = environ.get('PATH_INFO', None)
# ...
This could easily be translated to a dict. The class is more flexible for future additions and could be fast with __slots__. So would there be a benefit of using a dict instead? Would a dict be faster than a class? And faster than a class with slots?
Use a dictionary unless you need the extra mechanism of a class. You could also use a namedtuple for a hybrid approach:
>>> from collections import namedtuple
>>> request = namedtuple("Request", "environ request_method url_scheme")
>>> request
<class '__main__.Request'>
>>> request.environ = "foo"
>>> request.environ
'foo'
Performance differences here will be minimal, although I would be surprised if the dictionary wasn't faster.
Why would you make this a dictionary? What's the advantage? What happens if you later want to add some code? Where would your __init__ code go?
Classes are for bundling related data (and usually code).
Dictionaries are for storing key-value relationships, where usually the keys are all of the same type, and all the values are also of one type. Occasionally they can be useful for bundling data when the key/attribute names are not all known up front, but often this a sign that something's wrong with your design.
Keep this a class.
A class in python is a dict underneath. You do get some overhead with the class behavior, but you won't be able to notice it without a profiler. In this case, I believe you benefit from the class because:
All your logic lives in a single function
It is easy to update and stays encapsulated
If you change anything later, you can easily keep the interface the same
I think that the usage of each one is way too subjective for me to get in on that, so i'll just stick to numbers.
I compared the time it takes to create and to change a variable in a dict, a new_style class and a new_style class with slots.
Here's the code i used to test it(it's a bit messy but it does the job.)
import timeit
class Foo(object):
def __init__(self):
self.foo1 = 'test'
self.foo2 = 'test'
self.foo3 = 'test'
def create_dict():
foo_dict = {}
foo_dict['foo1'] = 'test'
foo_dict['foo2'] = 'test'
foo_dict['foo3'] = 'test'
return foo_dict
class Bar(object):
__slots__ = ['foo1', 'foo2', 'foo3']
def __init__(self):
self.foo1 = 'test'
self.foo2 = 'test'
self.foo3 = 'test'
tmit = timeit.timeit
print 'Creating...\n'
print 'Dict: ' + str(tmit('create_dict()', 'from __main__ import create_dict'))
print 'Class: ' + str(tmit('Foo()', 'from __main__ import Foo'))
print 'Class with slots: ' + str(tmit('Bar()', 'from __main__ import Bar'))
print '\nChanging a variable...\n'
print 'Dict: ' + str((tmit('create_dict()[\'foo3\'] = "Changed"', 'from __main__ import create_dict') - tmit('create_dict()', 'from __main__ import create_dict')))
print 'Class: ' + str((tmit('Foo().foo3 = "Changed"', 'from __main__ import Foo') - tmit('Foo()', 'from __main__ import Foo')))
print 'Class with slots: ' + str((tmit('Bar().foo3 = "Changed"', 'from __main__ import Bar') - tmit('Bar()', 'from __main__ import Bar')))
And here is the output...
Creating...
Dict: 0.817466186345
Class: 1.60829183597
Class_with_slots: 1.28776730003
Changing a variable...
Dict: 0.0735140918748
Class: 0.111714198313
Class_with_slots: 0.10618612142
So, if you're just storing variables, you need speed, and it won't require you to do many calculations, i recommend using a dict(you could always just make a function that looks like a method). But, if you really need classes, remember - always use __slots__.
Note:
I tested the 'Class' with both new_style and old_style classes. It turns out that old_style classes are faster to create but slower to modify(not by much but significant if you're creating lots of classes in a tight loop (tip: you're doing it wrong)).
Also the times for creating and changing variables may differ on your computer since mine is old and slow. Make sure you test it yourself to see the 'real' results.
Edit:
I later tested the namedtuple: i can't modify it but to create the 10000 samples (or something like that) it took 1.4 seconds so the dictionary is indeed the fastest.
If i change the dict function to include the keys and values and to return the dict instead of the variable containing the dict when i create it it gives me 0.65 instead of 0.8 seconds.
class Foo(dict):
pass
Creating is like a class with slots and changing the variable is the slowest (0.17 seconds) so do not use these classes. go for a dict (speed) or for the class derived from object ('syntax candy')
I agree with #adw. I would never represent an "object" (in an OO sense) with a dictionary. Dictionaries aggregate name/value pairs. Classes represent objects. I've seen code where the objects are represented with dictionaries and it's unclear what the actual shape of the thing is. What happens when certain name/values aren't there? What restricts the client from putting anything at all in. Or trying to get anything at all out. The shape of the thing should always be clearly defined.
When using Python it is important to build with discipline as the language allows many ways for the author to shoot him/herself in the foot.
I would recommend a class, as it is all sorts of information involved with a request. Were one to use a dictionary, I'd expect the data stored to be far more similar in nature. A guideline I tend to follow myself is that if I may want to loop over the entire set of key->value pairs and do something, I use a dictionary. Otherwise, the data apparently has far more structure than a basic key->value mapping, meaning a class would likely be a better alternative.
Hence, stick with the class.
If all that you want to achive is syntax candy like obj.bla = 5 instead of obj['bla'] = 5, especially if you have to repeat that a lot, you maybe want to use some plain container class as in martineaus suggestion. Nevertheless, the code there is quite bloated and unnecessarily slow. You can keep it simple like that:
class AttrDict(dict):
""" Syntax candy """
__getattr__ = dict.__getitem__
__setattr__ = dict.__setitem__
__delattr__ = dict.__delitem__
Another reason to switch to namedtuples or a class with __slots__ could be memory usage. Dicts require significantly more memory than list types, so this could be a point to think about.
Anyways, in your specific case, there doesn't seem to be any motivation to switch away from your current implementation. You don't seem to maintain millions of these objects, so no list-derived-types required. And it's actually containing some functional logic within the __init__, so you also shouldn't got with AttrDict.
It may be possible to have your cake and eat it, too. In other words you can create something that provides the functionality of both a class and dictionary instance. See the ActiveState's Dɪᴄᴛɪᴏɴᴀʀʏ ᴡɪᴛʜ ᴀᴛᴛʀɪʙᴜᴛᴇ-sᴛʏʟᴇ ᴀᴄᴄᴇss recipe and comments on ways of doing that.
If you decide to use a regular class rather than a subclass, I've found the Tʜᴇ sɪᴍᴘʟᴇ ʙᴜᴛ ʜᴀɴᴅʏ "ᴄᴏʟʟᴇᴄᴛᴏʀ ᴏғ ᴀ ʙᴜɴᴄʜ ᴏғ ɴᴀᴍᴇᴅ sᴛᴜғғ" ᴄʟᴀss recipe (by Alex Martelli) to be very flexible and useful for the sort of thing it looks like you're doing (i.e. create a relative simple aggregator of information). Since it's a class you can easily extend its functionality further by adding methods.
Lastly it should be noted that the names of class members must be legal Python identifiers, but dictionary keys do not—so a dictionary would provide greater freedom in that regard because keys can be anything hashable (even something that's not a string).
Update
A class object (which doesn't have a __dict__) subclass named SimpleNamespace (which does have one) was added to the types module Python 3.3, and is yet another alternative.
If the data, I mean set of fields, is not to be changed or extended in the future i would choose a class for representation such data. Why?
It's a little more clean and readable.
It's faster in terms of using it, which is much more important than creating it, which happens generally only once.
Even faster seems using just class as container for fields not object of the class.
extending alexpinho98 example:
import timeit
class Foo(object):
def __init__(self):
self.foo1 = 'test'
self.foo2 = 'test'
self.foo3 = 'test'
class FooClass:
foo1 = 'test'
foo2 = 'test'
foo3 = 'test'
def create_dict():
foo_dict = {}
foo_dict['foo1'] = 'test'
foo_dict['foo2'] = 'test'
foo_dict['foo3'] = 'test'
return foo_dict
class Bar(object):
__slots__ = ['foo1', 'foo2', 'foo3']
def __init__(self):
self.foo1 = 'test'
self.foo2 = 'test'
self.foo3 = 'test'
tmit = timeit.timeit
dict = create_dict()
def testDict():
a = dict['foo1']
b = dict['foo2']
c = dict['foo3']
dict_obj = Foo()
def testObjClass():
a = dict_obj.foo1
b = dict_obj.foo2
c = dict_obj.foo3
def testClass():
a = FooClass.foo1
b = FooClass.foo2
c = FooClass.foo3
print ('Creating...\n')
print ('Dict: ' + str(tmit('create_dict()', 'from __main__ import create_dict')))
print ('Class: ' + str(tmit('Foo()', 'from __main__ import Foo')))
print ('Class with slots: ' + str(tmit('Bar()', 'from __main__ import Bar')))
print ('=== Testing usage 1 ===')
print ('Using dict : ' + str(tmit('testDict()', 'from __main__ import testDict')))
print ('Using object: ' + str(tmit('testObjClass()', 'from __main__ import testObjClass')))
print ('Using class : ' + str(tmit('FooClass()', 'from __main__ import FooClass')))
Results are:
Creating...
Dict: 0.185864600000059
Class: 0.30627199999980803
Class with slots: 0.2572166999998444
=== Testing usage 1 ===
Using dict : 0.16507520000050135
Using object: 0.1266871000007086
Using class : 0.06327920000148879
class ClassWithSlotBase:
__slots__ = ('a', 'b',)
def __init__(self):
self.a: str = "test"
self.b: float = 0.0
def test_type_hint(_b: float) -> None:
print(_b)
class_tmp = ClassWithSlotBase()
test_type_hint(class_tmp.a)
I recommend a class. If you use a class, you can get type hint as shown. And Class support auto complete when class is argument of function.