We Keep Coding

Python - Problem returning True/False to class properties from class method

I have a class as below which within the __init__ method I am trying to assign a True/False value to a class property using a class method.
class Sensor:
def __init__(self, json_data):
self.sensor_eui = json_data['end_device_ids']['dev_eui']
self.reading1 = json_data['uplink_message']['decoded_payload']['temperature']
self.reading2 = json_data['uplink_message']['decoded_payload']['humidity']
self.tolerance_exceeded = self.tolerance_check
def tolerance_check(self):
sql = f"SELECT DefaultLowerLimit, DefaultUpperLimit FROM [dbo].[IoT_Sensors] WHERE
DeviceID = '{self.sensor_eui}'"
results = exec_sql(sql)
if (self.reading1 > int(results[0])) and (self.reading1 < int(results[1])):
return False
return True
The issue is, when trying to troubleshoot this and logging the objects to the console, instead of returning True or False as the assigned value of 'tolerance_exceeded' it returns the method and object:
logging.info(f'Tolerance Exceeded: {sensor.tolerance_exceeded}')
logs in the console as below:
[2022-10-26T12:08:08.025Z] Tolerance Exceeded: <bound method Sensor.tolerance_check of <__app__.IoT_Data-Handler.classes.Sensor object at 0x000001C834D45BE0>>
So what is going on here? I have not been coding long, but when I have done something similar in the past (assigning a string value from an API), it worked fine. This is for an Azure Function, but I cannot see how that would impact what I am trying to achieve.
Any help would be appreciated.

The issue in your code is that instead of calling the function you assign it. In order to call the function you have to add the parenthesis.
class Sensor:
def __init__(self, json_data):
self.sensor_eui = json_data['end_device_ids']['dev_eui']
self.reading1 = json_data['uplink_message']['decoded_payload']['temperature']
self.reading2 = json_data['uplink_message']['decoded_payload']['humidity']
# Calling tolerance_check and assigning return value to tolerance_exceeded
self.tolerance_exceeded = self.tolerance_check()

How to use a list as a class variable so that instance object (arguments) are appended to the list?

I would like to simply make a list of kinds of coffe, but get an error stating that the list is not defined. Do I have to use self in the constructor when referencing to a classvariable?
I have tried changing the return statement to return self.coffelist.append(name), but then get another error: 'Function' object has no attribute 'append'.
class coffe:
coffelist = []
def __init__(self,name,origin,price):
self.name = name
self.origin = origin
self.price = price
return (self.coffelist.append(self.name))
def coffelist(self):
print(coffelist)
c1=coffe("blackcoffe","tanz",55)
c2=coffe("fineroasted","ken",60)

This is because you named one of your methods as coffelist.

I think this shows how to do what you want. I also modified your code to follow the PEP 8 - Style Guide for Python Code and corrected some misspelled words.
class Coffee: # Class names should Capitalized.
coffeelist = [] # Class attribute to track instance names.
def __init__(self,name,origin,price):
self.name = name
self.origin = origin
self.price = price
self.coffeelist.append(self.name)
def print_coffeelist(self):
print(self.coffeelist)
c1 = Coffee("blackcoffee", "tanz", 55)
c1.print_coffeelist() # -> ['blackcoffee']
c2 = Coffee("fineroasted", "ken", 60)
c1.print_coffeelist() # -> ['blackcoffee', 'fineroasted']
# Can also access attribute directly through the class:
print(Coffee.coffeelist) # -> ['blackcoffee', 'fineroasted']

yes thanks that's exactly what I wanted!
I wasnt sure.. I thought you could do 2 things simultaneously in the return statement, both return append. I guess allot of times python is very flexible and sometimes not. thanks

Method __init__ has too many parameters

I'm super new to Python (I started about 3 weeks ago) and I'm trying to make a script that scrapes web pages for information. After it's retrieved the information it runs through a function to format it and then passes it to a class that takes 17 variables as parameters. The class uses this information to calculate some other variables and currently has a method to construct a dictionary. The code works as intended but a plugin I'm using with Pycharm called SonarLint highlights that 17 variables is too many to use as parameters?
I've had a look for alternate ways to pass the information to the class, such as in a tuple or a list but couldn't find much information that seemed relevant. What's the best practice for passing many variables to a class as parameters? Or shouldn't I be using a class for this kind of thing at all?
I've reduced the amount of variables and code for legibility but here is the class;
Class GenericEvent:
def __init__(self, type, date_scraped, date_of_event, time, link,
blurb):
countdown_delta = date_of_event - date_scraped
countdown = countdown_delta.days
if countdown < 0:
has_passed = True
else:
has_passed = False
self.type = type
self.date_scraped = date_scraped
self.date_of_event = date_of_event
self.time = time
self.link = link
self.countdown = countdown
self.has_passed = has_passed
self.blurb = blurb
def get_dictionary(self):
event_dict = {}
event_dict['type'] = self.type
event_dict['scraped'] = self.date_scraped
event_dict['date'] = self.date_of_event
event_dict['time'] = self.time
event_dict['url'] = self.link
event_dict['countdown'] = self.countdown
event_dict['blurb'] = self.blurb
event_dict['has_passed'] = self.has_passed
return event_dict
I've been passing the variables as key:value pairs to the class after I've cleaned up the data the following way:
event_info = GenericEvent(type="Lunar"
date_scraped=30/01/19
date_of_event=28/07/19
time=12:00
link="www.someurl.com"
blurb="Some string.")
and retrieving a dictionary by calling:
event_info.get_dictionary()
I intend to add other methods to the class to be able to perform other operations too (not just to create 1 dictionary) but would like to resolve this before I extend the functionality of the class.
Any help or links would be much appreciated!

One option is a named tuple:
from typing import Any, NamedTuple
class GenericEvent(NamedTuple):
type: Any
date_scraped: Any
date_of_event: Any
time: Any
link: str
countdown: Any
blurb: str
#property
def countdown(self):
countdown_delta = date_of_event - date_scraped
return countdown_delta.days
#property
def has_passed(self):
return self.countdown < 0
def get_dictionary(self):
return {
**self._asdict(),
'countdown': self.countdown,
'has_passed': self.has_passed,
}
(Replace the Anys with the fields’ actual types, e.g. datetime.datetime.)
Or, if you want it to be mutable, a data class.

I don't think there's anything wrong with what you're doing. You could, however, take your parameters in as a single dict object, and then deal with them by iterating over the dict or doing something explicitly with each one. Seems like that would, in your case, make your code messier.
Since all of your parameters to your constructor are named parameters, you could just do this:
def __init__(self, **params):
This would give you a dict named params that you could then process. The keys would be your parameter names, and the values the parameter values.
If you aligned your param names with what you want the keys to be in your get_dictionary method's return value, saving off this parameter as a whole could make that method trivial to write.
Here's an abbreviated version of your code (with a few syntax errors fixed) that illustrates this idea:
from pprint import pprint
class GenericEvent:
def __init__(self, **params):
pprint(params)
event_info = GenericEvent(type="Lunar",
date_scraped="30/01/19",
date_of_event="28/07/19",
time="12:00",
link="www.someurl.com",
blurb="Some string.")
Result:
{'blurb': 'Some string.',
'date_of_event': '28/07/19',
'date_scraped': '30/01/19',
'link': 'www.someurl.com',
'time': '12:00',
'type': 'Lunar'}

Copy an entity in Google App Engine datastore in Python without knowing property names at 'compile' time

In a Python Google App Engine app I'm writing, I have an entity stored in the datastore that I need to retrieve, make an exact copy of it (with the exception of the key), and then put this entity back in.
How should I do this? In particular, are there any caveats or tricks I need to be aware of when doing this so that I get a copy of the sort I expect and not something else.
ETA: Well, I tried it out and I did run into problems. I would like to make my copy in such a way that I don't have to know the names of the properties when I write the code. My thinking was to do this:
#theThing = a particular entity we pull from the datastore with model Thing
copyThing = Thing(user = user)
for thingProperty in theThing.properties():
copyThing.__setattr__(thingProperty[0], thingProperty[1])
This executes without any errors... until I try to pull copyThing from the datastore, at which point I discover that all of the properties are set to None (with the exception of the user and key, obviously). So clearly this code is doing something, since it's replacing the defaults with None (all of the properties have a default value set), but not at all what I want. Suggestions?

Here you go:
def clone_entity(e, **extra_args):
"""Clones an entity, adding or overriding constructor attributes.
The cloned entity will have exactly the same property values as the original
entity, except where overridden. By default it will have no parent entity or
key name, unless supplied.
Args:
e: The entity to clone
extra_args: Keyword arguments to override from the cloned entity and pass
to the constructor.
Returns:
A cloned, possibly modified, copy of entity e.
"""
klass = e.__class__
props = dict((k, v.__get__(e, klass)) for k, v in klass.properties().iteritems())
props.update(extra_args)
return klass(**props)
Example usage:
b = clone_entity(a)
c = clone_entity(a, key_name='foo')
d = clone_entity(a, parent=a.key().parent())
EDIT: Changes if using NDB
Combining Gus' comment below with a fix for properties that specify a different datastore name, the following code works for NDB:
def clone_entity(e, **extra_args):
klass = e.__class__
props = dict((v._code_name, v.__get__(e, klass)) for v in klass._properties.itervalues() if type(v) is not ndb.ComputedProperty)
props.update(extra_args)
return klass(**props)
Example usage (note key_name becomes id in NDB):
b = clone_entity(a, id='new_id_here')
Side note: see the use of _code_name to get the Python-friendly property name. Without this, a property like name = ndb.StringProperty('n') would cause the model constructor to raise an AttributeError: type object 'foo' has no attribute 'n'.

If you're using the NDB you can simply copy with:
new_entity.populate(**old_entity.to_dict())

This is just an extension to Nick Johnson's excellent code to address the problems highlighted by Amir in the comments:
The db.Key value of the ReferenceProperty is no longer retrieved via an unnecessary roundtrip to the datastore.
You can now specify whether you want to skip DateTime properties with the auto_now and/or auto_now_add flag.
Here's the updated code:
def clone_entity(e, skip_auto_now=False, skip_auto_now_add=False, **extra_args):
"""Clones an entity, adding or overriding constructor attributes.
The cloned entity will have exactly the same property values as the original
entity, except where overridden. By default it will have no parent entity or
key name, unless supplied.
Args:
e: The entity to clone
skip_auto_now: If True then all DateTimeProperty propertes will be skipped which have the 'auto_now' flag set to True
skip_auto_now_add: If True then all DateTimeProperty propertes will be skipped which have the 'auto_now_add' flag set to True
extra_args: Keyword arguments to override from the cloned entity and pass
to the constructor.
Returns:
A cloned, possibly modified, copy of entity e.
"""
klass = e.__class__
props = {}
for k, v in klass.properties().iteritems():
if not (type(v) == db.DateTimeProperty and ((skip_auto_now and getattr(v, 'auto_now')) or (skip_auto_now_add and getattr(v, 'auto_now_add')))):
if type(v) == db.ReferenceProperty:
value = getattr(klass, k).get_value_for_datastore(e)
else:
value = v.__get__(e, klass)
props[k] = value
props.update(extra_args)
return klass(**props)
The first if expression is not very elegant so I appreciate if you can share a better way to write it.

I'm neither Python nor AppEngine guru, but couldn't one dynamically get/set the properties?
props = {}
for p in Thing.properties():
props[p] = getattr(old_thing, p)
new_thing = Thing(**props).put()

A variation inspired in Nick's answer which handles the case in which your entity has a (repeated) StructuredProperty, where the StructuredProperty itself has ComputedProperties. It can probably be written more tersely with dict comprehension somehow, but here is the longer version that worked for me:
def removeComputedProps(klass,oldDicc):
dicc = {}
for key,propertType in klass._properties.iteritems():
if type(propertType) is ndb.StructuredProperty:
purged = []
for item in oldDicc[key]:
purged.append(removeComputedProps(propertType._modelclass,item))
dicc[key]=purged
else:
if type(propertType) is not ndb.ComputedProperty:
dicc[key] = oldDicc[key]
return dicc
def cloneEntity(entity):
oldDicc = entity.to_dict()
klass = entity.__class__
dicc = removeComputedProps(klass,oldDicc)
return klass(**dicc)

This can be tricky if you've renamed the underlying keys for your properties... which some people opt to do instead of making mass data changes
say you started with this:
class Person(ndb.Model):
fname = ndb.StringProperty()
lname = ndb.StringProperty()
then one day you really decided that it would be nicer to use first_name and last_name instead... so you do this:
class Person(ndb.Model):
first_name = ndb.StringProperty(name="fname")
last_name = ndb.StringProperty(name="lname")
now when you do Person._properties (or .properties() or person_instance._properties) you will get a dictionary with keys that match the underlying names (fname and lname)... but won't match the actual property names on the class... so it won't work if you put them into the constructor of a new instance, or use the .populate() method (the above examples will break)
In NDB anyways, instances of models have ._values dictionary which is keyed by the underlying property names... and you can update it directly. I ended up with something like this:
def clone(entity, **extra_args):
klass = entity.__class__
clone = klass(**extra_args)
original_values = dict((k,v) for k,v in entity._values.iteritems() if k not in clone._values)
clone._values.update(original_values)
return clone
This isn't really the safest way... as there are other private helper methods that do more work (like validation and conversion of computed properties by using _store_value() and _retrieve_value())... but if you're models are simple enough, and you like living on the edge :)

Here's the code provided by #zengabor with the if expression formatted for easier reading. It may not be PEP-8 compliant:
klass = e.__class__
props = {}
for k, v in klass.properties().iteritems():
if not (type(v) == db.DateTimeProperty and ((
skip_auto_now and getattr(v, 'auto_now' )) or (
skip_auto_now_add and getattr(v, 'auto_now_add')))):
if type(v) == db.ReferenceProperty:
value = getattr(klass, k).get_value_for_datastore(e)
else:
value = v.__get__(e, klass)
props[k] = value
props.update(extra_args)
return klass(**props)

Best way to construct a "complex" data structure in Python

I need to construct a tool that will be used to create field mappings (between tables) in the most automated manner possible.
Here is the deal: imagine a table being appended to other. (lets ignore field type, just for a second...)
CREATE OR REPLACE TABLE fooA(
id,
name,
type,
foo)
CREATE OR REPLACE TABLE otherFooTable(
idFoo,
nameFoo,
spam)
I am thinking to create a structure like this:
fieldMap = {'otherFooTable': [('idFoo','id'),('nameFoo','name'),('spam','foo')]}
I would be able to access this using (for example)
print fieldMap['tabelax'][0][1]
It´s not a very complex structure, but i can run into some problems using it? Is there any suggestions of how to handle this sort of issue? I need to store (for now) at least inputTable (i don´t want to repeat it for each field mapped), inputField,outputField. There is no reason to store outputTable, because that is always known beforehand.
Suggestions and past experiences are deeply appreciated.
PS: perhaps a formal structure (like a class) would be better?
Thanks

I'd honestly just take hints from (or use) SQLAlchemy or Django Models. These are tried and true data representation methods.

Here is a little wrapper class for FooB's to mimic FooA's, but still retain their FooB-ishness.
from collections import namedtuple
# use namedtuple to define some simple classes (requires Py2.6 or later)
FooA = namedtuple('FooA', 'id name type foo')
FooB = namedtuple('FooB', 'idfoo namefoo spam')
# create a wrapper class for FooB's to look like a FooA
class FooAMimic(object):
attrMap = dict(zip(FooA._fields, FooB._fields))
# or if the fields aren't nicely ordered, declare this mapping explicitly
#~ attrMap = { 'id' : 'idfoo', 'name' : 'namefoo', 'foo' : 'spam' }
def __init__(self, obj):
self.obj = obj
def __getattr__(self, aname):
ob = self.obj
if aname in self.attrMap:
return getattr(ob, self.attrMap[aname])
elif hasattr(ob, aname):
return getattr(ob, aname)
else:
raise AttributeError("no such attribute " + aname)
def __dir__(self):
return sorted(set(dir(super(FooAMimic,self))
+ dir(self.obj)
+ list(FooA._fields)))
Use it like this:
# make some objects, some FooA, some FooB
fa = FooA('a', 'b', 'c','d')
fb = FooB('xx', 'yy', 'zz')
fc = FooA('e', 'f', 'g','h')
# create list of items that are FooA's, or FooA lookalikes
coll = [fa, FooAMimic(fb), fc]
# access objects like FooA's, but notice that the wrapped FooB
# attributes are still available too
for f in sorted(coll, key=lambda k : k.id):
print f.id, '=',
try:
print f.namefoo, "(really a namefoo)"
except AttributeError:
print f.name
Prints:
a = b
e = f
xx = yy (really a namefoo)

Think about this
class Column( object ):
def __init__( self, name, type_information=None ):
self.name = name
self.type_information = type_information
self.pk = None
self.fk_ref = None
def fk( self, column ):
self.fk_ref = column
class Table( object ):
def __init__( self, name, *columns ):
self.name = name
self.columns = dict( (c.name, c) for c in columns )
def column( self, name ):
return self.columns[ name ]
Table( "FOOA", Column( "id" ), Column( "name" ), Column( "type" ), Column( "foo" ) )
Table( "otherFooTable", Column( "idFoo" ), Column( "nameFoo" ), Column( "spam" ) )
It's not clear at all what you're tying to do or why, so this is as good as anything, since it seems to represent the information you actually have.

Try to avoid accessing your data through fixed numerical indexes as in fieldMap['tabelax'][0][1]. After a year of not looking at your code, it may take you (or others) a while to figure out what it all means in human terms (e.g. "the value of idFoo in table tabelax"). Also, if you ever need to change your data structure (e.g. add another field) then some/all your numerical indexes may need fixing. Your code becomes ossified when the risk of breaking the logic prevents you from modifying the data structure.
It is much better to use a class and use class (accessor) methods to access the data structure. That way, the code outside of your class can be preserved even if you need to change your data structure (inside the class) at some future date.