This question already has answers here:
Possible to enforce type hints?
(2 answers)
Closed 12 months ago.
I struggle with Python(3.9.7) automatically adding singlequotes around a float-value, which leads to a failing assertion.
My goal is to use pytest to assert an expected output of a parsing-function. The parser takes a json-object (shortened code sample below) and returns a list of Signal objects. I copied the print-out from logging.debug(f"{signal_list}") inside the parsing function, and assigned it to expected_result in my test-function:
#Arrange
message = json.dumps({"data":{"name":"battery_volt","alias":"Volt","unit":"volt","value":12.0,"time":1644587969}})
expected_result = [Signal(id=None, name='battery_volt', description=None, value=12.0, type=None, unit='volt', time='2022-02-11T13:59:29')]
print(expected_result)
p = Parser(message)
#Act
result = p.parse_message()
#Assert
assert result == expected_result
Irritatingly, pytest -vv throws an AssertionError:
E Full diff:
E - [Signal(id=None, name='battery_volt', description=None, value='12.0', type=None, unit='volt', time='2022-02-11T13:59:29')]
E ? - -
E + [Signal(id=None, name='battery_volt', description=None, value=12.0, type=None, unit='volt', time='2022-02-11T13:59:29')]
The upper line seems to be the value of expected_result, because print(expected_result)
also adds the singlequotes around the 12.0
I assume the copied output from logging.debug(f"{signal_list}") isn't the same as the real value of result. I tried typecasting expected_result as list, str()-converting both result and expected_result inside the test, but expected_result always has '12.0' and result has 12.0.
I desperatly need a hint how to do this kind of assertion the correct way.
EDIT:
Here is the parsing function:
def parse_message(self):
message = json.loads(self.message)
#logging.debug(f"message is: {message}")
message_data = message.get('data', {})
parsed_data = []
try:
device = message_data.get('device', None)
if device is not None:
vehicle = self.parse_vehicle(device)
parsed_data.append(vehicle)
else:
logging.error("'device' was None!")
except Exception as e:
logging.error(e)
signals = []
try:
data = message_data.get('data', None)
if data is not None:
signals = self.parse_signals(data)
gps = message_data.get('gps', None)
if gps is not None:
gps_signal = self.parse_gps(gps)
signals.append(gps_signal)
parsed_data.append(signals)
except Exception as e:
logging.error(e)
return parsed_data
if __name__ == "__main__":
setup_logging()
message = json.dumps({"consumerid":redacted,"data":{"device":{"time":1644587969,"imei":"redacted","man_id":redacted,"car_id":999,"vin":"redacted"},"data":[{"name":"battery_volt","alias":"Volt","unit":"volt","value":12.0,"time":1644587969}],"gps":{"lat":51.437515,"lon":6.9281199,"dir":252,"alt":88,"sat":19,"time":1644587969}},"event":"redacted","itemid":redacted,"itemtype":1,"senderip":"redacted"})
p = Parser(message)
signal_list = p.parse_message()
logging.debug(f"{signal_list}")
Please note that the passed json-objects are more complex than the code-sample in the original post.
class Signal(BaseModel):
id: int = None
name: str = None
description: str = None
value: str = None
type: str = None
unit: str = None
time: str = None
EDIT2 - Assignment of Signal.value happens here:
def parse_signals(self, data):
signals = []
#logging.debug(f"data is : {data}")
for data_object in data:
signal = Signal()
try:
signal.name = data_object.get('name', None)
#get dtc-count value as seperate return-element, needed by the controller to handle dtc-codes
if signal.name == 'dtc_count':
self.dtc_count == data_object.get('value', None)
signal.description = data_object.get('description', None)
signal.value = data_object.get('value', None)
signal.time = datetime.datetime.utcfromtimestamp(data_object.get('time', None)).isoformat()
signal.unit = data_object.get('unit', None)
if signal.unit == "dtc":
signal.type = "1"
if signal.name is not None:
signals.append(signal)
#logging.debug(signal.__dict__)
except Exception as e:
logging.error(f"While parsing signal {data_object}, the following error occured: {e}")
return signals
When parse_message is called as __name__ == "main":, the testcode beneath outputs value=12.0
Ok, turns out python type hints don't enforce like I expected:
When parse_message is called by way of __name__ == "main":, the testcode beneath outputs value=12.0 Apparently, despite the type-hint :str for Signal.value, 12.0 was assigned as float. When I tried to sys.stdout.write(signal_list), I got a TypeError.
I now simply str()-convert in parse_signals() like this
signal.value = str(data_object.get('value', None))
resulting in having my value in single quotes consistently.
Related
I want the function to return a value when there is an error. I created this sample code, but it has two parameters. How can I make the function parameters optional? Because when I run the code below, my IDE returns an error that the parameters are required.
def test(error, parameter1):
if parameter1 == True:
print("true")
else:
print("false")
array = []
try:
array[0]
except Exception as e:
error = e
return error
value1 = 1
value2 = 2
if value1 == 1:
#I want this to print the error of the function above
print(test(error))
if value2 == 2:
# I want this to pass parameter to the function above
test(parameter1=True)
Optional paramteters are declared with =None, i.e.:
def test(error=None, parameter1=None):
...
in your function you'll have to check if the parameters are not none, i.e.:
if error is not None:
....
You could set the function parameters some default values so when no parameters are passed, the default values become the values of the variables:
def test(error = None, parameter1 = None):
if parameter1 == True:
print("true")
else:
print("false")
array = []
try:
array[0]
except Exception as e:
error = e
return error
Here None becomes a default value and if a parameter is passed, the value changes from None to the given value.
I wrote a code to catch my error message using try and except (I want to write it without using isinstance) and I am getting the error message when the input is not an integer. The problem is, the program is giving me error message even if the input is valid integer. Please give me some suggestions to make it run. My code is given below:
I tried using exception clause but that did not work.
class Hotel:
def __init__(self,room,catagory):
if room != int:
raise TypeError ()
self.room = room
self.catagory = catagory
self.catagories = {"A":"Elite","B":"Economy","C":"Regular"}
self.rooms = ["0","1","2","3","4","5"]
def getRoom(self):
return self.room
def getCatagory(self):
return self.catagory
return self.catagories.get(self.catagory)
def __str__(self):
return "%s and %s"%(self.rooms[self.room],self.catagories.get(self.catagory))
try:
room1 = Hotel(a,"A")
room2 = Hotel(1,"A")
print(room1.getRoom())
except:
print("there's an error")
I am expecting:
there's an error
1 and Elite
A
You are checking if room != int . It will give you error always.
You have to check type(room)!= int .
I have corrected the code below
class Hotel:
def __init__(self,room,catagory):
if type(room) != int:
raise TypeError ()
self.room = room
self.catagory = catagory
self.catagories = {"A":"Elite","B":"Economy","C":"Regular"}
self.rooms = ["0","1","2","3","4","5"]
Suppose I have some (simplified) BeautifulSoup code like this, pulling data into a dictionary:
tournament_info = soup.find_all('li')
stats['Date'] = tournament_info[0].text
stats['Location'] = tournament_info[1].text
stats['Prize'] = tournament_info[3].text.split(':')[1].strip()
In the case where the initial find_all returns an exception, I want all the dictionary entries to be 'None'. And in the case of any of the individual dictionary assignments returning an exception, I want 'None' too.
Is there any nice way to write this, other than something horrible like below?
try:
tournament_info = soup.find_all('li')
except:
m_stats['Date'] = 'None'
m_stats['Location'] = 'None'
m_stats['Prize'] = 'None'
try:
m_stats['Date'] = tournament_info[0].text
except:
m_stats['Date'] = 'None'
try:
m_stats['Location'] = tournament_info[1].text
except:
m_stats['Location'] = 'None'
try:
m_stats['Prize'] = tournament_info[3].text.split(':')[1].strip()
except:
m_stats['Prize'] = 'None'
Create own class
class Stats(dict):
tournament_info = []
def __init__(self, tournament_info, **kwargs):
super(Stats, self).__init__(**kwargs)
self.tournament_info = tournament_info
self['Date'] = self.get_tournament_info_text(0)
self['Location'] = self.get_tournament_info_text(1)
prize = self.get_tournament_info_text(2)
if prize is not None:
prize = prize.split(':')[1].strip()
self['Prize'] = prize
def get_tournament_info_text(self, index):
try:
return self.tournament_info[index]['text']
except:
return None
tournament_info = [
{
'text': 'aaa'
},
{},
{
'text': 'bbb:ccc '
}
]
m_stats = Stats(tournament_info)
print m_stats
Here's what I can suggest for your code:
info = soup.find_all('li')
if not info:
m_stats = dict.fromkeys(m_stats, None)
return
mappings = {
'Date': 0,
'Location': 1,
'Prize': 3
}
for key in mappings:
value = None
try:
value = info[mappings[key]].text
if mappings[key] == 3:
value = value.split(':')[1].strip()
except IndexError:
pass
m_stats[key] = value
Alternatively, you can create a function that will handle the exceptions for you:
def get_value(idx):
value = None
try:
value = info[idx].text
except IndexError:
pass
return value
m_stats['Date'] = get_value(0)
m_stats['Location'] = get_value(1)
m_stats['Prize'] = get_value(3)
if m_stats['Prize']:
m_stats['Prize'].split(':')[1].strip()
The solution I went for was to create a blank template dictionary (actually a JSON) with all the keys set to 'None'.
Every time the page is scraped, m_stats is first initialised with this blank dictionary (loaded from the JSON). If an exception occurs, it is just simply passed (with some logging), and the value is left as 'None'. There is then no need to explicitly assign 'None' every single time.
Not sure if it's correct to mark this as the "answer", as it is quite specific to my needs, but that's what I did anyway.
in fucntion getLink(urls), I have return (cloud,parent,children)
in main function, I have (cloud,parent,children) = getLink(urls) and I got error of this line: TypeError: 'NoneType' object is not iterable
parent and children are all list of http links. since, it is not able to paste them here, parent is a list contains about 30 links; children is a list contains about 30 items, each item is about 10-100 links which is divide by ",".
cloud is a list contain about 100 words, like that: ['official store', 'Java Applets Centre', 'About Google', 'Web History'.....]
I didnot know why I get an error. Is there anything wrong in passing parameter? Or because the list take too much space?
#crawler url: read webpage and return a list of url and a list of its name
def crawler(url):
try:
m = urllib.request.urlopen(url)
msg = m.read()
....
return (list(set(list(links))),list(set(list(titles))) )
except Exception:
print("url wrong!")
#this is the function has gone wrong: it throw an exception here, also the error I mentioned, also it will end while before len(parent) reach 100.
def getLink(urls):
try:
newUrl=[]
parent = []
children =[]
cloud =[]
i=0
while len(parent)<=100:
url = urls[i]
if url in parent:
i += 1
continue
(links, titles) = crawler(url)
parent.append(url)
children.append(",".join(links))
cloud = cloud + titles
newUrl= newUrl+links
print ("links: ",links)
i += 1
if i == len(urls):
urls = list(set(newUrl))
newUrl = []
i = 0
return (cloud,parent,children)
except Exception:
print("can not get links")
def readfile(file):
#not related, this function will return a list of url
def main():
file='sampleinput.txt'
urls=readfile(file)
(cloud,parent,children) = getLink(urls)
if __name__=='__main__':
main()
There might be a way that your function ends without reaching the explicit return statement.
Look at the following example code.
def get_values(x):
if x:
return 'foo', 'bar'
x, y = get_values(1)
x, y = get_values(0)
When the function is called with 0 as parameter the return is skipped and the function will return None.
You could add an explicit return as the last line of your function. In the example given in this answer it would look like this.
def get_values(x):
if x:
return 'foo', 'bar'
return None, None
Update after seing the code
When the exception is triggered in get_link you just print something and return from the function. You have no return statement, so Python will return None. The calling function now tries to expand None into three values and that fails.
Change your exception handling to return a tuple with three values like you do it when everything is fine. Using None for each value is a good idea for it shows you, that something went wrong. Additionally I wouldn't print anything in the function. Don't mix business logic and input/output.
except Exception:
return None, None, None
Then in your main function use the following:
cloud, parent, children = getLink(urls)
if cloud is None:
print("can not get links")
else:
# do some more work
Ok, I recently started programming in Python, and I really like it.
However, I have run into a little issue.
I want to be able to define a function to take in some data and assign it to a variable that I designate, rather than have to perform the operation every time I want to submit the value.
Here is a code fragment:
try:
if elem.virtual.tag:
virt = True
temp_asset.set_virtual(True)
except AttributeError:
temp_asset.set_virtual(False)
if virt: #if virtual, get only faction, value, and range for presence
try:
fac = elem.presence.faction #an xml tag (objectified)
except AttributeError:
fac = "faction tag not found"
temp_asset.misload = True
try:
val = elem.presence.value
except AttributeError:
val = "value tag not found"
temp_asset.misload = True
try:
rang = elem.presence.range
except AttributeError:
rang = "range tag not found"
temp_asset.misload = True
#Set presence values
temp_asset.set_presence(fac, val, rang)
The functions set the values, but I want to be able to perform the error checking with something like this:
def checkval(self, variable_to_set, tag_to_use)
try:
variable_to_set = tag_to_use
except AttributeError:
variable_to_set = "tag not found"
temp_asset.misload = True
Is this doable? Let me know if I need to show more code.
Edit: I don't need pointers per se, just anything that works this way and saves typing.
Edit 2: Alternatively, I need a solution of how to check whether an objectified xml node exists (lxml).
Have you tried/looked into the getattr and setattr functions?
For example, assuming these "variables" are object attributes:
def checkval(self, attr, presence, tagstr):
tag = getattr(presence, tagstr, None) # tag = presence."tagstr" or None
setattr(self, attr, tag or 'tag not found') # ?? = presence."tagstr" or 'tag not found'
if tag is None:
self.temp_asset.misload = True
You call it like,
your_object.checkval('fac', elem.presence, 'faction')
Alternatively, you can pre-define these variables and set them default values before you attempt to look up the tags. For example:
class YourObject(object):
_attrmap = {
'fac': 'faction',
'val': 'value',
'rang': 'range',
}
def __init__(self):
# set default values
for attr, tagstr in self._attrmap.items():
setattr(self, attr, '%s tag not found' % tagstr)
def checkval(self, attr, presence):
for attr, tagstr in self._attrmap.items():
tag = getattr(presence, tagstr, None)
if tag is not None:
setattr(self, attr, tag)
else:
self.temp_asset.misload = True