We Keep Coding

How to use a custom TF.lite model with 2 classes on a Rasperry Pi with a Coral?

Two days ago, I created a custom model in Tflite from an image dataset. The accuracy is 97.4 % and it has only 2 classes (person, flower)
I converted the model to use it inside my Rasberry Pi with TPU Google Coral.
At the moment, I'm stuck with some problems. The documentation of Google Coral isn't really for me.
Language: Python3
Libraries
Keras
Tensorflow
Pillow
Picamera
Numpy
EdgeTPU-Engine
Project tree:
-------->model(sub-folder)
----------->model.tflite
----------->labels.txt
-------->video_detection.py
This is Python code: (Actually the code is from the documentation)
import argparse
import io
import time
import numpy as np
import picamera
import edgetpu.classification.engine
def main():
parser = argparse.ArgumentParser()
parser.add_argument(
'--model', help='File path of Tflite model.', required=True)
parser.add_argument(
'--label', help='File path of label file.', required=True)
args = parser.parse_args()
with open(args.label, 'r', encoding="utf-8") as f:
pairs = (l.strip().split(maxsplit=2) for l in f.readlines())
labels = dict((int(k), v) for k, v in pairs)
engine = edgetpu.classification.engine.ClassificationEngine(args.model)
with picamera.PiCamera() as camera:
camera.resolution = (640, 480)
camera.framerate = 30
_, width, height, channels = engine.get_input_tensor_shape()
camera.start_preview()
try:
stream = io.BytesIO()
for foo in camera.capture_continuous(stream,
format='rgb',
use_video_port=True,
resize=(width, height)):
stream.truncate()
stream.seek(0)
input = np.frombuffer(stream.getvalue(), dtype=np.uint8)
start_ms = time.time()
results = engine.ClassifyWithInputTensor(input, top_k=1)
elapsed_ms = time.time() - start_ms
if results:
camera.annotate_text = "%s %.2f\n%.2fms" % (
labels[results[0][0]], results[0][1], elapsed_ms*1000.0)
finally:
camera.stop_preview()
if __name__ == '__main__':
main()
How to run the script
python3 video_detection.py --model model/model.tflite --label model/labels.txt
Error
`Traceback (most recent call last):
File "video_detection.py", line 41, in <module>
main()
File "video_detection.py", line 16, in main
labels = dict((int(k), v) for k, v in pairs)
File "video_detection.py", line 16, in <genexpr>
labels = dict((int(k), v) for k, v in pairs)
ValueError: not enough values to unpack (expected 2, got 1)`
For me now it's very hard to integrate a custom model and use it with coral.
Documentation:
https://coral.withgoogle.com/docs/edgetpu/models-intro/
https://coral.withgoogle.com/docs/edgetpu/api-intro/
https://coral.withgoogle.com/docs/edgetpu/tflite-python/
https://coral.googlesource.com/edgetpu/+/refs/heads/release-chef/edgetpu/
Thanks for reading, best regards
E.

The error is in labels.txt file:
labels = dict((int(k), v) for k, v in pairs)
ValueError: not enough values to unpack (expected 2, got 1)`
Looks like you have some line(s) that have only one value instead of two

I get an JSON decode error when using Python and Google Vision to detect text on PDF file

I am trying to work with Google Vision and Python. I am using the sample files but I keep getting the same error message:
Traceback (most recent call last):
File "C:\Program Files (x86)\Python37-32\lib\site-packages\google\protobuf\jso
n_format.py", line 416, in Parse
js = json.loads(text, object_pairs_hook=_DuplicateChecker)
File "C:\Program Files (x86)\Python37-32\lib\json\__init__.py", line 361, in l
oads
return cls(**kw).decode(s)
File "C:\Program Files (x86)\Python37-32\lib\json\decoder.py", line 338, in de
code
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Program Files (x86)\Python37-32\lib\json\decoder.py", line 356, in ra
w_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "sample.py", line 72, in <module>
async_detect_document('gs://matr/file_1035.pdf','gs://matr/output/')
File "sample.py", line 59, in async_detect_document
json_string, vision.types.AnnotateFileResponse())
File "C:\Program Files (x86)\Python37-32\lib\site-packages\google\protobuf\jso
n_format.py", line 418, in Parse
raise ParseError('Failed to load JSON: {0}.'.format(str(e)))
google.protobuf.json_format.ParseError: Failed to load JSON: Expecting value: li
ne 1 column 1 (char 0).
I am guessing it has something to do with the resulting JSON file. It does produce a JSON file but i guess it should print it out to the command line. Here are the first few lines of the JSON file:
{
"inputConfig": {
"gcsSource": {
"uri": "gs://python-docs-samples-tests/HodgeConj.pdf"
},
"mimeType": "application/pdf"
},
I resulting file does load into a JSON object by using
data = json.load(jsonfile)
I have tried print (json_string) but I only get b'placeholder'
How can I get this to work? I am using Python 3.7.2
My code is below:
def async_detect_document(gcs_source_uri, gcs_destination_uri):
"""OCR with PDF/TIFF as source files on GCS"""
from google.cloud import vision
from google.cloud import storage
from google.protobuf import json_format
import re
# Supported mime_types are: 'application/pdf' and 'image/tiff'
mime_type = 'application/pdf'
# How many pages should be grouped into each json output file.
batch_size = 2
client = vision.ImageAnnotatorClient()
feature = vision.types.Feature(
type=vision.enums.Feature.Type.DOCUMENT_TEXT_DETECTION)
gcs_source = vision.types.GcsSource(uri=gcs_source_uri)
input_config = vision.types.InputConfig(
gcs_source=gcs_source, mime_type=mime_type)
gcs_destination = vision.types.GcsDestination(uri=gcs_destination_uri)
output_config = vision.types.OutputConfig(
gcs_destination=gcs_destination, batch_size=batch_size)
async_request = vision.types.AsyncAnnotateFileRequest(
features=[feature], input_config=input_config,
output_config=output_config)
operation = client.async_batch_annotate_files(
requests=[async_request])
print('Waiting for the operation to finish.')
operation.result(timeout=180)
# Once the request has completed and the output has been
# written to GCS, we can list all the output files.
storage_client = storage.Client()
match = re.match(r'gs://([^/]+)/(.+)', gcs_destination_uri)
bucket_name = match.group(1)
prefix = match.group(2)
bucket = storage_client.get_bucket(bucket_name=bucket_name)
# List objects with the given prefix.
blob_list = list(bucket.list_blobs(prefix=prefix))
print('Output files:')
for blob in blob_list:
print(blob.name)
# Process the first output file from GCS.
# Since we specified batch_size=2, the first response contains
# the first two pages of the input file.
output = blob_list[0]
json_string = output.download_as_string()
response = json_format.Parse(
json_string, vision.types.AnnotateFileResponse())
# The actual response for the first page of the input file.
first_page_response = response.responses[0]
annotation = first_page_response.full_text_annotation
# Here we print the full text from the first page.
# The response contains more information:
# annotation/pages/blocks/paragraphs/words/symbols
# including confidence scores and bounding boxes
print(u'Full text:\n{}'.format(
annotation.text))
async_detect_document('gs://my_bucket/file_1035.pdf','gs://my_bucket/output/')

I received an answer from a user on a github page.
https://github.com/GoogleCloudPlatform/python-docs-samples/issues/2086#issuecomment-487635159
I had this issue and determined it was caused by the prefix being iterated as part of the bloblist. I can see that "output/" is listed as a file in your output, and subsequently has parsing attempted on it causing the error.
Try hardcoding a prefix something like prefix = 'output/out' and that folder won't be included in the list.
The demo code should probably be modified to handle this simple case a little better.
import re
def async_detect_document(gcs_source_uri, gcs_destination_uri):
"""OCR with PDF/TIFF as source files on GCS"""
from google.cloud import vision
from google.cloud import storage
from google.protobuf import json_format
# Supported mime_types are: 'application/pdf' and 'image/tiff'
mime_type = 'application/pdf'
# How many pages should be grouped into each json output file.
batch_size = 2
client = vision.ImageAnnotatorClient()
feature = vision.types.Feature(
type=vision.enums.Feature.Type.DOCUMENT_TEXT_DETECTION)
gcs_source = vision.types.GcsSource(uri=gcs_source_uri)
input_config = vision.types.InputConfig(
gcs_source=gcs_source, mime_type=mime_type)
gcs_destination = vision.types.GcsDestination(uri=gcs_destination_uri)
output_config = vision.types.OutputConfig(
gcs_destination=gcs_destination, batch_size=batch_size)
async_request = vision.types.AsyncAnnotateFileRequest(
features=[feature], input_config=input_config,
output_config=output_config)
operation = client.async_batch_annotate_files(
requests=[async_request])
print('Waiting for the operation to finish.')
operation.result(timeout=180)
# Once the request has completed and the output has been
# written to GCS, we can list all the output files.
storage_client = storage.Client()
match = re.match(r'gs://([^/]+)/(.+)', gcs_destination_uri)
bucket_name = match.group(1)
prefix = match.group(2)
bucket = storage_client.get_bucket(bucket_name=bucket_name)
print ('prefix: ' + prefix)
prefix = 'output/out'
print ('prefix new: ' + prefix)
# List objects with the given prefix.
blob_list = list(bucket.list_blobs(prefix=prefix))
print('Output files:')
for blob in blob_list:
print(blob.name)
# Process the first output file from GCS.
# Since we specified batch_size=2, the first response contains
# the first two pages of the input file.
output = blob_list[0]
json_string = output.download_as_string()
response = json_format.Parse(
json_string, vision.types.AnnotateFileResponse())
# The actual response for the first page of the input file.
first_page_response = response.responses[0]
annotation = first_page_response.full_text_annotation
# Here we print the full text from the first page.
# The response contains more information:
# annotation/pages/blocks/paragraphs/words/symbols
# including confidence scores and bounding boxes
print(u'Full text:\n{}'.format(
annotation.text))
async_detect_document('gs://my_bucket/my_file.pdf','gs://my_bucket/output/out')

TypeError: coercing to Unicode: need string or buffer, type found when running python file

I have a python script that reads in data from a .csv file and uses it to make mathematical calculation on the data. When I run it, I get this error:
Traceback (most recent call last):
File "HW1_PythonTemplate.py", line 120, in <module>
print ','.join(map(str,calculate(args.data, args.i)))
File "HW1_PythonTemplate.py", line 56, in calculate
with open(file, 'r') as csvfile:
TypeError: coercing to Unicode: need string or buffer, type found
My code looks like:
import argparse
import csv
import sys
def calculate( dataFile, ithAttr):
numObj, minValue, maxValue, mean, stdev, Q1, median, Q3, IQR = [0,"inf","-inf",0,0,0,0,0,0]
rows = []
with open(file, 'r') as csvfile:
csvreader = csv.reader(csvfile)
for row in csvreader:
rows.append(row)
columniStr = [row[ithAttr-1] for row in rows]
columniFloat = []
for value in columniStr:
try:
columniFloat.append(float(value))
except ValueError:
pass
In the calculate function, everything past that is just arbitrary math.
My main looks like:
if __name__ == "__main__":
parser = argparse.ArgumentParser(description='calc')
parser.add_argument('--i', type=int,
help="ith attribute of the dataset (2 <= i <= 29)",
default=5,
choices=range(2,30),
required=True)
parser.add_argument("--data", type=str,
help="Location of the dataset file",
default="energydata_complete.csv",
required=True)
args = parser.parse_args()
print ','.join(map(str,calculate(args.data, args.i)))

with open(file
You mis-spelled dataFile.
file is the built-in Python datatype for file objects, so you're accidentally trying to open a type.

Python 3.5 Tkinter how to save dict to file

So my my problem is that I'm running a program once it has finished base functions a pop up box appears asking the user if they would like to save file, if 'yes' then a save dialog box appears. Because the data I'm saving is a dict value I'm receiving an Error from tkinter. I have attempted to use the ".csv" extension as a save point as i read somewhere that dict's can be saved to them, but i'm either going about this wrong way or there is an issue within my code.
Updated Code and explanation why below
Original snippet of code:
def flag_detection():
total_count = Counter(traffic_light)
total_count.keys()
for key, value in total_count.items():
EWT = tkinter.messagebox.askquestion('File Level', 'Would you like to save')
file_opt = options = {}
options['filetypes'] = [('all files', '.*'), ('text files', '.csv')]
options['initialfile'] = 'myfile.csv'
if EWT == 'yes':
savename = asksaveasfile(file_opt, defaultextension=".csv")
savename.write(key, ':', value)
Error message:
Exception in Tkinter callback
Traceback (most recent call last):
File "C:\Users\Lewis Collins\AppData\Local\Programs\Python\Python35-32\lib\tkinter\__init__.py", line 1550, in __call__
return self.func(*args)
File "C:/Users/Lewis Collins/PycharmProjects/program_06.01.17/Home.py", line 108, in run_language_model
main.flag_detection()
File "C:\Users\Lewis Collins\PycharmProjects\program_06.01.17\main_code\main.py", line 179, in flag_detection
savename = asksaveasfile(file_opt, defaultextension=".csv")
File "C:\Users\Lewis Collins\AppData\Local\Programs\Python\Python35-32\lib\tkinter\filedialog.py", line 423, in asksaveasfile
return open(filename, mode)
TypeError: open() argument 2 must be str, not dict
Because of Tkinter throwing back that it can not save a dict to file i tried the below solution of converting the dict to a str which has also caused problems
Code Snippet of Function attempt to convert to str for tkinter:
def flag_detection():
total_count = Counter(traffic_light)
total_count.keys()
for key, value in str(total_count.items()):
EWT = tkinter.messagebox.askquestion('File Level', 'Would you like to save')
file_opt = options = {}
options['filetypes'] = [('all files', '.*'), ('text files', '.csv')]
options['initialfile'] = 'myfile.csv'
if EWT == 'yes':
savename = asksaveasfile(file_opt, defaultextension=".csv")
savename.write(key, ':', value)
So I've updated my code to try and use the str(total_count.items()): to convert to dict as i didn't quite understand the json and pickle libraries after reading them they seemed to complicated for what i needed which is a simple output to a file for a user to be able to go and view.
I am now receiving this Error:
Exception in Tkinter callback
Traceback (most recent call last):
File "C:\Users\Lewis Collins\AppData\Local\Programs\Python\Python35-32\lib\tkinter\__init__.py", line 1550, in __call__
return self.func(*args)
File "C:/Users/Lewis Collins/PycharmProjects/program_05.0.17/Home.py", line 108, in run_language_model
main.flag_detection()
File "C:\Users\Lewis Collins\PycharmProjects\program_05.0.17\main_code\main.py", line 169, in flag_detection
for key, value in str(total_count.items()):
ValueError: not enough values to unpack (expected 2, got 1)
Any suggestions or feedback is welcome, Thanks in advance.

The first problem is this line:
savename = asksaveasfile(file_opt, defaultextension=".csv")
That is simply not how to call asksaveasfile. asksaveasfile doesn't take a dictionary as its first argument. You should call it this way if you want to use the options in file_opt1:
savename = asksaveasfile(defaultextension=".csv", **file_opt)
When you fix that, the next problem is where you try to write with this statement:
savename.write(key, ':', value)
You get this error message: TypeError: write() takes exactly 1 argument (3 given). It means exactly what it says: you need to provide a single argument rather than three arguments. You can solve that by giving write exactly 1 argument:
savename.write("%s: %s" % (key, value))
However, if all you want to do is save a dictionary to a file, the json module makes this quite easy, without having to iterate over the values.
To save as json, change your flag_detection method to look like this:
import json
...
def flag_detection():
total_count = Counter(traffic_light)
EWT = tkinter.messagebox.askquestion('File Level', 'Would you like to save')
file_opt = options = {}
options['filetypes'] = [('all files', '.*'), ('text files', '.json')]
options['initialfile'] = 'myfile.json'
if EWT == 'yes':
savefile = asksaveasfile(defaultextension=".json", **file_opt)
json.dump(total_count, savefile)
If you want to save as a csv file, read the documentation on the DictWriter class which works in a similar way.

Unable to serialize JSON serialize dictionary to file in Python

Apologize pretty new to python and I'm not 100% sure why this is failing since all example code I see is really similar.
import io
import json
import argparse
from object_detection.helpers import average_bbox
ap = argparse.ArgumentParser()
ap.add_argument("-o","--output",required=True,help="Output file name.")
ap.add_argument("-c","--class",help="Object class name")
ap.add_argument("-a","--annotations",required=True,help="File path annotations are located in")
args = vars(ap.parse_args())
(avgW,avgH) = average_bbox(args["annotations"])
if args["class"] is None:
name = args["annotations"].split("/")[-1]
else:
name = args["class"]
with io.open(args["output"],'w') as f:
o = {}
o["class"] = name
o["avgWidth"] = avgW
o["avgHeight"] = avgH
f.write(json.dumps(o,f))
name, avgW and avgH are all valid values. avgW and avgH are numbers and name is a string. The output seems like a valid path to create a file.
Error I get is
Traceback (most recent call last):
File "compute_average_bbox.py", line 19, in <module>
with io.open(argparse["output"],'w') as f:
TypeError: 'module' object has no attribute '__getitem__'
Any help would be appreciated.