We Keep Coding

Using String parameter for nvidia triton

I'm trying to deploy a simple model on the Triton Inference Server. It is loaded well but I'm having trouble formatting the input to do a proper inference request.
My model has a config.pbtxt set up like this
max_batch_size: 1
input: [
{
name: "examples"
data_type: TYPE_STRING
format: FORMAT_NONE
dims: [ -1 ]
is_shape_tensor: false
allow_ragged_batch: false
optional: false
}
]
I've tried using a pretty straightforward python code to setup the input data like this (the outputs are not written but are setup correctly)
bytes_data = [input_data.encode('utf-8')]
bytes_data = np.array(bytes_data, dtype=np.object_)
bytes_data = bytes_data.reshape([-1, 1])
inputs = [
httpclient.InferInput('examples', bytes_data.shape, "BYTES"),
]
inputs[0].set_data_from_numpy(bytes_data)
But I keep getting the same error message
tritonclient.utils.InferenceServerException: Could not parse example input, value: '[my text input here]'
[[{{node ParseExample/ParseExampleV2}}]]
I've tried multiple ways of encoding the input, as bytes or even as TFX serving used to ask like this { "instances": [{"b64": "CjEKLwoJdXR0ZXJhbmNlEiIKIAoecmVuZGV6LXZvdXMgYXZlYyB1biBjb25zZWlsbGVy"}]}
I'm not exactly sure where the problems comes from if anyone knows?

If anyone gets this same problem, this solved it. I had to create a tf.train.Example() and set the data correctly
example = tf.train.Example()
example_bytes = str.encode(input_data)
example.features.feature['utterance'].bytes_list.value.extend([example_bytes])
inputs = [
httpclient.InferInput('examples', [1], "BYTES"),
]
inputs[0].set_data_from_numpy(np.asarray(example.SerializeToString()).reshape([1]), binary_data=False)

How to find distance between face encodings when stored multiple for the same person, in THIS scenario... ( Python )

I am working on a project based on the facial recognition and verification.
I am using Siamese network to get the 128 vector of the face ( Embeddings ).
I am storing the encodings/embeddings of the person's face in the database and then checking or say matching the incoming face's encodings with the previously stored encodings to recognize the person.
To make a robust system, I have to store more than one encodings of the same person.
When I have used only a single encoding vector, and matched with :
face_recognition.compare_faces( stored_list_of_encodings, checking_image_encodings )
That dosen't work all the time because I have only compared with a single encoding.
To make a system sufficient for most cases, I want to store minimum 3 encodings of a same person and then compare with the new data.
Now the question :
How to store multiple embeddings of a same person and then compare the distance ?
Please Help me with this...
( Using face_recognition as the library and Siamese Network for feature extraction )

In order to get encoding of each images of person you can use face_encodings function of face_recognition library, then by comparing distance between these encodings you can improve your system in your own way.
for example:
encoding_1 = face_recognition.face_encodings(img_1)
encoding_2 = face_recognition.face_encodings(img_2)
encoding_3 = face_recognition.face_encodings(img_3)
encodings = [encoding_1,encoding_2,encoding3]
new_encoding = face_recognition.face_encodings(new_image)
for encoding in encodings:
#to look at distance between two encoding vector, you can use numpy linear algebra functions (you can also use other distance metrics)
distance = np.linalg.norm(new_encoding - encoding)
if distance < threshold:
return True

How do I compare multiple faces per person
Given that your database looks something like:
face_table {
'person_name' : String,
'embedding' : Array of 128 floats,
'other_data' : Something,
...
}
and you have a function
face_recognition.compare_faces(encoding_list, reference_encoding)
which returns a probability score per element of encoding_list and how well it matches the referene_encoding.
and your main loop does something like
name_list, encoding_list = zip(query_database_for_encodings())
results = face_recognition.compare_faces(encoding_list, reference_encoding)
best = np.argmax(results)
return name_list[best]
Solution 1 - add records
In this solution you can simply add multiple records to your database for the same person.
{ # record 1
'person_name' : 'Alice',
'embedding' : Array of 128 floats,
'photo_number' : 1
}
{ # record 2
'person_name' : 'Alice',
'embedding' : Array of 128 floats,
'photo_number' : 2
}
{ # record 3
'person_name' : 'Bob',
'embedding' : Array of 128 floats,
'photo_number' : 1
}
and so on having multiple records per person.
Solution 2 - Store an array of embeddings
In this solution you just have an array of embeddings per person in the database like so
face_table {
'person_name' : String,
'embedding_list' : Array of (Array of 128 floats),
'other_data' : Something,
...
}
and your other main loop code just changes to handle it possibly like so
records = query_database_returning_dictionary_results()
def generate_name_embedding_pairs(records):
for record in records:
name = record['person_name']
for encoding_list in record['embedding_list']:
yield name, encoding_list
name_list, encoding_list = zip(list(generate_name_embedding_pairs(records)))
results = face_recognition.compare_faces(encoding_list, reference_encoding)
best = np.argmax(results)
return name_list[best]

How can I filter tf.data.Dataset by specific values?

I create a dataset by reading the TFRecords, I map the values and I want to filter the dataset for specific values, but since the result is a dict with tensors, I am not able to get the actual value of a tensor or to check it with tf.cond() / tf.equal. How can I do that?
def mapping_func(serialized_example):
feature = { 'label': tf.FixedLenFeature([1], tf.string) }
features = tf.parse_single_example(serialized_example, features=feature)
return features
def filter_func(features):
# this doesn't work
#result = features['label'] == 'some_label_value'
# neither this
result = tf.reshape(tf.equal(features['label'], 'some_label_value'), [])
return result
def main():
file_names = ["/var/data/file1.tfrecord", "/var/data/file2.tfrecord"]
dataset = tf.contrib.data.TFRecordDataset(file_names)
dataset = dataset.map(mapping_func)
dataset = dataset.shuffle(buffer_size=10000)
dataset = dataset.filter(filter_func)
dataset = dataset.repeat()
iterator = dataset.make_one_shot_iterator()
sample = iterator.get_next()

I am answering my own question. I found the issue!
What I needed to do is tf.unstack() the label like this:
label = tf.unstack(features['label'])
label = label[0]
before I give it to tf.equal():
result = tf.reshape(tf.equal(label, 'some_label_value'), [])
I suppose the problem was that the label is defined as an array with one element of type string tf.FixedLenFeature([1], tf.string), so in order to get the first and single element I had to unpack it (which creates a list) and then get the element with index 0, correct me if I'm wrong.

I think you don't need to make label a 1-dimensional array in the first place.
with:
feature = {'label': tf.FixedLenFeature((), tf.string)}
you won't need to unstack the label in your filter_func

Reading, filtering a dataset is very easy and there is no need to unstack anything.
to read the dataset:
print(my_dataset, '\n\n')
##let us print the first 3 records
for record in my_dataset.take(3):
##below could be large in case of image
print(record)
##let us print a specific key
print(record['key2'])
To filter is equally simple:
my_filtereddataset = my_dataset.filter(_filtcond1)
where you define _filtcond1 however you want. Let us say there is a 'true' 'false' boolean flag in your dataset, then:
#tf.function
def _filtcond1(x):
return x['key_bool'] == 1
or even a lambda function:
my_filtereddataset = my_dataset.filter(lambda x: x['key_int']>13)
If you are reading a dataset which you havent created or you are unaware of the keys (as seems to be the OPs case), you can use this to get an idea of the keys and structure first:
import json
from google.protobuf.json_format import MessageToJson
for raw_record in noidea_dataset.take(1):
example = tf.train.Example()
example.ParseFromString(raw_record.numpy())
##print(example) ##if image it will be toooolong
m = json.loads(MessageToJson(example))
print(m['features']['feature'].keys())
Now you can proceed with the filtering

You should try to use the apply function from
tf.data.TFRecordDataset tensorflow documentation
Otherwise... read this article about TFRecords to get a better knowledge about TFRecords TFRecords for humans
But the most likely situation is that you can not access neither modify a TFRecord...there is a request on github about this topic TFRecords request
My advice is to make the things as easy as you can...you have to know that you are you working with graph and sessions...
In any case...if everything fail try the part of the code that does not work in a tensorflow session as simple as you can do it...probably all these operations should be done when tf.session is running...

how to store numpy arrays as tfrecord?

I am trying to create a dataset in tfrecord format from numpy arrays. I am trying to store 2d and 3d coordinates.
2d coordinates are numpy array of shape (2,10) of type float64
3d coordinates are numpy array of shape (3,10) of type float64
this is my code:
def _floats_feature(value):
return tf.train.Feature(float_list=tf.train.FloatList(value=value))
train_filename = 'train.tfrecords' # address to save the TFRecords file
writer = tf.python_io.TFRecordWriter(train_filename)
for c in range(0,1000):
#get 2d and 3d coordinates and save in c2d and c3d
feature = {'train/coord2d': _floats_feature(c2d),
'train/coord3d': _floats_feature(c3d)}
sample = tf.train.Example(features=tf.train.Features(feature=feature))
writer.write(sample.SerializeToString())
writer.close()
when i run this i get the error:
feature = {'train/coord2d': _floats_feature(c2d),
File "genData.py", line 19, in _floats_feature
return tf.train.Feature(float_list=tf.train.FloatList(value=value))
File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\google\protobuf\internal\python_message.py", line 510, in init
copy.extend(field_value)
File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\google\protobuf\internal\containers.py", line 275, in extend
new_values = [self._type_checker.CheckValue(elem) for elem in elem_seq_iter]
File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\google\protobuf\internal\containers.py", line 275, in <listcomp>
new_values = [self._type_checker.CheckValue(elem) for elem in elem_seq_iter]
File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\site-packages\google\protobuf\internal\type_checkers.py", line 109, in CheckValue
raise TypeError(message)
TypeError: array([-163.685, 240.818, -114.05 , -518.554, 107.968, 427.184,
157.418, -161.798, 87.102, 406.318]) has type <class 'numpy.ndarray'>, but expected one of: ((<class 'numbers.Real'>,),)
I dont know how to fix this. should i store the features as int64 or bytes? I have no clue how to go about this since i am completely new to tensorflow. any help would be great! thanks

The function _floats_feature described in the Tensorflow-Guide expects a scalar (either float32 or float64) as input.
def _float_feature(value):
"""Returns a float_list from a float / double."""
return tf.train.Feature(float_list=tf.train.FloatList(value=[value]))
As you can see the inputted scalar is written into a list (value=[value]) which is subsequently given to tf.train.FloatList as input. tf.train.FloatList expects an iterator that outputs a float in each iteration (as the list does).
If your feature is not a scalar but a vectur, _float_feature can be rewritten to pass the iterator directly to tf.train.FloatList (instead of putting it into a list first).
def _float_array_feature(value):
return tf.train.Feature(float_list=tf.train.FloatList(value=value))
However if your feature has two or more dimensions this solution does not work anymore. Like #mmry described in his answer in this case flattening your feature or splitting it into several one-dimensional features would be a solution. The disadvantage of these two approaches is that the information about the actual shape of the feature is lost if no further effort is invested.
Another possibility to write an example message for a higher dimensional array is to convert the array into a byte string and then use the _bytes_feature function described in the Tensorflow-Guide to write the example message for it. The example message is then serialized and written into a TFRecord file.
import tensorflow as tf
import numpy as np
def _bytes_feature(value):
"""Returns a bytes_list from a string / byte."""
if isinstance(value, type(tf.constant(0))): # if value ist tensor
value = value.numpy() # get value of tensor
return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))
def serialize_array(array):
array = tf.io.serialize_tensor(array)
return array
#----------------------------------------------------------------------------------
# Create example data
array_blueprint = np.arange(4, dtype='float64').reshape(2,2)
arrays = [array_blueprint+1, array_blueprint+2, array_blueprint+3]
#----------------------------------------------------------------------------------
# Write TFrecord file
file_path = 'data.tfrecords'
with tf.io.TFRecordWriter(file_path) as writer:
for array in arrays:
serialized_array = serialize_array(array)
feature = {'b_feature': _bytes_feature(serialized_array)}
example_message = tf.train.Example(features=tf.train.Features(feature=feature))
writer.write(example_message.SerializeToString())
The serialized example messages stored in the TFRecord file can be accessed via tf.data.TFRecordDataset. After the example messages have been parsed, the original array needs to be restored from the byte string it was converted to. This is possible via tf.io.parse_tensor.
# Read TFRecord file
def _parse_tfr_element(element):
parse_dic = {
'b_feature': tf.io.FixedLenFeature([], tf.string), # Note that it is tf.string, not tf.float32
}
example_message = tf.io.parse_single_example(element, parse_dic)
b_feature = example_message['b_feature'] # get byte string
feature = tf.io.parse_tensor(b_feature, out_type=tf.float64) # restore 2D array from byte string
return feature
tfr_dataset = tf.data.TFRecordDataset('data.tfrecords')
for serialized_instance in tfr_dataset:
print(serialized_instance) # print serialized example messages
dataset = tfr_dataset.map(_parse_tfr_element)
for instance in dataset:
print()
print(instance) # print parsed example messages with restored arrays

The tf.train.Feature class only supports lists (or 1-D arrays) when using the float_list argument. Depending on your data, you might try one of the following approaches:
Flatten the data in your array before passing it to tf.train.Feature:
def _floats_feature(value):
return tf.train.Feature(float_list=tf.train.FloatList(value=value.reshape(-1)))
Note that you might need to add another feature to indicate how this data should be reshaped when you parse it again (and you could use an int64_list feature for that purpose).
Split the multidimensional feature into multiple 1-D features. For example, if c2d contains an N * 2 array of x- and y-coordinates, you could split that feature into separate train/coord2d/x and train/coord2d/y features, each containing the x- and y-coordinate data, respectively.

The documentation about Tfrecord recommends to use serialize_tensor
TFRecord and tf.train.Example
Note: To stay simple, this example only uses scalar inputs. The simplest way to handle non-scalar features is to use tf.io.serialize_tensor to convert tensors to binary-strings. Strings are scalars in tensorflow. Use tf.io.parse_tensor to convert the binary-string back to a tensor.
2 lines of code does the trick for me:
tensor = tf.convert_to_tensor(array)
result = tf.io.serialize_tensor(tensor)

Tensorflow Split Using Feed Dict Input Dimension

I'm trying to tf.split a tensor based on the dimension of an input fed in using feed_dict (dimension of input changes with each batch). Currently I keep getting an error saying that a tensor cannot be split with a "Dimension". Is there a way to get the value of the dimension and split using it?
Thanks!
input_d = tf.placeholder(tf.int32, [None, None], name="input_d")
# toy feed dict
feed = {
input_d: [[20,30,40,50,60],[2,3,4,5,-1]] # document
}
W_embeddings = tf.get_variable(shape=[vocab_size, embedding_dim], \
initializer=tf.random_uniform_initializer(-0.01, 0.01),\
name="W_embeddings")
document_embedding = tf.gather(W_embeddings, input_d)
timesteps_d = document_embedding.get_shape()[1]
doc_input = tf.split(1, timesteps_d, document_embedding)

tf.split takes a python integer for the num_split argument. However, document_embedding.get_shape() returns a TensorShape, and document_embedding.get_shape()[1] gives a Dimension instance, hence you get an error says "can't split with a Dimension".
Try timestep_ds = document_embedding.get_shape().as_list()[1], this statement should give you a python integer.
Here are some relevant documentations for tf.split and tf.Tensor.get_shape