We Keep Coding

How to implement Multinomial conditional distributions depending on the conditional binary value in Tensorflow Probability?

I am trying to build a graphical model in Tensorflow Probability, where we first sample a number of positive (1) and negative (0) examples (count_i) from Categorical distribution and then construct Multinomial distribution (Y_i) depending on the value of (count_i). These events (Y_i) are mutually exclusive :
Y_1 ~ Multinomial([.9, 0.1, 0.05, 0.05, 0.1], total_count = [tf.reduce_sum(tf.cast(count==1, tf.float32))
Y_2 ~ Multinomial([0.99, 0.01, 0., 0., 0.], total_count = [tf.reduce_sum(tf.cast(count==0, tf.float32))
I have read these tutorials, however I am stuck with two issues:
This code generates two arrays of length 500, whereas I only need 1 array of 500. What should I change so we only get 1 sample from Categorical distribution and then depending on the overall count of the value we are conditioning on, Multinomial is constructed ?
The sample from Categorical distribution gives only values of 0, whereas it should be a blend between 0 and 1. What am I doing wrong here?
My code is as follows. You can run these to replicate the behaviour:
def simplied_model():
return tfd.JointDistributionSequential([
tfd.Uniform(low=0., high = 1., name = 'e'), #e
lambda e: tfd.Sample(tfd.Categorical(probs = tf.stack([e, 1.-e], 0)), sample_shape =500), #count #should it be independent?
lambda count: tfd.Multinomial(probs = tf.constant([[.9, 0.1, 0.05, 0.05, 0.1], [0.99, 0.01, 0., 0., 0.]]), total_count = tf.cast(tf.stack([tf.reduce_sum(tf.cast(count==1, tf.float32)),tf.reduce_sum(tf.cast(count==0, tf.float32))], 0), dtype= tf.float32))
])
tt = simplied_model()
tt.resolve_graph()
tt.sample(1)

The first array will be your Y_{1} and the second will be your Y_{2}. The key is that your output will always be of shape (2, 5) because that is the length of the probabilities you are passing to tfd.Multinomial.
Code:
import tensorflow as tf
import tensorflow_probability as tfp
from tensorflow_probability import distributions as tfd
# helper function
def _get_counts(vec):
zeros = tf.reduce_sum(tf.cast(vec == 0, tf.float32))
ones = tf.reduce_sum(tf.cast(vec == 1, tf.float32))
return tf.stack([ones, zeros], 0)
joint = tfd.JointDistributionSequential([
tfd.Sample( # sample from uniform to make it 2D
tfd.Uniform(0., 1., name="e"), 1),
lambda e: tfd.Sample(
tfd.Categorical(probs=tf.stack([e, 1.-e], -1)), 500),
lambda c: tfd.Multinomial(
probs=[
[0.9, 0.1, 0.05, 0.05, 0.1],
[0.99, 0.01, 0., 0., 0.],
],
total_count=_get_counts(c),
)
])
joint.sample(5) # or however many you want to sample
Output:
# [<tf.Tensor: shape=(5, 1), dtype=float32, numpy=
# array([[0.5611458 ],
# [0.48223293],
# [0.6097224 ],
# [0.94013655],
# [0.14861858]], dtype=float32)>,
# <tf.Tensor: shape=(5, 1, 500), dtype=int32, numpy=
# array([[[1, 0, 0, ..., 1, 0, 1]],
#
# [[1, 1, 1, ..., 1, 0, 0]],
#
# [[0, 0, 0, ..., 1, 0, 0]],
#
# [[0, 0, 0, ..., 0, 0, 0]],
#
# [[1, 0, 1, ..., 1, 0, 1]]], dtype=int32)>,
# <tf.Tensor: shape=(2, 5), dtype=float32, numpy=
# array([[ 968., 109., 0., 0., 0.],
# [1414., 9., 0., 0., 0.]], dtype=float32)>]

How to find adjacent lines on a regular 3D grid in python

I have the coordinate of a bunch of points and want to create surfaces out of them in a python package. I want to arrange my data before importing them into the package. Points are coming from a regular grid. Firstly, I am creating lines based on the location of points. In this step I just define which point numbers create my lines. My input data is:
coord = np.array(
[[0., 0., 2.], [0., 1., 3.], [0., 2., 2.], [1., 0., 1.], [1., 1., 3.],
[1., 2., 1.], [2., 0., 1.], [2., 1., 1.], [3., 0., 1.], [4., 0., 1.]])
The figure below shows the numbers of the grid points (gray) and the numbers of the lines (blue and red).
The lines are modeled through dictionaries, in which the key is the line number, and the value is a tuple with the start and end points numbers:
In [906]: blue_line
Out[906]: {1: (1, 2), 2: (2, 3), 3: (4, 5), 4: (5, 6), 5: (7, 8)}
In [907]: red_line
Out[907]:
{6: (1, 4),
7: (2, 5),
8: (3, 6),
9: (4, 7),
10: (5, 8),
11: (7, 9),
12: (9, 10)}
To learn more about how the lines are generated, check out this thread. The lines that are used to create the surfaces are stored in a list:
surfaces = [(1, 6, 3, 7), (2, 7, 4, 8), (3, 9, 5, 10)]
As the last step, I want to find the number of lines which are not used in creating the surfaces or are used but are closer than a limit the the dashed line in the figure above. Again, I have the coordinate of the two points creating that dashed line:
coord_dash = [(2., 2., 2.), (5., 0., 1.)]
adjacency_threshold = 2
I want to have these adjacent lines as another list (shown by a red arrow in the figure):
adjacent_lines = [4, 10, 5, 11, 12]
I have only this rough idea and do not know how to code it in Python. I can only create line numbers and surfaces and need help to find those close lines.

Determining what lines have not been used is straightforward (NumPy's setdiff1d comes in handy for this task):
In [924]: all_line = {**blue_line, **red_line}
In [925]: lines = list(all_line.keys())
In [926]: lines
Out[926]: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
In [927]: used_lines = np.ravel(surfaces)
In [928]: used_lines
Out[928]: array([ 1, 6, 3, 7, 2, 7, 4, 8, 3, 9, 5, 10])
In [929]: unused_lines = np.setdiff1d(lines, used_lines)
In [930]: unused_lines
Out[930]: array([11, 12])
The adjacent lines can be obtained by using NumPy's linalg.norm:
In [954]: midpoints
Out[954]:
{1: array([0. , 0.5, 2.5]),
2: array([0. , 1.5, 2.5]),
3: array([1. , 0.5, 2. ]),
4: array([1. , 1.5, 2. ]),
5: array([2. , 0.5, 1. ]),
6: array([0.5, 0. , 1.5]),
7: array([0.5, 1. , 3. ]),
8: array([0.5, 2. , 1.5]),
9: array([1.5, 0. , 1. ]),
10: array([1.5, 1. , 2. ]),
11: array([2.5, 0. , 1. ]),
12: array([3.5, 0. , 1. ])}
In [955]: mid_dash = np.array(coord_dash).mean(axis=0)
In [956]: mid_dash
Out[956]: array([3.5, 1. , 1.5])
In [957]: adjacent_lines = []
...: for idx, point in midpoints.items():
...: if np.linalg.norm(point - mid_dash) < adjacency_threshold:
...: adjacent_lines.append(idx)
In [958]: adjacent_lines
Out[958]: [5, 11, 12]

What is Tensorflow equivalent of pytorch's conv1d?

Just wondering how I can perform 1D convolution in tensorflow. Specifically, looking to replace this code to tensorflow:
inputs = F.pad(inputs, (kernel_size-1,0), 'constant', 0)
output = F.conv1d(inputs, weight, padding=0, groups=num_heads)

Tensorflow equivalent of PyTorch's
torch.nn.functional.conv1d() is
tf.nn.conv1d() and torch.nn.functional.pad() is tf.pad().
For Example:
(PyTorch code)
import torch.nn as nn
import torch
inputs = torch.tensor([1, 0, 2, 3, 0, 1, 1], dtype=torch.float32)
filters = torch.tensor([2, 1, 3], dtype=torch.float32)
inputs = inputs.unsqueeze(0).unsqueeze(0) # torch.Size([1, 1, 7])
filters = filters.unsqueeze(0).unsqueeze(0) # torch.Size([1, 1, 3])
conv_res = F.conv1d(inputs, filters, padding=0, groups=1) # torch.Size([1, 1, 5])
pad_res = F.pad(conv_res, (1, 1), mode='constant', value=0) # torch.Size([1, 1, 7])
output:
tensor([[[ 0., 8., 11., 7., 9., 4., 0.]]])
(Tensorflow code)
import tensorflow as tf
tf.enable_eager_execution()
i = tf.constant([1, 0, 2, 3, 0, 1, 1], dtype=tf.float32)
k = tf.constant([2, 1, 3], dtype=tf.float32, name='k')
data = tf.reshape(i, [1, int(i.shape[0]), 1], name='data')
kernel = tf.reshape(k, [int(k.shape[0]), 1, 1], name='kernel')
res = tf.nn.conv1d(data, kernel, 1, 'VALID')
res = tf.pad(res[0], [[1, 1], [0, 0]], "CONSTANT")
output:
<tf.Tensor: id=555, shape=(7, 1), dtype=float32, numpy=
array([[ 0.],
[ 8.],
[11.],
[ 7.],
[ 9.],
[ 4.],
[ 0.]], dtype=float32)>

Get an index of a sorted matrix

I have a 2D np.array:
array([[ 1523., 172., 1613.],
[ 3216., 117., 1999.],
[ 85., 1271., 4.]])
I would to extract the sorted indexes of this np.array by value.
The results should be (for example) :
[[2,2],[2,0],[1,1],[0,1],[2,1],[0,0],[0,2],[1,2],[1,0]]
I already saw how to extract the min :
np.unravel_index(np.argmin(act),act.shape) #(2,2)
Thank you

Using numpy.argsort with axis=None (assuming flatten array):
>>> import numpy as np
>>>
>>> act = np.array([[ 1523., 172., 1613.],
... [ 3216., 117., 1999.],
... [ 85., 1271., 4.]])
>>> n = act.shape[1]
>>> zip(*np.argsort(act, axis=None).__divmod__(n))
[(2, 2), (2, 0), (1, 1), (0, 1), (2, 1), (0, 0), (0, 2), (1, 2), (1, 0)]

Split NumPy array according to values in the array (a condition)

I have an array:
arr = [(1,1,1), (1,1,2), (1,1,3), (1,1,4)...(35,1,22),(35,1,23)]
I want to split my array according to the third value in each ordered pair. I want each third value of 1 to be the start
of a new array. The results should be:
[(1,1,1), (1,1,2),...(1,1,35)][(1,2,1), (1,2,2),...(1,2,46)]
and so on. I know numpy.split should do the trick but I'm lost as to how to write the condition for the split.

Here's a quick idea, working with a 1d array. It can be easily extended to work with your 2d array:
In [385]: x=np.arange(10)
In [386]: I=np.where(x%3==0)
In [387]: I
Out[387]: (array([0, 3, 6, 9]),)
In [389]: np.split(x,I[0])
Out[389]:
[array([], dtype=float64),
array([0, 1, 2]),
array([3, 4, 5]),
array([6, 7, 8]),
array([9])]
The key is to use where to find the indecies where you want split to act.
For a 2d arr
First make a sample 2d array, with something interesting in the 3rd column:
In [390]: arr=np.ones((10,3))
In [391]: arr[:,2]=np.arange(10)
In [392]: arr
Out[392]:
array([[ 1., 1., 0.],
[ 1., 1., 1.],
...
[ 1., 1., 9.]])
Then use the same where and boolean to find indexes to split on:
In [393]: I=np.where(arr[:,2]%3==0)
In [395]: np.split(arr,I[0])
Out[395]:
[array([], dtype=float64),
array([[ 1., 1., 0.],
[ 1., 1., 1.],
[ 1., 1., 2.]]),
array([[ 1., 1., 3.],
[ 1., 1., 4.],
[ 1., 1., 5.]]),
array([[ 1., 1., 6.],
[ 1., 1., 7.],
[ 1., 1., 8.]]),
array([[ 1., 1., 9.]])]

I cannot think of any numpy functions or tricks to do this . A simple solution using for loop would be -
In [48]: arr = [(1,1,1), (1,1,2), (1,1,3), (1,1,4),(1,2,1),(1,2,2),(1,2,3),(1,3,1),(1,3,2),(1,3,3),(1,3,4),(1,3,5)]
In [49]: result = []
In [50]: for i in arr:
....: if i[2] == 1:
....: tempres = []
....: result.append(tempres)
....: tempres.append(i)
....:
In [51]: result
Out[51]:
[[(1, 1, 1), (1, 1, 2), (1, 1, 3), (1, 1, 4)],
[(1, 2, 1), (1, 2, 2), (1, 2, 3)],
[(1, 3, 1), (1, 3, 2), (1, 3, 3), (1, 3, 4), (1, 3, 5)]]

From looking at the documentation it seems like specifying the index of where to split on will work best. For your specific example the following works if arr is already a 2dimensional numpy array:
np.split(arr, np.where(arr[:,2] == 1)[0])
arr[:,2] returns a list of the 3rd entry in each tuple. The colon says to take every row and the 2 says to take the 3rd column, which is the 3rd component.
We then use np.where to return all the places where the 3rd coordinate is a 1. We have to do np.where()[0] to get at the array of locations directly.
We then plug in the indices we've found where the 3rd coordinate is 1 to np.split which splits at the desired locations.
Note that because the first entry has a 1 in the 3rd coordinate it will split before the first entry. This gives us one extra "split" array which is empty.