We Keep Coding

Is there something wrong with my Python Hamming distance code?

I am trying to implement the Hamming distance in Python. Hamming distance is typically used to measure the distance between two codewords. The operation is simply performing exclusive OR. For example, if we have the codewords 10011101 and 10111110, then their exclusive OR would be 00100011, and the Hamming distance is said to be 1 + 1 + 1 = 3.
My code is as follows:
def hamming_distance(codeword1, codeword2):
"""Calculate the Hamming distance between two bit strings"""
assert len(codeword1) == len(codeword2)
x, y = int(codeword1, 2), int(codeword2, 2) # '2' specifies that we are reading a binary number
count, z = 0, x^y
while z:
count += 1
z &= z - 1
return count
def checking_codewords(codewords, received_data):
closestDistance = len(received_data) # set default/placeholder closest distance as the maximum possible distance.
closestCodeword = received_data # default/placeholder closest codeword
for i in codewords:
if(hamming_distance(i, received_data) < closestDistance):
closestCodeword = i
closestDistance = hamming_distance(i, received_data)
return closestCodeword
print(checking_codewords(['1010111101', '0101110101', '1110101110', '0000000110', '1100101001'], '0001000101'))
hamming_distance(codeword1, codeword2) takes the two input parameters codeword1 and codeword2 in the form of binary values and returns the Hamming distance between the two input codewords.
checking_codewords(codewords, received_data) should determine the correct codeword IFF there are any errors in received data (i.e., the output is the corrected codeword string). Although, as you can see, I haven't added the "IFF there are any errors in received data" part yet.
I just tested the checking_codewords function with a set of examples, and it seems to have worked correctly for all of them except one. When I use the set of codewords ['1010111101', '0101110101', '1110101110', '0000000110', '1100101001'] and the received data '0001000101' the output is 0101110101, which is apparently incorrect. Is there something wrong with my code, or is 0101110101 actually correct and there is something wrong with the example? Or was this just a case where there was no error in the received data, so my code missed it?

For my point of view, is not clear why your algorithm transforms the initial string into an integer to do a bitwise difference.
I mean, after the assert the equal length you can simply compute the difference using the zip function:
sum([c1!=c2 for c1,c2 in zip(codeword1,codeword2)])
For sum function, python consider True==1, False==0.
Doing a little simplification on your code:
def hamming_distance(codeword1, codeword2):
"""Calculate the Hamming distance between two bit strings"""
assert len(codeword1) == len(codeword2)
return sum([c1!=c2 for c1,c2 in zip(codeword1,codeword2)])
def checking_codewords(codewords, received_data):
min_dist, min_word = min([(hamming_distance(i, received_data), received_data) for i in codewords])
return min_word
print(checking_codewords(['1010111101', '0101110101', '1110101110', '0000000110', '1100101001'], '0001000101'))

How to count elements in an array withtin a given increasing interval?

I have an array of time values. I want to know how many values are in each 0.05 seconds window.
For example, some values of my array are: -1.9493, -1.9433, -1.911 , -1.8977, -1.8671,..
In the first interval of 0.050 seconds (from -1.9493 to -1.893) I´m expecting to have 3 elements
I already create another array with the 0.050 seconds steps.
a=max(array)
b=min(array)
ventanalinea1=np.arange(b,a,0.05)
v1=np.array(ventanalinea1)
In other words, I would like to compare my original array with this one.
I would like to know if there is a way to ask python to evaluate my array within a given dynamic range.

One of the variants:
import numpy as np
# original array
a = [-1.9493, -1.9433, -1.911 , -1.8977, -1.8671]
step = 0.05
bounds = np.arange(min(a), max(a) + step, step)
result = [
list(filter(lambda x: b[i] <= x <= b[i+1], a))
for i in range(len(b)-1)
]

I have found a cool python library python-intervals that simplify your problem a lot:
Install it with pip install python-intervals and try the code below.
import intervals as I
# This is a recursive function
def counter(timevalues, w=0.050):
if not timevalues:
return "" # stops recursion when timevalues is empty
# Make an interval object that provides convenient interval operations like 'contains'
window = I.closed(
timevalues[0], timevalues[0] + w)
interval = list(
filter(window.contains, timevalues))
count = len(interval)
timevalues = timevalues[count:]
print(f"[{interval[0]} : {interval[-1]}] : {count}")
return counter(timevalues)
if __name__ == "__main__":
times = [-1.9493, -1.9433, -1.911, -1.8977, -1.8671]
print(counter(times))
Adapt it as you wish, for example you might want to return a dictionary rather that a string.
You could still get around this without using the python-intervals library here but i have introduced it here because it will be very likely that you would need other complex operations along the way on your code.

Nested Anova in python with Spm1d. Can't print f statistics and p values

I'm looking for a simple solution to perform multi-factor ANOVA analysis in python. A 2-factor nested ANOVA is what I'm after, and the SPM1D python module is one way to do that, however I am having an issue.
http://www.spm1d.org/doc/Stats1D/anova.html#two-way-nested-anova
for any of the nested approach examples, there is never any F-statistic or p_values printed, nor can I find any way to print them or send them to a variable.
To go through the motions of running one of their examples, where B is nested inside A, with Y observations:
import numpy as np
from matplotlib import pyplot
import spm1d
dataset = spm1d.data.uv1d.anova2nested.SPM1D_ANOVA2NESTED_3x3()
Y,A,B = dataset.get_data()
#(1) Conduct ANOVA:
alpha = 0.05
FF = spm1d.stats.anova2nested(Y, A, B, equal_var=True)
FFi = FF.inference(0.05)
print( FFi )
#(2) Plot results:
pyplot.close('all')
FFi.plot(plot_threshold_label=True, plot_p_values=True)
pyplot.show()
The only indication of statistical significance provided is whether the h0 hypothesis is rejected or not.
> print( FFi )
SPM{F} inference list
design : ANOVA2nested
nEffects : 2
Effects:
A z=(1x101) array df=(2, 6) h0reject=True
B z=(1x101) array df=(6, 36) h0reject=False
In reality, that should be enough. However, in science, scientists like to think of something as more or less significant, which is actually kind of crap... significance is binary. But that's how they think about it, so I have to play along in order to get work published.
The example code produces a matplotlib plot, and this DOES have the f statistic and p_values on it!
#(2) Plot results:
pyplot.close('all')
FFi.plot(plot_threshold_label=True, plot_p_values=True)
pyplot.show()
But I can't seem to get any output which prints it.
FFi.get_p_values
and
FFi.get_f_values
produce the output:
<bound method SPMFiList.get_p_values <kabammi edit -- or get_f_values> of SPM{F} inference list
design : ANOVA2nested
nEffects : 2
Effects:
A z=(1x101) array df=(2, 6) h0reject=True
B z=(1x101) array df=(6, 36) h0reject=False
So I don't know what to do. Clearly the FFi.plot class can access the p_values (with plot_p_values) but FFi.get_p_values cant!!? Can anyone lend a hand?
cheers,
K

The easiest way to get the p values is to use the get_p_values method that you mention, you just need to call the method by adding () to the end.
p = FFi.get_p_values()
print(p)
This yields:
([0.016584151119287904], [])
To see more detailed information for each effect in 2+-way ANOVA, including p values, use print along with the individual F statistics like this:
print( FFi[0] )
print( FFi[1] )
The first print statement will produce output like this:
SPM{F} inference field
SPM.effect : Main A
SPM.z : (1x101) raw test stat field
SPM.df : (2, 6)
SPM.fwhm : 11.79254
SPM.resels : (1, 8.47993)
Inference:
SPM.alpha : 0.050
SPM.zstar : 24.30619
SPM.h0reject : True
SPM.p_set : 0.017
SPM.p_cluster : (0.017)
You can retrieve the clusters' p values like this:
p = [F.p for F in FFi]
which gives the same result as calling get_p_values.
Note that there are no p values in this case for FFi[1] because the test statistic fails to cross the alpha-defined threshold (see the "Main B" panel in the figure above). If you need to report p values in this case as well, one option is simply to use "p > alpha". More precise p value are available parametrically up until about p = 0.5, but larger p values than that are not very accurate using parametric methods, so if you need p values for all cases consider using the nonparametric version: spm1d.stats.nonparam.anova2nested.

'bounds' doesn't work in scipy.stats.minimize function

I had a very strange problem in the minimize function. I wrote following code and hoped to output result from (0,1).
cons = ({'type':'eq','fun':lambda x: 1-sum(x)})
bnds = [(0,1)]*len(w)
minS = minimize(min_S_function, w, method = 'SLSQP', bounds = bnds, constraints = cons)
However, the result output many extremely small numbers instead of zero even though I set bounds between 0 to 1. Why is that?
In [68]:minS.x
Out[68]:
array([ 2.18674802e-14, -2.31905438e-14, 4.05696128e-01,
1.61295198e-14, 4.98954818e-02, -2.75073615e-14,
3.97195447e-01, 1.09796187e-14, -4.33297358e-15,
2.38805100e-14, 7.73037793e-15, 3.21824430e-14,
-1.42202909e-14, -1.08110329e-14, -1.83513297e-14,
-1.37745269e-14, 3.37854385e-14, 4.69473932e-14,
-1.09088800e-15, -1.57169147e-14, 7.47784562e-02,
1.32782180e-02, 1.64441640e-14, 2.72140153e-15,
5.23069695e-14, 5.91562687e-02, 2.16467506e-15,
-6.50672519e-15, 2.53337977e-15, -6.68019297e-14])

This is an acceptable solution!
Those iterative solvers only guarantee a local-optimum approximate-solution and that's what you got!
Check your numbers and you will see, that the distance to the lower-bound (of your negative out-of-bound values) is within 10^-14 = 0,00000000000001, an acceptable error (we use floating-point math after all).

getting more precise answer in python

I am trying to implement a simple formula with python . In the case of "RLSR" there two approach to calculate the coefficient "C" by C=k^-1*y where k is the kernel matrix and y is the target value vector.
The second way is by using eigenvalues(w) and eigenvectors(v) of the kernel method and using formula :
Sum(u(i)*<u(i),y> / w(i) ).
to check the answer of the result I'm checking whether y=kc or not! in the first case it's ok , but the latter case it's not.but the problem is my c vectors in both method look the same.
output from invers algorithm :
[ 19.49840251 18.82695226 20.08390355 15.01043404 14.79353281
16.75316736 12.88504257 16.92127176 16.77292954 17.81827473
20.90503787 17.09359467 18.76366701 18.14816903 20.03491117
22.56668264 21.45176136 25.44051036 30.40312692 22.61466379
22.86480382 19.34631818 17.0169598 19.85244414 16.63702471
20.35280156 20.58093488 22.42058736 20.54935198 19.35541575
20.39006958 19.74766081 20.41781019 22.33858797 17.57962283
22.61915219 22.54823733 24.96292824 22.82888425 34.18952603
20.7487537 24.82019935 22.40621769 21.15767304 27.58919263
18.39293156 21.55455108 18.69532341]
output from second(eigendecomposition ) algorithm :
[ 19.25280289 18.73927731 19.77184991 14.7650427 14.87364331
16.2273648 12.29183797 16.52024239 16.66669961 17.59282615
20.83059115 17.02815857 18.3635798 18.16931845 20.50528549
22.67690164 21.40479524 25.54544 30.94618128 22.72992565
23.06289609 17.6485592 15.2758427 19.50578691 16.45571607
20.19960765 20.35352859 22.60091638 20.36586912 18.90760728
20.57141151 19.43677153 20.43437031 22.39310576 17.72296978
22.82139991 22.50744791 25.10496617 22.30462867 34.80540213
20.77064617 25.18071618 22.5500315 20.71481252 27.91939784
18.29868659 22.00800019 18.71266093]
this how I have implemented it:
so let say if we have 48 samples the size of K is 48x48 , y is 48x1
def cpu_compute(y,k):
w, u=LA.eigh(k)
uDoty=np.dot(u,y)
temp=np.transpose(np.tile(w,(len(u),1)))
div=np.divide(u,temp)
r=np.tile(uDoty,(len(div),1))
a=div*r.T
c=sum(a)
return c
and the result from
print np.allclose(Y,np.dot(K,c))
is false.
also the norm of difference from the true result is 3.13014997999.
now I have no idea how can I fix this . I thought maybe some how by doing more precise answer.
Appreciate any help!

To solve k c = y with numpy, use numpy.linalg.solve:
c = solve(k, y)
This uses an algorithm that is much more robust than the methods you are trying.