Scipy optimize.curve_fit sometimes won't converge - python

I'm trying to use numpy.optimize.curve_fit to estimate the frequency and phase of an on/off sequence.
This is the code I'm using:
from numpy import *
from scipy import optimize
row = array([0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 0.0, 0.0,])
def fit_func(x, a, b, c, d):
return c * sin (a * x + b) + d
p0 = [(pi/10.0), 5.0, row.std(), row.mean()]
result = optimize.curve_fit(fit_func, arange(len(row)), row, p0)
print result
This works. But on some rows, even though they seem perfectly ok, it fails.
Example of failing row:
row = array([1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,])
The error is:
RuntimeError: Optimal parameters not found: Both actual and predicted relative reductions in the sum of squares are at most 0.000000 and the relative error between two consecutive iterates is at most 0.000000
Which tells me very little about what's happened.
A quick test shows that varying the parameters in p0 will cause that row to succeed... and others to fail. Why is that?


I tried both rows of data that you provided and both worked for me just fine. I'm using Scipy 0.8.0rc3. What version are you using? Another thing that might help is to set c and d to fixed values since they really should be the same every time. I set c to 0.6311786 and d to .5. You could also use an fft with zero padding and quadratic fitting around the peak to find the frequency if you want another method. Really, any pitch estimation method is applicable since you are looking for the fundamental frequency.

Related

Python matplotlib histogram does not plot all bins

When I put the following data in a histogram, the result is not plotted correctly. For example, there is a wheight equal to 5 (near the end) but it is not plotted.
How can I solve this?
import matplotlib.pyplot as plt
a=[]
for i in range(1000):
a.append(i*0.001)
b=[2.0, 1.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 5.0, 0.0, 1.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 1.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 2.0, 0.0, 1.0, 0.0, 1.0, 0.0, 2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 2.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 2.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 2.0, 0.0, 1.0, 0.0, 0.0, 0.0, 2.0, 0.0, 2.0, 1.0, 0.0, 1.0, 1.0, 2.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 2.0, 1.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 0.0, 1.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 3.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 1.0, 1.0, 0.0, 1.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 3.0, 0.0, 1.0, 1.0, 1.0, 0.0, 1.0, 0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 2.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 2.0, 2.0, 0.0, 0.0, 0.0, 1.0, 3.0, 0.0, 0.0, 1.0, 0.0, 3.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 3.0, 0.0, 3.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 1.0, 0.0, 2.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 0.0, 2.0, 0.0, 1.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 3.0, 0.0, 2.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 2.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 2.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 3.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 2.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 2.0, 0.0, 1.0, 2.0, 2.0, 0.0, 1.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 2.0, 1.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 2.0, 1.0, 0.0, 2.0, 0.0, 2.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 2.0, 3.0, 2.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 3.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 2.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 2.0, 0.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 3.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 2.0, 0.0, 1.0, 1.0, 2.0, 0.0, 0.0, 1.0, 1.0, 0.0, 1.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 2.0, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 4.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 2.0, 0.0, 2.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 2.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 3.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 2.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 2.0, 0.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 1.0, 2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.0, 0.0, 0.0, 0.0, 2.0, 1.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 2.0, 1.0, 0.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 2.0, 0.0, 1.0, 1.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 2.0, 3.0, 1.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 3.0, 0.0, 0.0, 1.0, 2.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 0.0, 1.0, 2.0, 1.0, 1.0, 2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 1.0, 0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 2.0, 0.0, 2.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 2.0, 0.0, 2.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 1.0, 1.0, 2.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 2.0, 1.0, 2.0, 0.0, 0.0, 2.0, 1.0, 0.0, 0.0, 3.0, 0.0, 0.0, 0.0, 0.0, 3.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 2.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 2.0, 1.0, 0.0, 1.0, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 1.0, 2.0, 1.0, 1.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 3.0, 0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 3.0, 0.0, 0.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 2.0, 0.0, 5.0, 0.0, 1.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0]
plt.hist(a,1000, weights=b)
plt.show()

Check out the documentation of the matplotlib.pyplot.hist function here.
The first parameter, in your case a, specifies the values you plot.
The weights parameter says how much every value from a contributes to the accumulated weight.
Not sure why you are using a and b the way you do, but if you run this:
plt.hist(b,1000)
You will see the value 5.
You can also use the third parameter - the range of the histogram.
Use (0, 5) to show values in this range now matter what you have in a:
plt.hist(a, 1000, (0,5), weights=b)

'ValueError: setting an array element with a sequence' while using scipy.spacial.distance function cdist

I am trying to compute the distance between vectors in two pandas dataframes using cdist from scipy.spatial.distance, but the output is all wrong and I can't pinpoint where is fails. This is actually a follow up on the question Problems computing cdist of two columns in two different dataframes
So, my original dataframes are of the type:
df_sample =
Fingerprint
1272 [0.0, 4.0, 8.0, 15.0, 10.0, 8.0, 2.54, 2.0, 4.91, 0.0, 0.0, 0.0, 0.0, 3.59, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 2.0, 8.0]
657 [1.44, 12.0, 10.0, 5.0, 6.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 2.0, 8.23, 4.36, 15.0]
806 [4.58, 13.09, 15.46, 3.59, 3.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 2.0, 0.0, 6.31]
and
DF =
barcode \
4538 A4060462000516278
5043 A4050494272716275
11663 A4070271111316245
2701 A4060462848716270
825 A4060454573516274
8679 A4060462010016274
11700 A4060462080916270
8594 A4060461067716272
8707 A4060454363916275
1071 A4060463723916275
Geopos Ack
4538 [0.0, 0.0, 0.0, 0.0, 6.0, 15.0, 16.0, 0.0, 0.0, 5.0, 0.0, 15.0, 0.0, 0.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.5, 0.0, 3.0]
5043 [0.0, 0.0, 0.0, 0.0, 0.0, 16.0, 12.0, 0.0, 13.0, 15.0, 0.0, 15.0, 0.0, 0.0, 6.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.0, 3.0, 0.0]
11663 [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 5.0, 15.0, 0.0, 0.0, 0.0, 6.0, 2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
2701 [0.0, 0.0, 0.0, 8.0, 13.0, 16.0, 6.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 6.0, 0.0, 7.0]
825 [0.0, 0.0, 0.0, 0.0, 0.0, 11.0, 15.0, 0.0, 13.0, 16.0, 0.0, 9.0, 3.0, 0.0, 6.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
8679 [0.0, 4.0, 9.0, 15.0, 10.0, 3.0, 2.0, 0.0, 2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 2.0, 9.0]
11700 [0.0, 0.0, 6.0, 0.0, 15.0, 8.0, 2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 16.0, 0.0, 6.0]
8594 [12.0, 16.0, 16.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 8.0, 0.0, 5.0]
8707 [7.0, 5.0, 2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 2.0, 8.0, 15.0]
1071 [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 12.0, 15.5, 6.0, 3.5, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
(I provide dictionaries for both at the end of the question).
As you can see, they are of different dimension (although the vectors belong to the same space). So, to remedy this I create zero vectors in df_sample by doing this:
Number_AP = 26
number_zero_vectors = len(DF)-len(df_sample)
df =pd.DataFrame(columns = ['Fingerprint'])
for k in range(number_zero_vectors):
a = zerolistmaker(Number_AP)
df = df.append({'Fingerprint':a},ignore_index=True)
df_sample_ = pd.concat([df_sample, df])
Hence, DF and df_sample_ have the same shape. My first approach (described in detail in the question mentioned above) was to deal with both columns and all their rows simulataneously, but it did not work. So, my approach now is to deal with both column incrementally (e.g. take the first row of df_sample_ and compute the distance to every single vectors in DF, all this in a loop:
distances = []
for i in range(len(df_sample_)):
a = df_sample_['Fingerprint'][i:i+1]
print(a)
for j in range(len(DF)):
b = DF['Geopos Ack'][j:j+1]
print(b)
ax = a.to_numpy()
bx = b.to_numpy()
aa = ax.reshape(-1,1)
bb = bx.reshape(-1,1)
print(aa.shape)
print(bb.shape)
d = sp.cdist(aa,bb,'euclidean')
print(d)
### I leave out the part were distances.append(d) since what come before it fails
This returns:
1272 [0.0, 4.0, 8.0, 15.0, 10.0, 8.0, 2.54, 2.0, 4.91, 0.0, 0.0, 0.0, 0.0, 3.59, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 2.0, 8.0]
Name: Fingerprint, dtype: object
4538 [0.0, 0.0, 0.0, 0.0, 6.0, 15.0, 16.0, 0.0, 0.0, 5.0, 0.0, 15.0, 0.0, 0.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 3.5, 0.0, 3.0]
Name: Geopos Ack, dtype: object
(1, 1)
(1, 1)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
TypeError: only size-1 arrays can be converted to Python scalars
The above exception was the direct cause of the following exception:
ValueError Traceback (most recent call last)
<ipython-input-368-e29e9515d73a> in <module>
14 print(bb.shape)
15
---> 16 d = sp.cdist(aa,bb,'euclidean')
17 print(d)
18
~\Anaconda3\envs\conda-qgis\lib\site-packages\scipy\spatial\distance.py in cdist(XA, XB, metric, *args, **kwargs)
2802 metric_name = _METRIC_ALIAS.get(mstr, None)
2803 if metric_name is not None:
-> 2804 XA, XB, typ, kwargs = _validate_cdist_input(XA, XB, mA, mB, n,
2805 metric_name, **kwargs)
2806
~\Anaconda3\envs\conda-qgis\lib\site-packages\scipy\spatial\distance.py in _validate_cdist_input(XA, XB, mA, mB, n, metric_name, **kwargs)
245 typ = types[types.index(XA.dtype)] if XA.dtype in types else types[0]
246 # validate data
--> 247 XA = _convert_to_type(XA, out_type=typ)
248 XB = _convert_to_type(XB, out_type=typ)
249
~\Anaconda3\envs\conda-qgis\lib\site-packages\scipy\spatial\distance.py in _convert_to_type(X, out_type)
182
183 def _convert_to_type(X, out_type):
--> 184 return np.ascontiguousarray(X, dtype=out_type)
185
186
~\Anaconda3\envs\conda-qgis\lib\site-packages\numpy\core\_asarray.py in ascontiguousarray(a, dtype)
175
176 """
--> 177 return array(a, dtype, copy=False, order='C', ndmin=1)
178
179
ValueError: setting an array element with a sequence.
Where did I go wrong? All arrays consist of floats and the shapes look right.
I know another approach would be to use pairwise_distance from sklearn but I did not manage to apply it to my dataframes.
Any help would be appreciated.
Data:
df_sample =
{'Fingerprint': {1272: [0.0,
4.0,
8.0,
15.0,
10.0,
8.0,
2.54,
2.0,
4.91,
0.0,
0.0,
0.0,
0.0,
3.59,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
2.0,
8.0],
657: [1.44,
12.0,
10.0,
5.0,
6.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
2.0,
8.23,
4.36,
15.0],
806: [4.58,
13.09,
15.46,
3.59,
3.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
2.0,
0.0,
6.31]}}
and
DF =
{'barcode': {4538: 'A4060462000516278',
5043: 'A4050494272716275',
11663: 'A4070271111316245',
2701: 'A4060462848716270',
825: 'A4060454573516274',
8679: 'A4060462010016274',
11700: 'A4060462080916270',
8594: 'A4060461067716272',
8707: 'A4060454363916275',
1071: 'A4060463723916275'},
'Geopos Ack': {4538: [0.0,
0.0,
0.0,
0.0,
6.0,
15.0,
16.0,
0.0,
0.0,
5.0,
0.0,
15.0,
0.0,
0.0,
0.0,
0.0,
2.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
3.5,
0.0,
3.0],
5043: [0.0,
0.0,
0.0,
0.0,
0.0,
16.0,
12.0,
0.0,
13.0,
15.0,
0.0,
15.0,
0.0,
0.0,
6.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
3.0,
3.0,
0.0],
11663: [0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
5.0,
15.0,
0.0,
0.0,
0.0,
6.0,
2.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0],
2701: [0.0,
0.0,
0.0,
8.0,
13.0,
16.0,
6.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
6.0,
0.0,
7.0],
825: [0.0,
0.0,
0.0,
0.0,
0.0,
11.0,
15.0,
0.0,
13.0,
16.0,
0.0,
9.0,
3.0,
0.0,
6.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0],
8679: [0.0,
4.0,
9.0,
15.0,
10.0,
3.0,
2.0,
0.0,
2.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
2.0,
9.0],
11700: [0.0,
0.0,
6.0,
0.0,
15.0,
8.0,
2.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
16.0,
0.0,
6.0],
8594: [12.0,
16.0,
16.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
8.0,
0.0,
5.0],
8707: [7.0,
5.0,
2.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
2.0,
8.0,
15.0],
1071: [0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
12.0,
15.5,
6.0,
3.5,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0]}}

append list with for loop and get multiple values in each cell in python

I'm trying to run 3 optimization with for loop and store the results in one dataframe.
After each optimization (element of the for loop), I append lists of results and being able to get all the reults in one list. However, when I try to convert the list to dataframe, I get one row for each of the optimization and multiple values in each cell corresponding to the variable name and the optimization number like this:
Date = []
results = []
for idx, df in enumerate([df0,df1,df2]):
model = ConcreteModel()
model.T = Set(initialize=df.hour.tolist(), ordered=True)
...
# Solve model
solver = SolverFactory('glpk')
solver.solve(model)
Date = list(df['Date'])
results.append([Date, model.Ein.get_values().values(), model.Eout.get_values().values(),
model.Z.get_values().values(), model.NES.get_values().values(),
model.L.get_values().values()])
df_results = pd.DataFrame(results)
df_results.rename(columns = {0: 'Date', 1: 'Ein', 2:'Eout', 3:'Z', 4:'NES', 5:'L'}, inplace = True)
df_results
## The output of the df is:
Date Ein
0 [2019-01-01, 2019-01-01, 2019-01-01, 2019-01-0... (0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 0.0, ... (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... (0.0, 0.0, 1.0, 2.0, 3.0, 3.0, 4.0, 5.0, 5.0, ... (0.0, 0.0, -100.0, -100.0, -100.0, 0.0, -100.0... (16231.0, 16051.0, 15806.0, 15581.0, 15610.0, ...
1 [2019-01-16, 2019-01-16, 2019-01-16, 2019-01-1... (0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, ... (0.0, 0.0, 0.0, 0.0, 0.0, 0.5, 0.0, 0.0, 0.0, ... (0.0, 1.0, 1.0, 1.0, 1.0, 0.5, 1.5, 2.5, 3.5, ... (0.0, -100.0, 0.0, 0.0, 0.0, 50.0, -100.0, -10... (17643.0, 18654.0, 20462.0, 20448.0, 20305.0, ...
2 [2019-01-31, 2019-01-31, 2019-01-31, 2019-01-3... (0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, ... (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, ... (0.0, 0.0, 1.0, 1.0, 2.0, 3.0, 4.0, 3.0, 3.0, ... (0.0, 0.0, -100.0, 0.0, -100.0, -100.0, -100.0... (22155.0, 22184.0, 21510.0, 21193.0, 20884.0, ...
#The output of the list named results is:
[[['2019-01-01',
'2019-01-01',
'2019-01-01',
...
'2019-01-15',
'2019-01-15',
'2019-01-15',
'2019-01-15',
'2019-01-15',
'2019-01-15',
'2019-01-16',
'2019-01-16',
'2019-01-16',
'2019-01-16',
'2019-01-16'],
dict_values([0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,
...
-1.11022302462516e-16, 0.0, 1.0, 0.0, 0.0, 1.0, 1.0, 1.11022302462516e-16, 0.0, 1.0, 0.5, 0.0, 0.0, 0.0, 0.166666666666667, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.333333333333333, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.666666666666667, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]),
dict_values([0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.5, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.5, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.5, 0.25, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.25,
...
0.333333333333333, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.5, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.5, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.5, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.166666666666667, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.833333333333333, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.166666666666667, 0.0, 0.0, 0.0, 0.666666666666667, 0.333333333333333, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.5, 0.25, 0.125, 0.0625, 0.03125, 0.015625, 0.0078125, 0.00390625, 0.001953125, 0.0009765625, 0.00048828125, 0.0]),
dict_values([0.0, 0.0, 1.0, 2.0, 3.0, 3.0, 4.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 5.0, 4.0, 3.0, 2.0, 1.0, 1.0, 1.0, 1.0, 0.5, 0.5, 0.5, 1.5, 2.5, 3.5, 4.5, 4.5, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 3.0, 3.0, 2.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 2.0, 3.0, 4.0, 5.0, 5.0, 5.0, 4.0, 3.0, 3.0, 3.0,
...
0.142857142857143, 0.142857142857143, 1.0, 2.0, 3.0, 4.0, 4.0, 4.0, 4.0, 4.0, 4.0, 3.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 2.0, 3.0, 4.0, 5.0, 5.0, 5.0, 4.0, 3.0, 3.0, 3.0, 2.0, 1.0, 1.0, 1.0, 1.0, 1.33333333333333, 1.33333333333333, 0.666666666666667, 0.666666666666667, 0.5, 0.25, 0.125, 0.0625, 0.03125, 0.015625, 0.0078125, 0.00390625, 0.001953125, 0.0009765625, 0.00048828125, 0.00048828125]),
[['2019-01-16',
'2019-01-16',
'2019-01-16',
'2019-01-16',
'2019-01-16',
'2019-01-16',
'2019-01-16',
'2019-01-16',
'2019-01-16',
'2019-01-16',
'2019-01-16',
'2019-01-16',
'2019-01-16',
'2019-01-16',
'2019-01-16',
...
Is it because each result in the for loop has de different dictionary? How could my results in this form:
Date Ein Eout Z NES L
0 2019-01-01 1.0 0.0 1.0 -100.0 16231.0
1 2019-01-01 1.0 1.0 0.0 100.0 16051.0,
...

You're constantly appending to results creating a list of lists of the wrong dimension. I hope this solution works for you -
df_results = pd.DataFrame(zip(Date, model.Ein.get_values().values(), model.Eout.get_values().values(),
model.Z.get_values().values(), model.NES.get_values().values(),
model.L.get_values().values()))
Let me know if it doesn't.

Plotting a histogram using a range of values and their frequency as a dictionary

Assume that I have the following dictionary:
scenario_summary = {'Day1': {'22459-22585': 0.0, '22585-22711': 0.0, '22711-22837': 0.0, '22837-22963': 0.0, '22963-23089': 0.0, '23089-23215': 0.0, '23215-23341': 0.0, '23341-23467': 0.0, '23467-23593': 0.0, '23593-23719': 0.0, '23719-23845': 0.0, '23845-23971': 0.0, '23971-24097': 0.0, '24097-24223': 0.0, '24223-24349': 0.0, '24349-24475': 0.0, '24475-24601': 0.0, '24601-24727': 0.0, '24727-24853': 0.0, '24853-24979': 0.0, '24979-25105': 0.0, '25105-25231': 0.0, '25231-25357': 0.0, '25357-25483': 0.0, '25483-25609': 0.0, '25609-25735': 0.0, '25735-25861': 0.0, '25861-25987': 0.0, '25987-26113': 1.0, '26113-26239': 1.0, '26239-26365': 0.0, '26365-26491': 2.0, '26491-26617': 5.0, '26617-26743': 5.0, '26743-26869': 5.0, '26869-26995': 12.0, '26995-27121': 19.0, '27121-27247': 7.000000000000001, '27247-27373': 11.0, '27373-27499': 15.0, '27499-27625': 7.000000000000001, '27625-27751': 4.0, '27751-27877': 4.0, '27877-28003': 2.0, '28003-28129': 0.0, '28129-28255': 0.0, '28255-28381': 0.0, '28381-28507': 0.0, '28507-28633': 0.0, '28633-28759': 0.0, '28759-28885': 0.0, '28885-29011': 0.0, '29011-29137': 0.0, '29137-29263': 0.0, '29263-29389': 0.0, '29389-29515': 0.0, '29515-29641': 0.0, '29641-29767': 0.0, '29767-29893': 0.0, '29893-30019': 0.0, '30019-30145': 0.0, '30145-30271': 0.0, '30271-30397': 0.0, '30397-30523': 0.0, '30523-30649': 0.0, '30649-30775': 0.0, '30775-30901': 0.0, '30901-31027': 0.0, '31027-31153': 0.0, '31153-31279': 0.0, '31279-31405': 0.0, '31405-31531': 0.0, '31531-31657': 0.0, '31657-31783': 0.0, '31783-31909': 0.0, '31909-32035': 0.0, '32035-32161': 0.0, '32161-32287': 0.0, '32287-32413': 0.0, '32413-32539': 0.0, '32539-32665': 0.0, '32665-32791': 0.0, '32791-32917': 0.0, '32917-33043': 0.0, '33043-33169': 0.0, '33169-33295': 0.0, '33295-33421': 0.0, '33421-33547': 0.0, '33547-33673': 0.0, '33673-33799': 0.0, '33799-33925': 0.0, '33925-34051': 0.0, '34051-34177': 0.0, '34177-34303': 0.0, '34303-34429': 0.0, '34429-34555': 0.0, '34555-34681': 0.0, '34681-34807': 0.0}}
As you can see, the dictionary consists of a range of values in string and their frequency. I would like to plot this as a histogram, but I don't know how I would be able to transform the string into a form that pandas or plotly would understand. What would your approach be? Or is there an easier way to do it, instead of hardcoding things? Or, would another module be easier option in doing so?
Thanks!

Since the bins (ranges) are already defined and their counts are already aggregated at an initial level, maybe it can help if you build something that overlays a histogram (distribution) on the top of the existing bin ranges:
import matplotlib
%matplotlib inline
def plot_hist(bins,input_dict):
df1 = pd.DataFrame(input_dict).reset_index()
df1['min'] = df1['index'].apply(lambda x:x.split('-')[0]).astype(int)
df1['max'] = df1['index'].apply(lambda x:x.split('-')[1]).astype(int)
df1['group'] = pd.cut(df1['max'],bins,labels=False)
df2 = df1.groupby('group' [['Day1','min','max']].agg({'min':'min','max':'max','Day1':'sum'}).reset_index()
df2['range_new'] = df2['min'].astype(str) + str('-') + df2['max'].astype(str)
df2.plot(x='range_new',y='Day1',kind='bar')
...and call the function by choosing bins lesser than the length of the dictionary - or the first level of 98 bins that are already there, like, say if you want a distribution of 20 groups aggregate:
plot_hist(20,scenario_summary)
Result Image :
hope it helps...

A histogram is basically a simple bar chart, where each bar represents a bin (usually in the form of a range) and a frequency of the elements that fall into that bin.
This is exactly the data that you already have. So instead of computing values for a histogram (as it would be done with plt.hist), you can simply pass your data to plt.bar, as it is. The result would then be this:
The code with your data, as a MCVE :
import matplotlib.pyplot as plt
scenario_summary = { 'Day1': {
'22459-22585': 0.0, '22585-22711': 0.0, '22711-22837': 0.0,
'22837-22963': 0.0, '22963-23089': 0.0, '23089-23215': 0.0,
'23215-23341': 0.0, '23341-23467': 0.0, '23467-23593': 0.0,
'23593-23719': 0.0, '23719-23845': 0.0, '23845-23971': 0.0,
'23971-24097': 0.0, '24097-24223': 0.0, '24223-24349': 0.0,
'24349-24475': 0.0, '24475-24601': 0.0, '24601-24727': 0.0,
'24727-24853': 0.0, '24853-24979': 0.0, '24979-25105': 0.0,
'25105-25231': 0.0, '25231-25357': 0.0, '25357-25483': 0.0,
'25483-25609': 0.0, '25609-25735': 0.0, '25735-25861': 0.0,
'25861-25987': 0.0, '25987-26113': 1.0, '26113-26239': 1.0,
'26239-26365': 0.0, '26365-26491': 2.0, '26491-26617': 5.0,
'26617-26743': 5.0, '26743-26869': 5.0, '26869-26995': 12.0,
'26995-27121': 19.0, '27121-27247': 7.0, '27247-27373': 11.0,
'27373-27499': 15.0, '27499-27625': 7.0, '27625-27751': 4.0,
'27751-27877': 4.0, '27877-28003': 2.0, '28003-28129': 0.0,
'28129-28255': 0.0, '28255-28381': 0.0, '28381-28507': 0.0,
'28507-28633': 0.0, '28633-28759': 0.0, '28759-28885': 0.0,
'28885-29011': 0.0, '29011-29137': 0.0, '29137-29263': 0.0,
'29263-29389': 0.0, '29389-29515': 0.0, '29515-29641': 0.0,
'29641-29767': 0.0, '29767-29893': 0.0, '29893-30019': 0.0,
'30019-30145': 0.0, '30145-30271': 0.0, '30271-30397': 0.0,
'30397-30523': 0.0, '30523-30649': 0.0, '30649-30775': 0.0,
'30775-30901': 0.0, '30901-31027': 0.0, '31027-31153': 0.0,
'31153-31279': 0.0, '31279-31405': 0.0, '31405-31531': 0.0,
'31531-31657': 0.0, '31657-31783': 0.0, '31783-31909': 0.0,
'31909-32035': 0.0, '32035-32161': 0.0, '32161-32287': 0.0,
'32287-32413': 0.0, '32413-32539': 0.0, '32539-32665': 0.0,
'32665-32791': 0.0, '32791-32917': 0.0, '32917-33043': 0.0,
'33043-33169': 0.0, '33169-33295': 0.0, '33295-33421': 0.0,
'33421-33547': 0.0, '33547-33673': 0.0, '33673-33799': 0.0,
'33799-33925': 0.0, '33925-34051': 0.0, '34051-34177': 0.0,
'34177-34303': 0.0, '34303-34429': 0.0, '34429-34555': 0.0,
'34555-34681': 0.0, '34681-34807': 0.0}}
data = scenario_summary['Day1']
x = range(len(data))
y = list(data.values())
plt.figure(figsize=(16, 9))
plt.bar(x, y)
plt.subplots_adjust(bottom=0.2)
plt.xticks(x, data.keys(), rotation='vertical')
plt.show()

You can use pandas module to convert dictionary data into data frame:
import pandas as pd
import matplotlib.pyplot as plt
scenario_summary = {'Day1': {'22459-22585': 0.0, '22585-22711': 0.0, '22711-22837': 0.0,
'22837-22963': 0.0, '22963-23089': 0.0, '23089-23215': 0.0,
'23215-23341': 0.0, '23341-23467': 0.0, '23467-23593': 0.0,
'23593-23719': 0.0, '23719-23845': 0.0, '23845-23971': 0.0,
'23971-24097': 0.0, '24097-24223': 0.0, '24223-24349': 0.0,
'24349-24475': 0.0, '24475-24601': 0.0, '24601-24727': 0.0,
'24727-24853': 0.0, '24853-24979': 0.0, '24979-25105': 0.0,
'25105-25231': 0.0, '25231-25357': 0.0, '25357-25483': 0.0,
'25483-25609': 0.0, '25609-25735': 0.0, '25735-25861': 0.0,
'25861-25987': 0.0, '25987-26113': 1.0, '26113-26239': 1.0,
'26239-26365': 0.0, '26365-26491': 2.0, '26491-26617': 5.0,
'26617-26743': 5.0, '26743-26869': 5.0, '26869-26995': 12.0,
'26995-27121': 19.0, '27121-27247': 7.000000000000001, '27247-27373': 11.0,
'27373-27499': 15.0, '27499-27625': 7.000000000000001, '27625-27751': 4.0,
'27751-27877': 4.0, '27877-28003': 2.0, '28003-28129': 0.0,
'28129-28255': 0.0, '28255-28381': 0.0, '28381-28507': 0.0,
'28507-28633': 0.0, '28633-28759': 0.0, '28759-28885': 0.0,
'28885-29011': 0.0, '29011-29137': 0.0, '29137-29263': 0.0,
'29263-29389': 0.0, '29389-29515': 0.0, '29515-29641': 0.0,
'29641-29767': 0.0, '29767-29893': 0.0, '29893-30019': 0.0,
'30019-30145': 0.0, '30145-30271': 0.0, '30271-30397': 0.0,
'30397-30523': 0.0, '30523-30649': 0.0, '30649-30775': 0.0,
'30775-30901': 0.0, '30901-31027': 0.0, '31027-31153': 0.0,
'31153-31279': 0.0, '31279-31405': 0.0, '31405-31531': 0.0,
'31531-31657': 0.0, '31657-31783': 0.0, '31783-31909': 0.0,
'31909-32035': 0.0, '32035-32161': 0.0, '32161-32287': 0.0,
'32287-32413': 0.0, '32413-32539': 0.0, '32539-32665': 0.0,
'32665-32791': 0.0, '32791-32917': 0.0, '32917-33043': 0.0,
'33043-33169': 0.0, '33169-33295': 0.0, '33295-33421': 0.0,
'33421-33547': 0.0, '33547-33673': 0.0, '33673-33799': 0.0,
'33799-33925': 0.0, '33925-34051': 0.0, '34051-34177': 0.0,
'34177-34303': 0.0, '34303-34429': 0.0, '34429-34555': 0.0,
'34555-34681': 0.0, '34681-34807': 0.0}}
# convert to data frame
data_frame = pd.DataFrame.from_dict(scenario_summary)
# plot data
plt.hist(data_frame['Day1'], density=1, bins=20)
plt.show()

Error with matplotlib. Don't display plot

i'm trying to display a plot with Matplotlib, using a array data from a .txt file, but when the figure is showed, don't have a plot, and the label is repeated with the number of positions of the array. What's happening?
The intro datafile is like this:
0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 2.0, 25.0, 25.0, 25.0, 25.0, 25.0, 25.0, 25.0, 25.0, 25.0, 25.0
Then show me this plot:
Plot from array
And that's the code:
import matplotlib.pyplot as plt
import codecs
converted = []
reward = open('reward_5_clusters','r')
acum = reward.readlines()
for line in acum:
if line.startswith(codecs.BOM_UTF8):
line = line[len(codecs.BOM_UTF8):]
x = line.split(', ')
converted.append(x)
plt.plot(converted, label='5 clusters')
plt.ylabel('Reward')
plt.xlabel('Time')
plt.title('Cumulative Reward')
plt.grid(True)
plt.legend(loc=0)
plt.show(block=False)
plt.savefig('cumulative_reward.png')
How to fix this?

Change converted.append to converted.extend. You are passing a nested list to plt.plot, when you want to pass a single series.

Categories