Related
I have this dataframe with euclidean distances:
import pandas as pd
df = pd.DataFrame({
'O1': [0.0, 1.7, 1.4, 0.4, 2.2, 3.7, 5.2, 0.2, 4.3, 6.8, 6.0],
'O2': [1.7, 0.0, 1.0, 2.0, 1.3, 2.6, 4.5, 1.8, 3.2, 5.9, 5.2],
'O3': [1.4, 1.0, 0.0, 1.7, 0.9, 2.4, 4.1, 1.5, 3.0, 5.5, 4.8],
'O4': [0.4, 2.0, 1.7, 0.0, 2.6, 4.0, 5.5, 0.3, 4.6, 7.1, 6.3],
'O5': [2.2, 1.3, 0.9, 2.6, 0.0, 1.7, 3.4, 2.4, 2.1, 4.8, 4.1],
'O6': [3.7, 2.6, 2.4, 4.0, 1.7, 0.0, 2.0, 3.8, 1.6, 3.3, 2.7],
'O7': [5.2, 4.5, 4.1, 5.5, 3.4, 2.0, 0.0, 5.4, 2.5, 1.6, 0.9],
'O8': [0.2, 1.8, 1.5, 0.3, 2.4, 3.8, 5.4, 0.0, 4.4, 6.9, 6.1],
'O9': [4.3, 3.2, 3.0, 4.6, 2.1, 1.6, 2.5, 4.4, 0.0, 3.4, 2.9],
'O10':[6.8, 5.9, 5.5, 7.1, 4.8, 3.3, 1.6, 6.9, 3.4, 0.0, 1.0],
'O11': [6.0, 5.2, 4.8, 6.3, 4.1, 2.7, 0.9, 6.1, 2.9, 1.0, 0.0]
})
Whereas O1, O2, O3, O4, O5, O6, O7, O8 is class 0 and O9, O10 and O11 is class 1.
I want to change the dataframe above to a dataframe with columns: x, y and class. So I am able to split into train and test sets to then fit a simple classifier.
I am confused how I can achieve dataframe described above. How is this performed in python? Is it possible?
Steps afterwards when dataframe is achieved:
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import GaussianNB
import seaborn as sns
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.25)
model = GaussianNB()
model.fit(X_train, y_train)
y_pred = model.predict(X_test)
y_proba = model.predict_proba(X_test)
sns.scatterplot(x = X_test['x'], y = X_test['y'], hue = y_pred)
You mainly want to include the point name as an additional column in the dataframe. Here I am using point indices as x and y:
import pandas as pd
df = pd.DataFrame({
'x': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11],
1: [0.0, 1.7, 1.4, 0.4, 2.2, 3.7, 5.2, 0.2, 4.3, 6.8, 6.0],
2: [1.7, 0.0, 1.0, 2.0, 1.3, 2.6, 4.5, 1.8, 3.2, 5.9, 5.2],
3: [1.4, 1.0, 0.0, 1.7, 0.9, 2.4, 4.1, 1.5, 3.0, 5.5, 4.8],
4: [0.4, 2.0, 1.7, 0.0, 2.6, 4.0, 5.5, 0.3, 4.6, 7.1, 6.3],
5: [2.2, 1.3, 0.9, 2.6, 0.0, 1.7, 3.4, 2.4, 2.1, 4.8, 4.1],
6: [3.7, 2.6, 2.4, 4.0, 1.7, 0.0, 2.0, 3.8, 1.6, 3.3, 2.7],
7: [5.2, 4.5, 4.1, 5.5, 3.4, 2.0, 0.0, 5.4, 2.5, 1.6, 0.9],
8: [0.2, 1.8, 1.5, 0.3, 2.4, 3.8, 5.4, 0.0, 4.4, 6.9, 6.1],
9: [4.3, 3.2, 3.0, 4.6, 2.1, 1.6, 2.5, 4.4, 0.0, 3.4, 2.9],
10: [6.8, 5.9, 5.5, 7.1, 4.8, 3.3, 1.6, 6.9, 3.4, 0.0, 1.0],
11: [6.0, 5.2, 4.8, 6.3, 4.1, 2.7, 0.9, 6.1, 2.9, 1.0, 0.0]
})
That allows you to reshape the dataframe to your desired form:
model_df = df.melt(id_vars='x', var_name='y', value_name='distance')
Finally, define a class e.g. using:
def assign_class(x):
return 0 if x <= 8 else 1
model_df["class_x"] = model_df["x"].apply(assign_class),
model_df["class_y"] = model_df["y"].apply(assign_class)
This will give you a dataframe that you can pass to the model. Note that the input matrix is symmetric, so you may want to only keep unique records (drop [y, x] if you already have [x, y]).
from pylab import *
def x(t) :
if 0 <= t < 8 :
return(2*t)
elif 8 <= t < 20 :
return(t**3)
t = arange(5.0, 20, 0.3)
print([i for i in t])
Output is
[5.0, 5.3, 5.6, 5.8999999999999995, 6.199999999999999, 6.499999999999999, 6.799999999999999, 7.099999999999999, 7.399999999999999, 7.699999999999998, 7.999999999999998, 8.299999999999997, 8.599999999999998, 8.899999999999999, 9.199999999999998, 9.499999999999996, 9.799999999999997, 10.099999999999998, 10.399999999999997, 10.699999999999996, 10.999999999999996, 11.299999999999997, 11.599999999999996, 11.899999999999995, 12.199999999999996, 12.499999999999996, 12.799999999999995, 13.099999999999994, 13.399999999999995, 13.699999999999996, 13.999999999999995, 14.299999999999994, 14.599999999999994, 14.899999999999995, 15.199999999999994, 15.499999999999993, 15.799999999999994, 16.099999999999994, 16.39999999999999, 16.699999999999992, 16.999999999999993, 17.299999999999994, 17.599999999999994, 17.89999999999999, 18.199999999999992, 18.499999999999993, 18.79999999999999, 19.09999999999999, 19.39999999999999, 19.699999999999992, 19.999999999999993]
What I want is
[5.0, 5.3, 5.6, 5.9, 6.2, 6.5, 6.8, 7.1, 7.4, 7.7, 8.0, so on]
When it comes to 8.0, my output is 7.999999999999998 < 8.
So wrong answer.
I want 8.0.
So that I can plot function.
plot(t, array([x(i) for i in t]))
I guess a simple rounding off is all you need.
Change the last line to this:
print([round(i,1) for i in t])
Output:
[5.0, 5.3, 5.6, 5.9, 6.2, 6.5, 6.8, 7.1, 7.4, 7.7, 8.0, 8.3, 8.6, 8.9, 9.2, 9.5, 9.8, 10.1, 10.4, 10.7, 11.0, 11.3, 11.6, 11.9, 12.2, 12.5, 12.8, 13.1, 13.4, 13.7, 14.0, 14.3, 14.6, 14.9, 15.2, 15.5, 15.8, 16.1, 16.4, 16.7, 17.0, 17.3, 17.6, 17.9, 18.2, 18.5, 18.8, 19.1, 19.4, 19.7]
So in your case the code becomes something like:
from pylab import *
def x(t) :
if 0 <= t < 8 :
return(2*t)
elif 8 <= t < 20 :
return(t**3)
t = arange(5.0, 20, 0.3)
t = [round(i,1) for i in t]
print(t)
Now you can use this t and get the following plot:
def process_timecards():
timecards = []
with open("timecards.txt") as f:
reader = csv.reader(f)
listoftimecards = [list(map(float,row)) for row in reader]
print(listoftimecards)
list1 = listoftimecards.pop(0)
print(list1)
[[688997.0, 5.0, 6.8, 8.0, 7.7, 6.6, 5.2, 7.1, 4.0, 7.5, 7.6], [939825.0, 7.9, 6.6, 6.8, 7.4, 6.4, 5.1, 6.7, 7.3, 6.8, 4.1], [900100.0, 5.1, 6.8, 5.0, 6.6, 7.7, 5.1, 7.5], [969829.0, 6.4, 6.6, 4.4, 5.0, 7.1, 7.1, 4.1, 6.5], [283809.0, 7.2, 5.8, 7.6, 5.3, 6.4, 4.6, 6.4, 5.0, 7.5], [224568.0, 5.2, 6.9, 4.2, 6.4, 5.3, 6.8, 4.4], [163695.0, 4.8, 7.2, 7.2, 4.7, 5.1, 7.3, 7.5, 4.5, 4.6, 7.0], [454912.0, 5.5, 5.3, 4.5, 4.3, 5.5], [285767.0, 7.5, 6.5, 6.3, 4.7, 6.8, 7.1, 6.6, 6.6], [674261.0, 7.2, 6.2, 4.9, 6.5, 7.2, 7.5, 5.0, 7.9], [426824.0, 7.4, 6.5, 5.7, 8.0, 6.9, 7.5, 6.5, 7.5], [934003.0, 5.8, 7.5, 5.8, 4.8, 5.9, 4.8, 4.0, 6.6, 5.5, 7.2]]
This is the list of lists that I have, and i need to grab the first value of each list inside of the list of lists and store that into a list.
I thought i could use pop, but that only goes to the first list. It results in just printing out the first value of the list, which is the first list.
Any advice? I was thinking maybe a for loop, but i have no idea how i would format it.
list1 = [sublist[0] for sublist in listoftimecards]
since you want only the first element you could grab from each row only the first element using the built-in function next:
def process_timecards():
timecards = []
with open("timecards.txt") as f:
reader = csv.reader(f)
list1 = [next(map(float,row)) for row in reader]
print(list1)
list1 = []
for item in listoftimecards:
list1.append(item[0])
This loops through each item in listoftimecards and appends the first item of every list in listoftimecardsto list1
This code below does the same thing as the code above.
list1 = [x[0] for x in listoftimecards]
In Python 2, you can use:
map(lambda lst:lst[0], listoftimecards)
In Python 3, map returns you a generator so you need to call list():
list(map(lambda lst:lst[0], listoftimecards))
but if you only wish to iterate the result using map will be memory efficient.
When I print my list I get something like this
[[6.0, 0.5], [6.1, 1.0], [6.2, 1.5], [6.3, 2.0], [6.4, 2.5], [6.5, 3.0], [6.6, 3.5], [6.7, 4.0], [6.8, 4.5]]
I want to extract first and second elements from above list into separate lists so that I can ask the plt to plot it for me.
So my results should be
[6.0,6.1,6.2 ... 6.8] and [0.5,1.0,1.5,2.0 , ... .4.5]
I want to know if we have a cleaner solution than to
for sublist in l:
i=0
for item in sublist:
flat_list.append(item)
break #get first element of each
You can try list indexing:
data = [[6.0, 0.5], [6.1, 1.0], [6.2, 1.5], [6.3, 2.0], [6.4, 2.5], [6.5, 3.0], [6.6, 3.5], [6.7, 4.0], [6.8, 4.5]]
d1 = [item[0] for item in data]
print d1
d2 = [item[1] for item in data]
print d2
output :
[6.0, 6.1, 6.2, 6.3, 6.4, 6.5, 6.6, 6.7, 6.8]
[0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5]
zip() will provide the required output.
xy = [[6.0, 0.5], [6.1, 1.0], [6.2, 1.5], [6.3, 2.0], [6.4, 2.5], [6.5, 3.0], [6.6, 3.5], [6.7, 4.0], [6.8, 4.5]]
x,y = zip(*xy)
print(x)
print(y)
Output:
(6.0, 6.1, 6.2, 6.3, 6.4, 6.5, 6.6, 6.7, 6.8)
(0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5)
zip() aggregates the elements from all the iterable. zip(x,y) would provide the list you currently have. zip() with * can be used to unzip a list.
Also, there is no need to convert the tuples to list since pyplot.plot() takes an array-like parameter.
import matplotlib.pyplot as plt
plt.plot(x,y)
plt.show()
I would recommend using numpy arrays. For example:
import matplotlib.pyplot as plt
import numpy as np
a= np.array([[6.0, 0.5], [6.1, 1.0], [6.2, 1.5], [6.3, 2.0], [6.4, 2.5], [6.5, 3.0], [6.6, 3.5], [6.7, 4.0], [6.8, 4.5]])
plt.plot(a[:,0], a[:,1])
plt.show()
Output:
Here a try with zip, zip() will makes iterator that aggregates elements based on the iterables passed, and returns an iterator of tuples, so map() function is used to make the tuples to list :
l = [[6.0, 0.5], [6.1, 1.0], [6.2, 1.5], [6.3, 2.0], [6.4, 2.5], [6.5, 3.0], [6.6, 3.5], [6.7, 4.0], [6.8, 4.5]]
a,b = map(list,zip(*l))
print(a,b)
O/P will be like :
[6.0, 6.1, 6.2, 6.3, 6.4, 6.5, 6.6, 6.7, 6.8] [0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5]
One-liner using zip built-in and unpacking
>>> original = [[6.0, 0.5], [6.1, 1.0], [6.2, 1.5], [6.3, 2.0], [6.4, 2.5], [6.5, 3.0], [6.6, 3.5], [6.7, 4.0], [6.8, 4.5]]
>>> left, right = zip(*original)
>>> left
(6.0, 6.1, 6.2, 6.3, 6.4, 6.5, 6.6, 6.7, 6.8)
>>> right
(0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5)
if you are embarassed that results are tuples we can turn them into lists simply using map built-in:
>>> left, right = map(list, zip(*original))
>>> left
[6.0, 6.1, 6.2, 6.3, 6.4, 6.5, 6.6, 6.7, 6.8]
>>> right
[0.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5]
Lots of pure Python approaches here. But given that your goal is to plot the separated values, I think there's a case to be made here for the simplicity of Pandas - just drop the list as-is into a data frame and plot():
import pandas as pd
pd.DataFrame(data).plot(x=0, y=1)
l = [[6.0, 0.5], [6.1, 1.0], [6.2, 1.5], [6.3, 2.0], [6.4, 2.5], [6.5, 3.0], [6.6, 3.5], [6.7, 4.0], [6.8, 4.5]]
a,b=list(zip(*l))
print('first elements:',a)
print('second elements:',a)
To plot:
import matplotlib.pyplot as plt
l = [[6.0, 0.5], [6.1, 1.0], [6.2, 1.5], [6.3, 2.0], [6.4, 2.5], [6.5, 3.0], [6.6, 3.5], [6.7, 4.0], [6.8, 4.5]]
a,b=list(zip(*l))
print('first elements:',a)
print('second elements:',a)
plt.plot(a,b)
plt.show()
This question already has answers here:
Python rounding error with float numbers [duplicate]
(2 answers)
How computer does floating point arithmetic?
(5 answers)
Closed 8 years ago.
I have a list of Base Resistor Values
BaseRes=[1.0,1.1,1.2,1.3,1.5,1.6,1.8,2.0,2.2,2.4,2.7,3.0,3.3,3.6,3.9,4.3,4.7,5.1,5.6,6.2,6.8,7.5,8.2,9.1]
I then want to make a list containing these base values and those values multiplied by 10:
Resistors = BaseRes
Resistors = Resistors+[res*10 for res in BaseRes]
output:
Resistors
[1.0, 1.1, 1.2, 1.3, 1.5, 1.6, 1.8, 2.0, 2.2, 2.4, 2.7, 3.0, 3.3, 3.6, 3.9, 4.3, 4.7, 5.1, 5.6, 6.2, 6.8, 7.5, 8.2, 9.1, 10.0, 11.0, 12.0, 13.0, 15.0, 16.0, 18.0, 20.0, 22.0, 24.0, 27.0, 30.0, 33.0, 36.0, 39.0, 43.0, 47.0, 51.0, 56.0, 62.0, 68.0, 75.0, 82.0, 91.0]
As you can see this works fine for me. However when I want to add the list of the base resistor values multiplied by 100 I get some odd values for a few of the values:
Resistors = Resistors+[res*100 for res in BaseRes]
output:
Resistors
[1.0, 1.1, 1.2, 1.3, 1.5, 1.6, 1.8, 2.0, 2.2, 2.4, 2.7, 3.0, 3.3, 3.6, 3.9, 4.3, 4.7, 5.1, 5.6, 6.2, 6.8, 7.5, 8.2, 9.1, 10.0, 11.0, 12.0, 13.0, 15.0, 16.0, 18.0, 20.0, 22.0, 24.0, 27.0, 30.0, 33.0, 36.0, 39.0, 43.0, 47.0, 51.0, 56.0, 62.0, 68.0, 75.0, 82.0, 91.0, 100.0, 110.00000000000001, 120.0, 130.0, 150.0, 160.0, 180.0, 200.0, 220.00000000000003, 240.0, 270.0, 300.0, 330.0, 360.0, 390.0, 430.0, 470.0, 509.99999999999994, 560.0, 620.0, 680.0, 750.0, 819.9999999999999, 910.0]
mainly the 110 value, the 220 value, the 510 value and the 820 value are confusing me.
I do not understand what is going on here.
Any help would be greatly appreciated
While it is true that what you're seeing is rounding errors in floating-point representations, you're probably here for a fix. I'd recommend Python's decimal module, which is designed to handle situations where you need precise, human math and not fast binary approximations.
from decimal import *
getcontext().prec = 3
resBase = [Decimal(x) for x in [1.0, 1.1, 1.2, 1.3, 1.5, 1.6, 1.8,
2.0, 2.2, 2.4, 2.7, 3.0, 3.3, 3.6, 3.9, 4.3, 4.7, 5.1, 5.6,
6.2, 6.8, 7.5, 8.2, 9.1]]
print [res * 100 for res in resBase]
Output:
[Decimal('100'), Decimal('110'), Decimal('120'), Decimal('130'), Decimal('150'),
Decimal('160'), Decimal('180'), Decimal('200'), Decimal('220'), Decimal('240'),
Decimal('270'), Decimal('300'), Decimal('330'), Decimal('360'), Decimal('390'),
Decimal('430'), Decimal('470'), Decimal('510'), Decimal('560'), Decimal('620'),
Decimal('680'), Decimal('750'), Decimal('820'), Decimal('910')]
As others have pointed out, this is a floating-point issue. You're probably wondering how to fix it, though.
Try something like:
import math
Resistors = Resistors+[math.floor(res*100) for res in BaseRes]
I recommend reading the documentation for Python's math package for more information and some useful functions that can really help out. Happy coding!