We Keep Coding

How to create a column that contains the penultimate value of each row?

I have a DataFrame and I need to create a new column which contains the second largest value of each row in the original Dataframe.
Sample:
df = pd.DataFrame(np.random.randint(1,100, 80).reshape(8, -1))
Desired output:
0 1 2 3 4 5 6 7 8 9 penultimate
0 52 69 62 7 20 69 38 10 57 17 62
1 52 94 49 63 1 90 14 76 20 84 90
2 78 37 58 7 27 41 27 26 48 51 58
3 6 39 99 36 62 90 47 25 60 84 90
4 37 36 91 93 76 69 86 95 69 6 93
5 5 54 73 61 22 29 99 27 46 24 73
6 71 65 45 9 63 46 4 93 36 18 71
7 85 7 76 46 65 97 64 52 28 80 85
How can this be done in as little code as possible?

You could use NumPy for this:
import numpy as np
df = pd.DataFrame(np.random.randint(1,100, 80).reshape(8, -1))
df['penultimate'] = np.sort(df.values, 1)[:, -2]
print(df)
Using NumPy is faster.

Here is a simple lambda function!
# Input
df = pd.DataFrame(np.random.randint(1,100, 80).reshape(8, -1))
# Output
out = df.apply(lambda x: x.sort_values().unique()[-2], axis=1)
df['penultimate'] = out
print(df)
Cheers!

Is there a better way to keep few columns by dropping several columns from a DataFrame than to use .drop()?

Curious to know if there is a better way to keep needed columns in a dataframe if those I need to keep are a small number and the ones to remove are several of them
import numpy as np
df1 = pd.DataFrame(np.random.randint(10,99, size=(13, 26)), columns =list('abcdefghijklmnopqrstuvwxyz'))
df1
Output:
a b c d e f g h i j ... q r s t u v w x y z
0 78 60 27 38 21 93 74 47 16 53 ... 79 56 40 41 87 80 14 82 12 50
1 84 73 59 46 91 43 22 28 57 52 ... 27 65 81 72 68 90 68 61 22 44
2 56 37 29 52 57 14 87 82 46 90 ... 67 57 29 14 55 30 46 72 56 91
3 86 44 46 79 41 74 32 49 42 32 ... 33 34 40 17 30 78 29 75 80 52
4 14 89 90 79 67 17 34 39 57 37 ... 93 49 78 91 26 73 40 48 91 36
5 16 62 32 87 56 81 82 17 59 57 ... 84 24 97 39 46 40 68 53 73 40
6 69 72 16 47 37 20 27 56 13 37 ... 10 28 17 35 39 14 51 85 69 53
7 81 34 35 20 66 44 86 23 94 57 ... 38 45 76 53 82 72 64 34 81 43
8 95 90 97 31 18 85 74 18 43 22 ... 20 20 96 25 53 76 55 96 58 98
9 73 53 72 94 55 33 22 40 11 64 ... 84 66 85 34 94 32 78 72 10 62
10 73 24 57 17 63 24 94 25 59 84 ... 34 45 27 28 47 23 38 80 45 41
11 69 18 22 42 95 38 16 47 68 36 ... 59 69 35 39 78 75 85 86 53 55
12 46 27 53 77 48 15 57 90 32 57 ... 32 79 18 67 71 86 54 11 36 51
13 rows × 26 columns
Say, I have to only keep a few random columns , E.g. e,u,r,q,j ; is there a better way to keep them having to run df1.drop() with 21 column names passed in? I could not find a better way in any of the questions.
Edit:
Different from the solution in
Selecting multiple columns in a pandas dataframe
since the columns to choose to drop are random and not sequential

You can copy all the rows you want to keep into a new dataframe and then overwrite your first dataframe like so:
import numpy as np
import pandas as pd
df1 = pd.DataFrame(np.random.randint(10,99, size=(13, 26)), columns =list('abcdefghijklmnopqrstuvwxyz'))
df2 = pd.DataFrame()
columns_to_keep = ["e", "r", "u"]
for column in columns_to_keep:
df2[column] = df1[column]
df1 = df2
df1
or alternatively using a for statement to drop any item not in a list:
columns_to_keep = ["e", "r", "u"]
for column_name, column_data in df1.iteritems():
if column_name not in columns_to_keep:
df1 = df1.drop(column_name, axis=1)
df1

Let's just use column filtering and reassign back to df1:
df1 = pd.DataFrame(np.random.randint(10,99, size=(13, 26)), columns =list('abcdefghijklmnopqrstuvwxyz'))
columns_to_keep = ["e", "r", "u"]
df1 = df1[columns_to_keep]
df1.head()
Output:
e r u
0 65 95 13
1 58 42 75
2 95 34 12
3 43 20 79
4 83 27 47

Sorting output from a while loop in Python 3

The following code prints out number sequences up to around 100 from a list. A fair amount of the sequences print out above 100. I want to know how to only print out the numbers that add up to 100 on the button. I have tried printing the results to a list without luck. I tried putting in if and else statements to filter the results but with no luck. I looked at list comprehensions but I know those don't use while loops and so I don't know how to get the same results with a for loop. The only information I can seem to find online is basic lessons on how to use a while loop and just printing a list of numbers out. I could not find anything about how to sort a list of numbers printed.
Here is the code:
import itertools
list1 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
for i in list1:
a = 0
num1 = 2
num2 = i
seq = ([a])
it = itertools.cycle((num1,num2))
while a < 100:
a += next(it)
print(a, end = " ")
seq.append(a)
print()
print("Here are the numbers", num1, "&", num2, "added together in a sequence")
print()
Output:
2 3 5 6 8 9 11 12 14 15 17 18 20 21 23 24 26 27 29 30 32 33 35 36 38 39 41 42 44 45 47 48 50 51 53 54 56 57 59 60 62 63 65 66 68 69 71 72 74 75 77 78 80 81 83 84 86 87 89 90 92 93 95 96 98 99 101
Here are the numbers 2 & 1 added together in a sequence
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72 74 76 78 80 82 84 86 88 90 92 94 96 98 100
Here are the numbers 2 & 2 added together in a sequence
2 5 7 10 12 15 17 20 22 25 27 30 32 35 37 40 42 45 47 50 52 55 57 60 62 65 67 70 72 75 77 80 82 85 87 90 92 95 97 100
Here are the numbers 2 & 3 added together in a sequence
2 6 8 12 14 18 20 24 26 30 32 36 38 42 44 48 50 54 56 60 62 66 68 72 74 78 80 84 86 90 92 96 98 102
Here are the numbers 2 & 4 added together in a sequence
2 7 9 14 16 21 23 28 30 35 37 42 44 49 51 56 58 63 65 70 72 77 79 84 86 91 93 98 100
Here are the numbers 2 & 5 added together in a sequence
2 8 10 16 18 24 26 32 34 40 42 48 50 56 58 64 66 72 74 80 82 88 90 96 98 104
Here are the numbers 2 & 6 added together in a sequence
2 9 11 18 20 27 29 36 38 45 47 54 56 63 65 72 74 81 83 90 92 99 101
Here are the numbers 2 & 7 added together in a sequence
2 10 12 20 22 30 32 40 42 50 52 60 62 70 72 80 82 90 92 100
Here are the numbers 2 & 8 added together in a sequence
2 11 13 22 24 33 35 44 46 55 57 66 68 77 79 88 90 99 101
Here are the numbers 2 & 9 added together in a sequence
2 12 14 24 26 36 38 48 50 60 62 72 74 84 86 96 98 108
Here are the numbers 2 & 10 added together in a sequence
What I want is:
2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72 74 76 78 80 82 84 86 88 90 92 94 96 98 100
Here are the numbers 2 & 2 added together in a sequence
2 5 7 10 12 15 17 20 22 25 27 30 32 35 37 40 42 45 47 50 52 55 57 60 62 65 67 70 72 75 77 80 82 85 87 90 92 95 97 100
Here are the numbers 2 & 3 added together in a sequence
2 7 9 14 16 21 23 28 30 35 37 42 44 49 51 56 58 63 65 70 72 77 79 84 86 91 93 98 100
Here are the numbers 2 & 5 added together in a sequence
2 10 12 20 22 30 32 40 42 50 52 60 62 70 72 80 82 90 92 100
Here are the numbers 2 & 8 added together in a sequence
Any and all help on this will be greatly appreciated.

Well, you only know if your sequence addition adds up to 100 once you are done, so you can't start printing before that point. This should do the job:
import itertools
list1 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
for i in list1:
a = 0
num1 = 2
num2 = i
seq = ([a])
it = itertools.cycle((num1,num2))
while a < 100:
a += next(it)
seq.append(a)
if seq[-1] == 100: # -1 as an index gets the last entry in a list
print(" ".join([str(val) for val in seq]))
print("Here are the numbers", num1, "&", num2, "added together in a sequence")
print()

Unable to return correct output from created function in JES

I am fairly new to jython and I am inquiring about creating a function that is dealing with a list. Basically what I am trying to do with the below function is create a function that will loop through the entirety of my list, then find the lowest value within that list and return the variable with the lowest number. Though, I keep getting a return of function min at 0x26 everytime I execute the main() I receive the same message but it seems as if the function min at 0x26 will count up ex: 0x27, 0x28... Not sure why this is. As my list only contains integers of minimum 0 to max 99.
Here is the sourcecode:
def min(dataset): #defining a function minimum, with input dataset(the list we are using)..
min = dataset[0]
for num in range(0, len(dataset)):
if dataset[num] < min:
min = dataset(num)
return min
minimum = min(dataset)
print(str(minimum))
Here is the code in its entirety. Though, I currently have a way to find the min/max values in the list. I am looking to move towards a function, as I want to know how to correctly use a function.
def main( ):
dataset = [0]
file = open("D:\numbs.dat", "r")
for line in file: #loop for writing over every line to a storage loc.
num = int(float(line)) #converting int to string
dataset.append(num) #appending the data to 'dataset' list
max = dataset[0] #setting an imaginary max initially
low = dataset[0] #setting an imaginary low initially
for i in range(0, len(dataset)): #for loop to scan thru entire list
if dataset[i] > max: #find highest through scan and replacing each max in var max
max = dataset[i]
if dataset[i] < low: #find lowest through scan and replacing each max in var max
low = dataset[i]
#printNow(dataset) #printing the list in its entirety
#printNow("The maximum is " +str(max)) #print each values of lowest and highest
#printNow("The lowest is " +str(low))
def min(dataset): #defining a function minimum..
min = dataset[0]
for num in range(0, len(dataset)):
if dataset[num] < min:
min = dataset(num)
return min
minimum = min(dataset)
print(str(minimum)) #test to see what output is.
As mentioned above, there is the for loop for finding max/min values. Though I tried doing the same exact thing for the function I am trying to create...
the contents of the numbs.dat can be found here (1001 entries):
70
75
76
49
73
76
52
63
11
25
19
89
17
48
5
48
29
41
23
84
28
39
67
48
97
34
0
24
47
98
0
64
24
51
45
11
37
77
5
54
53
33
91
0
27
0
80
5
11
66
45
57
48
25
72
8
38
29
93
29
58
5
72
36
94
18
92
17
43
82
44
93
10
38
31
52
44
10
50
22
39
71
46
40
33
51
51
57
27
24
40
61
88
87
40
85
91
99
6
3
56
10
85
38
61
91
31
69
39
74
9
17
80
96
49
0
47
68
12
5
6
60
81
51
62
87
70
66
50
30
30
22
45
35
2
39
23
63
35
69
83
84
69
6
54
74
3
29
31
54
45
79
21
74
30
77
77
80
26
63
84
21
58
54
69
2
50
79
90
26
45
29
97
28
57
22
59
2
72
1
92
35
38
2
47
23
52
77
87
34
84
15
84
13
23
93
19
50
99
74
59
4
73
93
29
61
8
45
10
20
15
95
58
43
75
19
61
39
68
47
69
58
88
82
33
30
72
21
74
12
18
0
52
50
62
21
66
26
56
84
16
12
7
45
58
22
26
95
82
6
74
12
16
2
61
58
22
39
0
53
88
79
71
13
54
25
31
93
48
91
90
45
23
54
42
39
78
25
95
58
2
41
61
72
98
91
48
97
93
11
12
1
35
80
81
86
38
70
67
55
55
87
73
79
31
43
97
79
3
51
17
58
70
34
59
61
28
46
13
42
18
0
18
75
75
62
50
62
85
49
83
71
63
32
27
59
42
46
8
13
39
25
13
94
17
48
73
40
31
31
86
23
81
40
92
24
94
67
30
18
74
78
62
89
1
27
95
99
33
53
74
5
84
88
8
52
0
24
21
99
1
74
84
94
29
25
83
93
98
40
21
66
93
28
72
63
77
9
71
18
87
50
77
48
68
88
22
33
16
79
68
69
94
64
5
28
33
22
21
74
44
62
68
47
93
69
9
42
44
87
64
97
42
34
90
70
91
12
18
84
65
23
99
1
55
6
1
23
92
50
96
96
68
27
17
98
42
10
27
26
20
13
94
73
75
12
12
25
33
1
33
67
61
0
98
71
35
75
68
56
45
11
1
69
57
9
15
96
69
2
0
65
44
86
78
97
17
4
81
23
4
43
24
72
70
57
21
91
84
94
40
96
40
78
46
67
6
7
16
49
24
14
12
82
73
60
42
76
62
10
84
49
75
89
43
47
31
68
15
11
32
37
98
72
40
25
69
30
64
60
48
21
11
74
54
24
60
10
96
29
39
53
48
24
68
4
52
12
6
91
15
86
77
65
68
22
91
36
72
82
81
9
77
0
5
83
27
88
17
35
66
76
78
81
19
51
87
66
26
59
65
2
37
37
73
34
98
37
78
92
17
52
62
40
50
84
34
22
25
42
90
19
86
76
68
42
9
89
57
78
64
89
12
34
94
9
77
58
32
27
97
93
79
35
32
75
97
79
65
90
53
43
98
4
99
5
79
38
99
60
78
64
90
2
39
42
52
2
21
77
15
8
87
13
0
4
7
43
76
31
74
16
87
50
73
49
14
35
10
37
91
44
88
71
95
75
98
7
17
23
13
16
77
20
50
50
74
78
58
30
21
74
76
93
5
74
94
83
23
67
18
5
50
47
56
79
26
84
78
48
71
43
41
8
91
23
7
11
96
87
12
42
32
44
99
67
99
64
96
52
19
79
60
66
52
62
17
61
54
24
25
36
4
78
3
94
91
62
65
76
94
2
52
25
61
55
49
88
85
96
5
46
56
48
17
25
3
70
62
3
50
45
47
58
12
41
27
42
90
91
71
53
4
79
47
68
43
87
35
63
10
49
4
81
45
88
80
6
92
47
70
40
7
33
70
61
30
9
55
42
83
26
72
57
77
91
13
15
33
13
62
49
43
65
73
98
59
56
77
62
12
25
33
53
78
73
1
17
44
56
95
10
33
89
33
20
56
69
66
60
53
83
58
43
33
25
21
8
28
65
51
70
53
78
49
30
64
17
76
9
2
32
87
77
39
25
21
66
65
54
81
49
15
27
7
14
4
11
94
9
84
23
13
95
45
67
57
20
3
58
50
97
35
68
47
41
84
59
46
34
19
25
77
29
41
89
80
61
70
40
1
18
32
70
86
76
25
98
99
40
43
92
43
4
70
78
72
71
85
14
84
73
92
60
23
57
44
56
6
96
39
91
63
43
39
71
80
18
93
54
1
4
46
68
93
74
74
88
52
88
55
24
19
92
53
59
1
91
48
47
Let me know what the heck I am doing wrong. Thanks!

#ohGosh welcome to stack overflow. You are almost there with the solution. There are few problems with your program
1) Nums.dat file contains just one line with numbers separated by spaces, not a new line(\n). In order to get the read the numbers from the file do the following
dataset = [] #Create an empty list
file = open("D:\numbs.dat", "r") #Open the file
for line in file:
tempData = line.split(" ") #Returns a list with space used as delimiter
dataset = map(int, tempData) #Convert string data to int
2) Wrong way to get data from a list in the min function
Use
min = dataset[num]
Instead of
min = dataset(num)
Fix this and your program will work. Cheers.

Object Similarity Pandas and Scikit Learn

Is there a way to find to find and rank rows in a Pandas Dataframe by their similarity to a row from another Dataframe?

My understanding of your question: you have two data frames, hopfully of the same column count. You want to rate first data frame's, the subject data frame, members by how close, i.e. similar, they are to any of the members of the target data frame.
I am not aware of a built in method.
It is probably not the most efficient way but here is how I'd approach:
#! /usr/bin/python3
import pandas as pd
import numpy as np
import pprint
pp = pprint.PrettyPrinter(indent=4)
# Simulate data
df_subject = pd.DataFrame(np.random.randint(0,100,size=(100, 4)), columns=list('ABCD')) # This is the one we're iterating to check similarity to target.
df_target = pd.DataFrame(np.random.randint(0,100,size=(100, 4)), columns=list('ABCD')) # This is the one we're checking distance to
# This will hold the min dstances.
distances=[]
# Loop to iterate over subject DF
for ix1,subject in df_subject.iterrows():
distances_cur=[]
# Loop to iterate over target DF
for ix2,target in df_target.iterrows():
distances_cur.append(np.linalg.norm(target-subject))
# Get the minimum distance for the subject set member.
distances.append(min(distances_cur))
# Distances to df
distances=pd.DataFrame(distances)
# Normalize.
distances=0.5-(distances-distances.mean(axis=0))/distances.max(axis=0)
# Column index joining, ordering and beautification.
Proximity_Ratings_name='Proximity Ratings'
distances=distances.rename(columns={0: Proximity_Ratings_name})
df_subject=df_subject.join(distances)
pp.pprint(df_subject.sort_values(Proximity_Ratings_name,ascending=False))
It should yeild something like the table below. Higher rating means there's a similar member in the target data frame:
A B C D Proximity Ratings
55 86 21 91 78 0.941537
38 91 31 35 95 0.901638
43 49 89 49 6 0.878030
98 28 98 98 36 0.813685
77 67 23 78 84 0.809324
35 52 16 36 58 0.802223
54 2 25 61 44 0.788591
95 76 3 60 46 0.766896
5 55 39 88 37 0.756049
52 79 71 90 70 0.752520
66 52 27 82 82 0.751353
41 45 67 55 33 0.739919
76 12 93 50 62 0.720323
94 99 84 39 63 0.716123
26 62 6 97 60 0.715081
40 64 50 37 27 0.714042
68 70 21 8 82 0.698824
47 90 54 60 65 0.676680
7 85 95 45 71 0.672036
2 14 68 50 6 0.661113
34 62 63 83 29 0.659322
8 87 90 28 74 0.647873
75 14 61 27 68 0.633370
60 9 91 42 40 0.630030
4 46 46 52 35 0.621792
81 94 19 82 44 0.614510
73 67 27 34 92 0.608137
30 92 64 93 51 0.608137
11 52 25 93 50 0.605770
51 17 48 57 52 0.604984
.. .. .. .. .. ...
64 28 56 0 9 0.397054
18 52 84 36 79 0.396518
99 41 5 32 34 0.388519
27 19 54 43 94 0.382714
92 69 56 73 93 0.382714
59 1 29 46 16 0.374878
58 2 36 8 96 0.362525
69 58 92 16 48 0.361505
31 27 57 80 35 0.349887
10 59 23 47 24 0.345891
96 41 77 76 33 0.345891
78 42 71 87 65 0.344398
93 12 31 6 27 0.329152
23 6 5 10 42 0.320445
14 44 6 43 29 0.319964
6 81 51 44 15 0.311840
3 17 60 13 22 0.293066
70 28 40 22 82 0.251549
36 95 72 35 5 0.249354
49 78 10 30 18 0.242370
17 79 69 57 96 0.225168
46 42 95 86 81 0.224742
84 58 81 59 86 0.221346
9 9 62 8 30 0.211659
72 11 51 74 8 0.159265
90 74 26 80 1 0.138993
20 90 4 6 5 0.117652
50 3 12 5 53 0.077088
42 90 76 42 1 0.075284
45 94 46 88 14 0.054244
Hope I understand correctly. Don't use if performance matters, I'm sure there's an algebraic way to approach this (Multiply matrices) that would run way faster.