We Keep Coding

Can a Dataframe of NBA players be sorted by various conditions: to combine the rows of players w/ multiple entries bc they played on many teams?

I want to remove any players who didn't have over 1000 MP(minutes played).
I could easily write:
league_stats= pd.read_csv("1996.csv")
league_stats = league_stats.drop("Player-additional", axis=1)
league_stats_1000 = league_stats[league_stats['MP'] > 1000]
However, because players sometimes play for multiple teams in a year...this code doesn't account for that.
For example, Sam Cassell has four entries and none are above 1000 MP, but in total his MP for the season was over 1000. By running the above code I remove him from the new dataframe.
I am wondering if there is a way to sort the Dataframe by matching Rank(the RK column gives players who played on different teams the same rank number for each team they played on) and then sort it by... if the total of their MP is 1000=<.
This is the page I got the data from: 1996-1997 season.
Above the data table and to the left of the blue check box there is a dropdown menu called "Share and Export". From there I clicked on "Get table as CSV (for Excel)". After that I saved the CSV to a text editor and change the file extension to .csv to upload it to Jupyter Notebook.
This is a solution I came up with:
url = 'https://www.basketball-reference.com/leagues/NBA_1997_totals.html'
df = pd.read_html(url)[0]
tot_df = df.loc[df['Tm'] == 'TOT']
mp_1000 = tot_df.loc[tot_df["MP"] < 1000]
# Create list of indexes with unnecessary entries to be removed. We have TOT and don't need these rows.
# *** For the record, I came up with this list by manually going through the data.
indexes_to_remove = [5,6,24, 25, 66, 67, 248, 249, 447, 448, 449, 275, 276, 277, 19, 20, 21, 377, 378, 477, 478, 479,
54, 55, 451, 452, 337, 338, 156, 157, 73, 74, 546, 547, 435, 436, 437, 142, 143, 421, 42, 43, 232,
233, 571, 572, 363, 364, 531, 532, 201, 202, 111, 112, 139, 140, 307, 308, 557, 558, 93, 94, 512,
513, 206, 207, 208, 250, 259, 286, 287, 367, 368, 271, 272, 102, 103, 34, 35, 457, 458, 190, 191,
372, 373, 165, 166
]
df_drop_tot = df.drop(labels=indexes_to_remove, axis=0)
df_drop_tot

First off, no need to manually download the csv and then read it into pandas. You can load in the table using pandas' .read_html().
And yes, you can simply get the list of ranks, player names, or whatever, that have greater than 1000 MP, then use that list to filter the dataframe.
import pandas as pd
url = 'https://www.basketball-reference.com/leagues/NBA_1997_totals.html'
df = pd.read_html(url)[0]
df = df[df['Rk'].ne('Rk')]
df['MP'] = df['MP'].astype(int)
players_1000_rk_list = list(df[df['MP'] >= 1000]['Rk']) #<- coverts the "Rk" column into a list. I can then use that in the next line to only keep the "Rk" values that are in the list of "Rk"s that are >= 1000 MPs
players_df = df[df['Rk'].isin(players_1000_rk_list)]
Output: filters down from 574 rows to 282 rows
print(players_df)
Rk Player Pos Age Tm G ... AST STL BLK TOV PF PTS
0 1 Mahmoud Abdul-Rauf PG 27 SAC 75 ... 189 56 6 119 174 1031
1 2 Shareef Abdur-Rahim PF 20 VAN 80 ... 175 79 79 225 199 1494
3 4 Cory Alexander PG 23 SAS 80 ... 254 82 16 146 148 577
7 6 Ray Allen* SG 21 MIL 82 ... 210 75 10 149 218 1102
10 9 Greg Anderson C 32 SAS 82 ... 34 63 67 73 225 322
.. ... ... .. .. ... .. ... ... ... .. ... ... ...
581 430 Walt Williams SF 26 TOR 73 ... 197 97 62 174 282 1199
582 431 Corliss Williamson SF 23 SAC 79 ... 124 60 49 157 263 915
583 432 Kevin Willis PF 34 HOU 75 ... 71 42 32 119 216 842
589 438 Lorenzen Wright C 21 LAC 77 ... 49 48 60 79 211 561
590 439 Sharone Wright C 24 TOR 60 ... 28 15 50 93 146 390
[282 rows x 30 columns]

Python: How to pass multiple x, y coordinates from a df into a function at once?

I have a df containing x, y coordinates that looks like this:
x y
280 230
230 247
219 255
209 270
203 290
199 313
198 336
204 365
208 372
220 392
253 429
281 448
... ...
The function that I created must take in 4 x coordinates and 4 corresponding y coordinates at a time, however I don't know how to iterate through a df and pass 4 rows of data at a time. Here is an example of my function call at the moment:
my_function(ax, ay, bx, by, cx, cy, dx, dy)
The function call doesn't have to be this way, I just need to be able to access all 4 x and y coordinates.
How would I go about passing the data into my function?
Edit: Just to clarify, the rows need to be passed in on a rolling basis, so I would like to call my_function(x1, y1, x2, y2, x3, y3, x4, y4) and then my_function(x2, y2, x3, y3, x4, y4, x5, y5) and so on...

You can set a custom grouper using np.arange(len(df))//4 to make groups of 4.
to get the function output back in the dataframe:
df.groupby(np.arange(len(df))//4).apply(lambda g: g.values.flatten())
output:
0 [280, 230, 230, 247, 219, 255, 209, 270]
1 [203, 290, 199, 313, 198, 336, 204, 365]
2 [208, 372, 220, 392, 253, 429, 281, 448]
dtype: object
to apply a function per group and get the result outside of the dataframe:
for name, g in df.groupby(np.arange(len(df))//4):
print(f'group: {name}')
print(g.values.flatten()) # replace print here with your function
output:
group: 0
[280 230 230 247 219 255 209 270]
group: 1
[203 290 199 313 198 336 204 365]
group: 2
[208 372 220 392 253 429 281 448]

Rolling over multiple columns in pandas is quite tricky, because it only supports single column aggregations. But you can roll over one column and slice the same rows out of multiple columns of you dataframe. Also rolling apply must return one single number. So, a rolling solution could be
def my_function(ax, ay, bx, by, cx, cy, dx, dy):
return ax + ay + bx + by + cx + cy + dx + dy
df['result'] = df.rolling(4, min_periods=4).x.apply(
lambda x: my_function(*df.loc[x.index, ['x','y']].to_numpy().ravel()))
df
Output
x y result
0 280 230 NaN
1 230 247 NaN
2 219 255 NaN
3 209 270 1940.0
4 203 290 1923.0
5 199 313 1958.0
6 198 336 2018.0
7 204 365 2108.0
8 208 372 2195.0
9 220 392 2295.0
10 253 429 2443.0
11 281 448 2603.0
Assuming you want to pass consecutive slices of rows to your function you can group every 4 rows and flatten the dataframe to get the shape you need.
def my_function(ax, ay, bx, by, cx, cy, dx, dy):
return f'ax={ax}, ay={ay}, bx={bx}, by={by}, cx={cx}, cy={cy}, dx={dx}, dy={dy}'
df.groupby(df.index//4).apply(lambda x: my_function(*x.to_numpy().ravel()))
Output
0 ax=280, ay=230, bx=230, by=247, cx=219, cy=255...
1 ax=203, ay=290, bx=199, by=313, cx=198, cy=336...
2 ax=208, ay=372, bx=220, by=392, cx=253, cy=429...
dtype: object
Dataframe used in this example
import pandas as pd
import io
t = '''
x y
280 230
230 247
219 255
209 270
203 290
199 313
198 336
204 365
208 372
220 392
253 429
281 448'''
df = pd.read_csv(io.StringIO(t), sep='\s+')
df
Output
x y
0 280 230
1 230 247
2 219 255
3 209 270
4 203 290
5 199 313
6 198 336
7 204 365
8 208 372
9 220 392
10 253 429
11 281 448

Get first element of list type data field

I have a dataframe with cells that contain lists.
Now I want to get every first element in the list, if there is one.
Like
416 []
417 []
419 [562]
425 [570, 632]
426 []
427 [511, 586, 753, 604]
428 []
And what I want is a something like this (which can be used i.e. to create a new row)
416 nan
417 nan
419 562
425 570
426 nan
427 511
428 nan
How can I do this? Or is this bad practice and should be avoided?

Use:
df['col'].str[0]
Example
df = pd.DataFrame({'col': {0: [],
1: [],
2: [562],
3: [570, 632],
4: [],
5: [511, 586, 753, 604],
6: []}})
print(df['col'].str[0])
0 NaN
1 NaN
2 562.0
3 570.0
4 NaN
5 511.0
6 NaN

Pyparsing two-dimensional list

I have the following sample data:
165 150 238 402 395 571 365 446 284 278 322 282 236
16 5 19 10 12 5 18 22 6 4 5
259 224 249 193 170 151 95 86 101 58 49
6013 7413 8976 10392 12678 9618 9054 8842 9387 11088 11393;
It is the equivalent of a two dimensional array (except each row does not have an equal amount of columns). At the end of each line is a space and then a \n except for the final entry which is followed by no space and only a ;.
Would anyone know the pyparsing grammer to parse this? I've been trying something along the following lines but it will not match.
data = Group(OneOrMore(Group(OneOrMore(Word(nums) + SPACE)) + LINE) + \
Group(OneOrMore(Word(nums) + SPACE)) + Word(nums) + Literal(";")
The desired output would ideally be as follows
[['165', '150', '238', '402', '395', '571', '365', '446', '284', '278',
'322', '282', '236'], ['16', '5', ... ], [...], ['6013', ..., '11393']]
Any assistance would be greatly appreciated.

You can use the stopOn argument to OneOrMore to make it stop matching. Then, since newlines are by default skippable whitespace, the next group can start matching, and it will just skip over the newline and start at the next integer.
import pyparsing as pp
data_line = pp.Group(pp.OneOrMore(pp.pyparsing_common.integer(), stopOn=pp.LineEnd()))
data_lines = pp.OneOrMore(data_line) + pp.Suppress(';')
Applying this to your sample data:
data = """\
165 150 238 402 395 571 365 446 284 278 322 282 236
16 5 19 10 12 5 18 22 6 4 5
259 224 249 193 170 151 95 86 101 58 49
6013 7413 8976 10392 12678 9618 9054 8842 9387 11088 11393;"""
parsed = data_lines.parseString(data)
from pprint import pprint
pprint(parsed.asList())
Prints:
[[165, 150, 238, 402, 395, 571, 365, 446, 284, 278, 322, 282, 236],
[16, 5, 19, 10, 12, 5, 18, 22, 6, 4, 5],
[259, 224, 249, 193, 170, 151, 95, 86, 101, 58, 49],
[6013, 7413, 8976, 10392, 12678, 9618, 9054, 8842, 9387, 11088, 11393]]

How to transform a 3d arrays into a dataframe in python

I have a 3d array as follows:
ThreeD_Arrays = np.random.randint(0, 1000, (5, 4, 3))
array([[[715, 226, 632],
[305, 97, 534],
[ 88, 592, 902],
[172, 932, 263]],
[[895, 837, 431],
[649, 717, 39],
[363, 121, 274],
[334, 359, 816]],
[[520, 692, 230],
[452, 816, 887],
[688, 509, 770],
[290, 856, 584]],
[[286, 358, 462],
[831, 26, 332],
[424, 178, 642],
[955, 42, 938]],
[[ 44, 119, 757],
[908, 937, 728],
[809, 28, 442],
[832, 220, 348]]])
Now I would like to have it into a DataFrame like this:
Add a Date column like indicated and the column names A, B, C.
How to do this transformation? Thanks!

Based on the answer to this question, we can use a MultiIndex. First, create the MultiIndex and a flattened DataFrame.
A = np.random.randint(0, 1000, (5, 4, 3))
names = ['x', 'y', 'z']
index = pd.MultiIndex.from_product([range(s)for s in A.shape], names=names)
df = pd.DataFrame({'A': A.flatten()}, index=index)['A']
Now we can reshape it however we like:
df = df.unstack(level='x').swaplevel().sort_index()
df.columns = ['A', 'B', 'C']
df.index.names = ['DATE', 'i']
This is the result:
A B C
DATE i
0 0 715 226 632
1 895 837 431
2 520 692 230
3 286 358 462
4 44 119 757
1 0 305 97 534
1 649 717 39
2 452 816 887
3 831 26 332
4 908 937 728
2 0 88 592 902
1 363 121 274
2 688 509 770
3 424 178 642
4 809 28 442
3 0 172 932 263
1 334 359 816
2 290 856 584
3 955 42 938
4 832 220 348

You could convert your 3D array to a Pandas Panel, then flatten it to a 2D DataFrame (using .to_frame()):
import numpy as np
import pandas as pd
np.random.seed(2016)
arr = np.random.randint(0, 1000, (5, 4, 3))
pan = pd.Panel(arr)
df = pan.swapaxes(0, 2).to_frame()
df.index = df.index.droplevel('minor')
df.index.name = 'Date'
df.index = df.index+1
df.columns = list('ABC')
yields
A B C
Date
1 875 702 266
1 940 180 971
1 254 649 353
1 824 677 745
...
4 675 488 939
4 382 238 225
4 923 926 633
4 664 639 616
4 770 274 378
Alternatively, you could reshape the array to shape (20, 3), form the DataFrame as usual, and then fix the index:
import numpy as np
import pandas as pd
np.random.seed(2016)
arr = np.random.randint(0, 1000, (5, 4, 3))
df = pd.DataFrame(arr.reshape(-1, 3), columns=list('ABC'))
df.index = np.repeat(np.arange(arr.shape[0]), arr.shape[1]) + 1
df.index.name = 'Date'
print(df)
yields the same result.

ThreeD_Arrays = np.random.randint(0, 1000, (5, 4, 3))
df = pd.DataFrame([list(l) for l in ThreeD_Arrays]).stack().apply(pd.Series).reset_index(1, drop=True)
df.index.name = 'Date'
df.columns = list('ABC')