We Keep Coding

I want to get grouped counts and percentages using pandas

table shows count and percentage count of cycle ride, grouped by membership type (casual, member).
What I have done in R:
big_frame %>%
group_by(member_casual) %>%
summarise(count = length(ride_id),
'%' = round((length(ride_id) / nrow(big_frame)) * 100, digit=2))
best I've come up with in Pandas, but I feel like there should be a better way:
member_casual_count = (
big_frame
.filter(['member_casual'])
.value_counts(normalize=True).mul(100).round(2)
.reset_index(name='percentage')
)
member_casual_count['count'] = (
big_frame
.filter(['member_casual'])
.value_counts()
.tolist()
)
member_casual_count
Thank you in advance

In R, you should be doing something like this:
big_frame %>%
count(member_casual) %>%
mutate(perc = n/sum(n))
In python, you can achieve the same like this:
(
big_frame
.groupby("member_casual")
.size()
.to_frame('ct')
.assign(n = lambda df: df.ct/df.ct.sum())
)

Is there an python process for this R code

i am working on python and i want to group by my data by to columns and at the same time add missing dates from a date1 corresponding to the occurrence of the event to another date2 corresponding to a date that a choose and fill the missing values into the columns i decided by forwarfill .
I tried the code bellow on r and its works i want to do the same in python
library(data.table)
library(padr)
library(dplyr)
data = fread("path", header = T)
data$ORDERDATE <- as.Date(data$ORDERDATE)
datemax = max(data$ORDERDATE)
data2 = data %>%
group_by(Column1, Column2) %>%
pad(.,group = c('Column1', 'Column2'), end_val = as.Date(datemax), interval = "day",break_above = 100000000000) %>%
tidyr::fill("Column3")
I search for the corresponding package library(padr) in python but couldn't find any.

Thank your for answering my request.
As required as an example i have this table:
users=['User1','User1','User2','User1','User2','User1','User2','User1','User2'],
products=['product1','product1','product1','product1','product1','product2','product2','product2','product2'],
quantities=[5,6,8,10,4,5,2,9,7],
prices=[2,2,5,5,6,6,6,7,7],
data = pd.DataFrame({'date':dates,'user':users,'product':products,'quantity':quantities,'price':prices}),
data['date'] = pd.to_datetime(data.date, format='%Y-%m-%d'),
data2=data.groupby(['user','product','date'],as_index=False).mean()```[enter image description here][1]
for User1 and product1 for exemple i want to input missing dates and fill the quantities column with the value 0 and the column price with backward values from a range of date that a choose.
And do the same by users and by product for remainings in my data.
the result should look like this:
[1]: https://i.stack.imgur.com/qOOda.png
the r code i used to generate the image is as follow:
```library(padr)
library(dplyr)
dates=c('2014-01-14','2014-01-14','2014-01-15','2014-01-19','2014-01-18','2014-01-25','2014-01-28','2014-02-05','2014-02-14')
users=c('User1','User1','User2','User1','User2','User1','User2','User1','User2')
products=c('product1','product1','product1','product1','product1','product2','product2','product2','product2')
quantities=c(5,6,8,10,4,5,2,9,7)
prices=c(2,2,5,5,6,6,6,7,7)
data=data.frame(date=c('2014-01-14','2014-01-14','2014-01-15','2014-01-19','2014-01-18','2014-01-25','2014-01-28','2014-02-05','2014-02-14'),user=c('User1','User1','User2','User1','User2','User1','User2','User1','User2'),product=c('product1','product1','product1','product1','product1','product2','product2','product2','product2'),quantity=c(5,6,8,10,4,5,2,9,7),price=c(2,2,5,5,6,6,6,7,7))
data$date <- as.Date(data$date)
datemax = max(data$date)
data2 = data %>% group_by(user, product) %>% pad(.,group = c('user', 'product'), end_val = as.Date(datemax), interval = "day",break_above = 100000000000)
data3=data2 %>% group_by(user,product,date) %>%
summarize(quantity=sum(quantity),price=mean(price))
data4=data3%>% tidyr::fill("price")%>% fill_by_value(quantity, value = 0)```

Translating a Python Pandas line to R:

I am following a blog post here and I am getting a little stuck on one part regarding the translation from Python pandas to R…
In the part of the blog:
Tick Bars
The author has the line:
data_tick_grp = data.reset_index().assign(grpId=lambda row: row.index // num_ticks_per_bar)
I understand that data is the "data frame" -
reset_index not sure what this is.
assing(grpId =…) - creating a new variable grpId
lambda row: - not sure what this does.
row.index - is this the same as row_number?
\\ - is this the same as floor() in R?
num_ticks_per_bar is calculated as.
total_ticks = len(data)
num_ticks_per_bar = total_ticks / num_time_bars
num_ticks_per_bar = round(num_ticks_per_bar, -3) # round to the nearest thousand
Which I understand it as:
ticks <- data %>%
filter(symbol == "XBTUSD") %>%
nrow()
ticks_per_bar <- ticks / 288
ticks_per_bar <- plyr::round_any(ticks_per_bar, 1000)
floor(1:nrow(data) / ticks_per_bar))
Can somebody help me translate the Python pandas line into R language?

Usually, Pandas best translates to base R:
reset_index same as resetting row.names for sequential numbering data.frame(..., row.names = NULL)
assign(grpId =…) same as assigning a column in place such as with transform, within or dplyr's mutate
lambda row this is required inside assign to reference data frame, here aliased as row
row.index is same as row number (remember Python is 0-index unlike R)
// is the integer division which in R one can be wrapped with as.integer or floor after division
Altogether, consider below adjustment to translate Pandas line:
data_tick_grp = (data.reset_index()
.assign(grpId=lambda row: row.index // num_ticks_per_bar)
)
To R:
data_tick_grp <- transform(data.frame(data, row.names = NULL),
grpId = floor(0:(nrow(data)-1) / num_ticks_per_bar))
Or in tidy format:
data_tick_grp <- data %>%
data.frame(row.names = NULL) %>%
mutate(grpId = floor(0:(nrow(data)-1) / num_ticks_per_bar))

Pandas groupby then assign

I have a dataframe in long format with columns: date, ticker, mcap, rank_mcap. The mcap columns is "marketcap" and measure how large a certain stock is, and mcap_rank is simply the ranked verson of it (where 1 is the largest marketcap).
I want to create a top 10 market cap weighted asset (e.g. S&P10). In R I do this
df %>%
filter(day(date) == 1, rank_mcap < 11) %>%
group_by(date) %>%
mutate(weight = mcap / sum(mcap)) %>%
ungroup() %>%
What do I do in pandas? I get the following error
AttributeError: Cannot access callable attribute 'assign' of 'DataFrameGroupBy' objects, try using the 'apply' method
when I tro do to a similar approach like the R method, namely in python do this:
df.\
query('included == True & date.dt.day == 1'). \
groupby('date').\
assign(w=df.mcap / df.mcap.sum())
I studied http://pandas.pydata.org/pandas-docs/stable/comparison_with_r.html and did not come to a conclusion.

How pandas achieve Mutate in R
df.query('included == True & date.dt.day == 1').\
assign(weight = lambda x : x.groupby('date',group_keys=False).
apply(lambda y: y.mcap / y.mcap.sum()))

You can do it in the same way as you did in R using datar:
from datar.all import f, filter, group_by, ungroup, mutate, sum
df >> \
filter(f.date.day == 1, f.rank_mcap < 11) >> \
group_by(f.date) >> \
mutate(weight = f.mcap / sum(f.mcap)) >> \
ungroup()
Disclaimer: I am the author of the datar package.

python pandas groupby complex calculations

I have dataframe df with columns: a, b, c,d. And i want to group data by a and make some calculations. I will provide code of this calculations in R. My main question is how to do the same in pandas?
library(dplyr)
df %>%
group_by(a) %>%
summarise(mean_b = mean(b),
qt95 = quantile(b, .95),
diff_b_c = max(b-c),
std_b_d = sd(b)-sd(d)) %>%
ungroup()
This example is synthetic, I just want to understand pandas syntaxis

I believe you need custom function with GroupBy.apply:
def f(x):
mean_b = x.b.mean()
qt95 = x.b.quantile(.95)
diff_b_c = (x.b - x.c).max()
std_b_d = x.b.std() - x.d.std()
cols = ['mean_b','qt95','diff_b_c','std_b_d']
return pd.Series([mean_b, qt95, diff_b_c, std_b_d], index=cols)
df1 = df.groupby('a').apply(f)