We Keep Coding

How to display the average price of bitcoin in Pandas and not through a loop?

y = []
n = 0
days = 1
for i in btc['Adj Close']:
averagePrice = (i + n) / days
n += i
days += 1
y.append(averagePrice)
btc['TopAverage'] = y

If btc is a pandas data frame (as it appears so), then:
btc.loc[:, 'Days'] = list(range(1, btc.shape[0] + 1))
btc.loc[:, 'n'] = btc['Adj Close'].cumsum().shift(periods=1, fill_value=0.)
btc.loc[:, 'TopAverage'] = (btc['Adj Close'] + btc['n']) / btc['Days']
reflects the logic in your code. This will add the columns 'Days' and 'n' to the data frame as well.

expectation and variance of future stock price under binary tree

Probably a over-simplified model for stock price: on each day, the price will go up by a factor 1.05 with probability 0.6 or will go down to 1/1.05 with probability 0.4. So this is a non-symmetrical binary tree. How can I analytically calculate the expectation and variance of this stock price on future date, say day 100. Also, is there any module in python to handle binary tree model like this? appreciate code to implement this.
Best regards

import random as r
s = 100 # starting value
^^Initial conditions. Simulating one day on the stock market:
def day(stock_value): #One day in the stock market
k = r.uniform(0,1)
if k < 0.6:
output = 1.05*stock_value
else:
output = stock_value/1.05
return(output)
Simulating 100 days on the stock market:
for j in range(100): #simulates 100 days in the stock market
s = day(s)
print(s)
Simulating 100 days 1000 times:
data = []
for i in range(1000):
s = [100]
for j in range(100):
s.append(day(s[j]))
data.append(s)
Converting the data to only consider the last day:
def mnnm(mat): #Makes an mxn matrix into an nxm matrix
out = []
for j in range(len(mat[0])):
out.append([])
for j in range(len(mat[0])):
for m in range(len(mat)):
out[j].append(mat[m][j])
return(out)
data = mnnm(data)
data = data[-1]
Taking a mean average:
def lst_avg(lst): #Returns the average of a list
output = 0
for j in range(len(lst)):
output+= lst[j]/len(lst)
return(output)
mean = lst_avg(data)
Variance:
import numpy as np
for h in range(len(data)):
data[h] = data[h]**2
mean_square = lst_avg(data)
variance = np.fabs(mean_square - mean**2)

The theoretical value after 1 day is (assuming value on day 0 is A)
A * 0.6 * 1.05 + 100 * 0.4/1.05
And after 100 days it's
A * (0.603 + 0.380952...)**100 so...
(in the following I use 1 as stock price on day 0.)
p1 = 0.6
p2 = 0.4
x1 = 1.05
x2 = 1/1.05
initial_value = 1
no_of_days = 100
# 1 day
expected_value_after_1_day = initial_value * ( p1*x1 + p2*x2)
print (expected_value_after_1_day, 'is the expected value of price after 1 day')
ex_squared_value_1_day = initial_value * (p1*x1**2 + p2*x2**2)
# variance can be calculated as follows
variance_day_1 = ex_squared_value_1_day - expected_value_after_1_day**2
# or an alternative calculation, summing the squares of the differences from the mean
alt_variance_day_1 = p1 * (x1 - expected_value_after_1_day) ** 2 + p2 * (x2 - expected_value_after_1_day) ** 2
print ('Variance after one day is', variance_day_1)
# 100 days
expected_value_n_days = initial_value * (p1*x1 + p2*x2) ** no_of_days
ex_squared_value_n_days = initial_value * (p1*x1**2 + p2*x2**2) ** no_of_days
ex_value_n_days_squared = expected_value_n_days ** 2
variance_n_days = ex_squared_value_n_days - ex_value_n_days_squared
print(expected_value_n_days, 'is the expected value of price after {} days'.format(no_of_days))
print(ex_squared_value_n_days, 'is the expected value of the square of the price after {} days'.format(no_of_days))
print(ex_value_n_days_squared, 'is the square of the expected value of the price after {} days'.format(no_of_days))
print(variance_n_days, 'is the variance after {} days'.format(no_of_days))
It probably looks a bit old-school, hope you don't mind!
Output
1.0109523809523808 is the expected value of price after 1 day
Variance after one day is 0.0022870748299321786
2.972144657651404 is the expected value of price after 100 days
11.046309656223373 is the expected value of the square of the price after 100 days
8.833643866005783 is the square of the expected value of the price after 100 days
2.2126657902175904 is the variance after 100 days

How to add/subract operation between two different times using timedelta?

So i'm trying know in what time of day the man will end running. When in last line of code, trying to use an operator addition this error is popping out. What i'm doing wrong?
TypeError: unsupported operand type(s) for +: 'datetime.time' and 'datetime.timedelta'
import datetime
# first run, 1 mile
pace_minutes1 = 8
pace_seconds1 = 15
distance = 1 #1 mile
b = ((pace_minutes1 * 60) + pace_seconds1) * distance
print(b)
# second run, 3 mile
pace_minutes2 = 7
pace_seconds2 = 12
distance = 3 # 3 miles
a = ((pace_minutes2 * 60) + pace_seconds2) * distance
print(a)
# third run, 1 mile run
pace_minutes3 = 8
pace_seconds3 = 15
distance = 1 # 1 mile
c = ((pace_minutes3 * 60) + pace_seconds3) * distance
print(c)
x = (a + b + c)
print(x, "seconds")
time = datetime.timedelta(seconds=x) # converse value "x" seconds to time (h,m,s)
print("All running time will gonna take",time,"seconds") # total time spent running
start_time = datetime.time(6,52,0) # time from where he started running
print("The running started in",start_time)
end_time = (start_time + time) # time when he ended running
print(end_time)

You can only add the timedelta to a datetime.datetime object, not a datetime.time object (probably in order to avoid overflows).
So simple fix, use start_time = datetime.datetime(2021, 04, 21, 6,52,0)

Please find below answer .check if it what you desired.May be you can check stack overflow answer
import datetime
# first run, 1 mile
pace_minutes1 = 8
pace_seconds1 = 15
distance = 1 #1 mile
b = ((pace_minutes1 * 60) + pace_seconds1) * distance
print(b)
# second run, 3 mile
pace_minutes2 = 7
pace_seconds2 = 12
distance = 3 # 3 miles
a = ((pace_minutes2 * 60) + pace_seconds2) * distance
print(a)
# third run, 1 mile run
pace_minutes3 = 8
pace_seconds3 = 15
distance = 1 # 1 mile
c = ((pace_minutes3 * 60) + pace_seconds3) * distance
print(c)
x = (a + b + c)
print(x, "seconds")
time = datetime.timedelta(seconds=x) # converse value "x" seconds to time (h,m,s)
print("All running time will gonna take",time,"seconds") # total time spent running
start_time = datetime.time(6,52,0) # time from where he started running
# print(type(start_time))
# print(type(time))
print("The running started in",start_time)
end_time = (dt.datetime.combine(dt.date(1,1,1),start_time) + time).time()
print(end_time)

Speeding Up Datafile Reading Program For School Project

I am in a lower-level coding class (Python) and have a major project due in three days. One of our grading criteria is program speed. My program runs in about 30 seconds, ideally it would execute in 15 or less. Here is my code:
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from datetime import datetime
import time
start_time = time.time()#for printing execution time
#function for appending any number of files to a dataframe
def read_data_files(pre, start, end): #reading in the data
data = pd.DataFrame()#dataframe with all the data from files
x = start
while x <= end:
filename = pre + str(x) + ".csv" #string manipulation
dpath = pd.read_csv("C:\\Users\\jacks\\Downloads\\BMEN 207 Project 1
Data\\" + filename )
for y in dpath:
dpath = dpath.rename(columns = {y: y})
data = data.append(dpath)
x += 1
return data
data = read_data_files("Data_", 5, 163) #start, end, prefix...
#converting to human time and adding to new column in dataframe
human_time = []
for i in data[' Time']:
i = int(i)
j = datetime.utcfromtimestamp(i).strftime('%Y-%m-%d %H:%M:%S')
human_time.append(j)
human_timen = np.array(human_time) #had issues here for some reason, so i
created another array to replace the time column in the dataframe
data[' Time'] = human_timen
hours = [] #for use as x-axis in plot
stdlist = [] #for use as y-axis in plot
histlist = [] #for storing magnitudes of most active hour
def magfind(row): #separate function to calculate the magnitude of each row in
each dataframe
return (row[' Acc X'] ** 2 + row[' Acc Y'] ** 2 + row[' Acc Z'] ** 2) ** .5
def filterfunction(intro1, intro2, first, last): #two different intros to deal
with the issue of '00:' versus '10:' timestamps
k = first
meanmax = 0
active = 0
while k <= last:
if 0 <= k < 7: #data from hours 0 to 6, none after
hr = intro1 + str(k) + ':'
tfilter = data[' Time'].str.contains(hr)
acc = data.loc[tfilter, [' Acc X', ' Acc Y', ' Acc Z']]
acc['magnitude'] = acc.apply(magfind, axis = 1) #creates magnitude
column using prior function, column has magnitudes for every row of every file
p = acc.loc[:, 'magnitude'].std()#finds std dev for the column and
appends to a list for graphing
m = acc.loc[:, 'magnitude'].mean()
stdlist.append(p)
elif k == 12 or 20 < k <= last: #data at 12 and beyond hour 20
hr = intro2 + str(k) + ":"
tfilter = data[' Time'].str.contains(hr)
acc = data.loc[tfilter, [' Acc X', ' Acc Y', ' Acc Z']]
acc['magnitude'] = acc.apply(magfind, axis = 1)
p = acc.loc[:, 'magnitude'].std()
m = acc.loc[:, 'magnitude'].mean()
stdlist.append(p)
else: #in the case that we are dealing with an hour that has no data
p = 0
m = 0
stdlist.append(p) #need this so that the hours with no data still
get graphed
if m > meanmax: # for determining which hour was the most active, and
appending those magnitudes to a list for histogramming
meanmax = m
active = k #most active hour
for i in acc['magnitude']:
histlist.append(i) #adding all the magnitudes for histogramming
hours.append(k)
k += 1
print("mean magnitude", meanmax)
print("most active hour", active)
return hours, stdlist, histlist
filterfunction(' 0', ' ', 0, 23)
The slow speed stems from the "filterfunction" function. What this program does is read data from over 100 files, and this function specifically sorts the data into a dataframe and analyzes by time (each individual hour) in order to calculate the data in all rows for that hour. I believe that it might be able to be sped up by changing up the way that the data is filtered to search by hour, but am not sure. The reason I have statements to dis-include certain k-values is that there are hours with no data to manipulate, which would mess up the list of standard deviation calculations as well as the plot that this data will father. Any tips or ideas for speeding this up would be greatly appreciated!

One suggestion to speed it up a bit is to remove this line since it is not being used anywhere in the program:
import matplotlib.pyplot as plt
matplotlib is a big library so removing it should improve performance.
Also I think you could get rid of numpy since it is used once only...consider using a tuple

I could not able to test because I am on mobile now. However my main idea is not making the code better or leen. I changed the functioning part of the process.
Integrated the 'multiprocessing' library(method) into your code and also calculated the system cpu cores and divide all processes between them.
Multiprocessing library detailed documentation: https://docs.python.org/2/library/multiprocessing.html
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import time
import psutil
from datetime import datetime
from multiprocessing import Pool
cores = psutil.cpu_count()
start_time = time.time()#for printing execution time
#function for appending any number of files to a dataframe
def read_data_files(pre, start, end): #reading in the data
data = pd.DataFrame()#dataframe with all the data from files
x = start
while x <= end:
filename = pre + str(x) + ".csv" #string manipulation
dpath = pd.read_csv("C:\\Users\\jacks\\Downloads\\BMEN 207 Project 1
Data\\" + filename )
for y in dpath:
dpath = dpath.rename(columns = {y: y})
data = data.append(dpath)
x += 1
return data
data = read_data_files("Data_", 5, 163) #start, end, prefix...
#converting to human time and adding to new column in dataframe
human_time = []
for i in data[' Time']:
i = int(i)
j = datetime.utcfromtimestamp(i).strftime('%Y-%m-%d %H:%M:%S')
human_time.append(j)
human_timen = np.array(human_time) #had issues here for some reason, so i
created another array to replace the time column in the dataframe
data[' Time'] = human_timen
hours = [] #for use as x-axis in plot
stdlist = [] #for use as y-axis in plot
histlist = [] #for storing magnitudes of most active hour
def magfind(row): #separate function to calculate the magnitude of each row in
each dataframe
return (row[' Acc X'] ** 2 + row[' Acc Y'] ** 2 + row[' Acc Z'] ** 2) ** .5
def filterfunction(intro1, intro2, first, last): #two different intros to deal
with the issue of '00:' versus '10:' timestamps
k = first
meanmax = 0
active = 0
while k <= last:
if 0 <= k < 7: #data from hours 0 to 6, none after
hr = intro1 + str(k) + ':'
tfilter = data[' Time'].str.contains(hr)
acc = data.loc[tfilter, [' Acc X', ' Acc Y', ' Acc Z']]
acc['magnitude'] = acc.apply(magfind, axis = 1) #creates magnitude
column using prior function, column has magnitudes for every row of every file
p = acc.loc[:, 'magnitude'].std()#finds std dev for the column and
appends to a list for graphing
m = acc.loc[:, 'magnitude'].mean()
stdlist.append(p)
elif k == 12 or 20 < k <= last: #data at 12 and beyond hour 20
hr = intro2 + str(k) + ":"
tfilter = data[' Time'].str.contains(hr)
acc = data.loc[tfilter, [' Acc X', ' Acc Y', ' Acc Z']]
acc['magnitude'] = acc.apply(magfind, axis = 1)
p = acc.loc[:, 'magnitude'].std()
m = acc.loc[:, 'magnitude'].mean()
stdlist.append(p)
else: #in the case that we are dealing with an hour that has no data
p = 0
m = 0
stdlist.append(p) #need this so that the hours with no data still
get graphed
if m > meanmax: # for determining which hour was the most active, and
appending those magnitudes to a list for histogramming
meanmax = m
active = k #most active hour
for i in acc['magnitude']:
histlist.append(i) #adding all the magnitudes for histogramming
hours.append(k)
k += 1
print("mean magnitude", meanmax)
print("most active hour", active)
return hours, stdlist, histlist
# Run this with a pool of 5 agents having a chunksize of 3 until finished
agents = cores
chunksize = (len(data) / cores)
with Pool(processes=agents) as pool:
pool.map(filterfunction, (' 0', ' ', 0, 23))

don't use apply, it's not vectorized. instead, use vectorized operations whenever you can. in this case, instead of doing df.apply(magfind, 1), do:
def add_magnitude(df):
df['magnitude'] = (df[' Acc X'] ** 2 + df[' Acc Y'] ** 2 + df[' Acc Z'] ** 2) ** .5

Conditionally setting values with df.loc inside a loop

I'm querying an MS Access db to retrieve a set of leases. My task is to calculate monthly totals for base rent for the next 60 months. The leases have dates related to start and end in order to calculate the correct periods in the event a lease terminates prior to 60 periods. My current challenge comes in when I attempt to increase the base rent by a certain amount whenever it's time to increment for that specific lease. I'm at a beginner level with Python/pandas so my approach is likely not optimum and the code rough looking. It's likely a vectorized approach is better suited however i'm not quite able to execute such code yet.
Data:
Lease input & output
Code:
try:
sql = 'SELECT * FROM [tbl_Leases]'
#sql = 'SELECT * FROM [Copy Of tbl_Leases]'
df = pd.read_sql(sql, conn)
#print df
#df.to_csv('lease_output.csv', index_label='IndexNo')
df_fcst_periods = pd.DataFrame()
# init increments
periods = 0
i = 0
# create empty lists to store looped info from original df
fcst_months = []
fcst_lease_num = []
fcst_base_rent = []
fcst_method = []
fcst_payment_int = []
fcst_rate_inc_amt = []
fcst_rate_inc_int = []
fcst_rent_start = []
# create array for period deltas, rent interval calc, pmt interval calc
fcst_period_delta = []
fcst_rate_int_bool = []
fcst_pmt_int_bool = []
for row in df.itertuples():
# get min of forecast period or lease ending date
min_period = min(fcst_periods, df.Lease_End_Date[i])
# count periods to loop for future periods in new df_fcst
periods = (min_period.year - currentMonth.year) * 12 + (min_period.month - currentMonth.month)
for period in range(periods):
nextMonth = (currentMonth + monthdelta(period))
period_delta = (nextMonth.year - df.Rent_Start_Date[i].year) * 12 + (nextMonth.month - df.Rent_Start_Date[i].month)
period_delta = float(period_delta)
# period delta values allow us to divide by the payment & rent intervals looking for integers
rate_int_calc = period_delta/df['Rate_Increase_Interval'][i]
pmt_int_calc = period_delta/df['Payment_Interval'][i]
# float.is_integer() method - returns bool
rate_int_bool = rate_int_calc.is_integer()
pmt_int_bool = pmt_int_calc.is_integer()
# conditional logic to handle base rent increases
if df['Forecast_Method'][i] == "Percentage" and rate_int_bool:
rate_increase = df['Base_Rent'][i] * (1 + df['Rate_Increase_Amt'][i]/100)
df.loc[df.index, "Base_Rent"] = rate_increase
fcst_base_rent.append(df['Base_Rent'][i])
print "Both True"
else:
fcst_base_rent.append(df['Base_Rent'][i])
print rate_int_bool
fcst_rate_int_bool.append(rate_int_bool)
fcst_pmt_int_bool.append(pmt_int_bool)
fcst_months.append(nextMonth)
fcst_period_delta.append(period_delta)
fcst_rent_start.append(df['Rent_Start_Date'][i])
fcst_lease_num.append(df['Lease_Number'][i])
#fcst_base_rent.append(df['Base_Rent'][i])
fcst_method.append(df['Forecast_Method'][i])
fcst_payment_int.append(df['Payment_Interval'][i])
fcst_rate_inc_amt.append(df['Rate_Increase_Amt'][i])
fcst_rate_inc_int.append(df['Rate_Increase_Interval'][i])
i += 1
df_fcst_periods['Month'] = fcst_months
df_fcst_periods['Rent_Start_Date'] = fcst_rent_start
df_fcst_periods['Lease_Number'] = fcst_lease_num
df_fcst_periods['Base_Rent'] = fcst_base_rent
df_fcst_periods['Forecast_Method'] = fcst_method
df_fcst_periods['Payment_Interval'] = fcst_payment_int
df_fcst_periods['Rate_Increase_Amt'] = fcst_rate_inc_amt
df_fcst_periods['Rate_Increase_Interval'] = fcst_rate_inc_int
df_fcst_periods['Period_Delta'] = fcst_period_delta
df_fcst_periods['Rate_Increase_Interval_bool'] = fcst_rate_int_bool
df_fcst_periods['Payment_Interval_bool'] = fcst_pmt_int_bool
except Exception, e:
print str(e)
conn.close()

I ended up initializing a variable before the periods loop which allowed me to perform a calculation when looping to obtain the correct base rents for subsequent periods.
# init base rent, rate increase amount, new rate for leases
base_rent = df['Base_Rent'][i]
rate_inc_amt = float(df['Rate_Increase_Amt'][i])
new_rate = 0
for period in range(periods):
nextMonth = (currentMonth + monthdelta(period))
period_delta = (nextMonth.year - df.Rent_Start_Date[i].year) * 12 + (nextMonth.month - df.Rent_Start_Date[i].month)
period_delta = float(period_delta)
# period delta values allow us to divide by the payment & rent intervals looking for integers
rate_int_calc = period_delta/df['Rate_Increase_Interval'][i]
pmt_int_calc = period_delta/df['Payment_Interval'][i]
# float.is_integer() method - returns bool
rate_int_bool = rate_int_calc.is_integer()
pmt_int_bool = pmt_int_calc.is_integer()
# conditional logic to handle base rent increases
if df['Forecast_Method'][i] == "Percentage" and rate_int_bool:
new_rate = base_rent * (1 + rate_inc_amt/100)
base_rent = new_rate
fcst_base_rent.append(new_rate)
elif df['Forecast_Method'][i] == "Manual" and rate_int_bool:
new_rate = base_rent + rate_inc_amt
base_rent = new_rate
fcst_base_rent.append(new_rate)
else:
fcst_base_rent.append(base_rent)
Still open for any alternative approaches though!