We Keep Coding

How can I get specific time periods ohlcv in CCXT bitget?

Even if I type "since" keyword in ccxt bitget, always get only the latest information. The same code worked on ftx, what's the problem?
bitget = ccxt.bitget({'apiKey' : self.KEY,
'secret' : self.SECRET_KEY,
'enableRateLimit' : True,
'options' : {'defaultType':'swap'}
})
yyyymmdd = '20220301'
since = int(datetime(int(yyyymmdd[:4]),int(yyyymmdd[4:6]),int(yyyymmdd[6:])).timestamp()*1000)
ohlcv = bitget.fetch_ohlcv('BTC/USDT', '1m', since, limit = 1000)
ohlcv = pd.DataFrame(ohlcv)
ohlcv.columns = ['time','open','high','low','close','volume']
ohlcv['time'] = ohlcv['time'].apply(lambda x : datetime.fromtimestamp(x/1000).strftime('%Y%m%d %H:%M'))
time open high low close volume
0 20220322 14:36 42957.24 42959.97 42927.88 42927.88 1.8439
1 20220322 14:37 42927.88 42957.04 42927.88 42951.36 1.2933
2 20220322 14:38 42951.36 42951.36 42928.46 42932.59 0.6664
3 20220322 14:39 42932.59 42938.0 42916.22 42916.22 2.0336
4 20220322 14:40 42916.22 42916.22 42891.29 42897.49 2.0132
5 20220322 14:41 42897.49 42900.14 42880.96 42884.51 1.6279
6 20220322 14:42 42884.51 42893.26 42870.46 42870.46 2.3478
.
.
.
How can I get specific time period information ?

maybe since=since
also millisecond unix
since = int(datetime.datetime.strptime("2021-05-18 11:20:00+00:00", "%Y-%m-%d %H:%M:%S%z").timestamp() * 1000)
EDIT: looks like your unix was right my bad

builtin_function_or_method' object is not iterable

I have the following function to calculate the simple moving average for my data frame.
## data from
import yfinance as yf
symbols=[ 'SPY', 'TSLA', 'AAPL', 'CAKE', 'JBLU', 'MSFT']
data = yf.download(symbols, start="2015-01-01", end="2021-04-20")
def trend(df, fast, slow):
Dataset= pd.DataFrame(index=df.index)
Dataset['SlowSMA'] = df['Close'].rolling(slow).mean()
Dataset['FastSMA'] = df['Close'].rolling(fast).mean()
return Dataset
This works fine if I have one stock as there is only one close, however, it gives me the following issue
builtin_function_or_method' object is not iterable
I want to iterate in the data for each ticker in the data frame and save it in the Dataset by its name so I know which one is which.

I've modified a bit your code but this works :
import yfinance as yf
symbols=[ 'SPY', 'TSLA', 'AAPL', 'CAKE', 'JBLU', 'MSFT']
data = yf.download(symbols, start="2015-01-01", end="2021-04-20")
def stock_trend(df, fast, slow):
Dataset= pd.DataFrame(index=df.index)
Dataset['SlowSMA']= df.rolling(slow).mean().values
Dataset['FastSMA']= df.rolling(fast).mean().values
return Dataset
def all_stock_trends(df, fast, slow, stocks: list, value_type: str):
allst = []
for stock in stocks:
data = df[value_type][stock]
st = stock_trend(data, fast, slow)
allst.append(st)
all_st = pd.concat(allst, axis=1)
all_st.columns = pd.MultiIndex.from_product([stocks, ['SlowSMA', 'FastSMA']])
return all_st
Then you can do :
all_stocks_close = all_stock_trends(data, '1d', '1d', symbols, 'Close')
Which gives the following as output :
SPY TSLA AAPL CAKE JBLU MSFT
SlowSMA FastSMA SlowSMA FastSMA SlowSMA FastSMA SlowSMA FastSMA SlowSMA FastSMA SlowSMA FastSMA
Date
2014-12-31 205.539993 205.539993 44.481998 44.481998 27.594999 27.594999 50.310001 50.310001 15.860000 15.860000 46.450001 46.450001
2015-01-02 205.429993 205.429993 43.862000 43.862000 27.332500 27.332500 50.110001 50.110001 15.790000 15.790000 46.759998 46.759998
2015-01-05 201.720001 201.720001 42.018002 42.018002 26.562500 26.562500 50.200001 50.200001 15.220000 15.220000 46.330002 46.330002
2015-01-06 199.820007 199.820007 42.256001 42.256001 26.565001 26.565001 49.820000 49.820000 14.970000 14.970000 45.650002 45.650002
2015-01-07 202.309998 202.309998 42.189999 42.189999 26.937500 26.937500 51.700001 51.700001 15.100000 15.100000 46.230000 46.230000
... ... ... ... ... ... ... ... ... ... ... ... ...
2021-04-13 412.859985 412.859985 762.320007 762.320007 134.429993 134.429993 57.610001 57.610001 20.770000 20.770000 258.489990 258.489990
2021-04-14 411.450012 411.450012 732.229980 732.229980 132.029999 132.029999 58.200001 58.200001 20.830000 20.830000 255.589996 255.589996
2021-04-15 415.869995 415.869995 738.849976 738.849976 134.500000 134.500000 57.689999 57.689999 20.690001 20.690001 259.500000 259.500000
2021-04-16 417.260010 417.260010 739.780029 739.780029 134.160004 134.160004 57.840000 57.840000 20.299999 20.299999 260.739990 260.739990
2021-04-19 415.209991 415.209991 714.630005 714.630005 134.839996 134.839996 58.200001 58.200001 20.190001 20.190001 258.739990 258.739990
1585 rows × 12 columns
To access the data for one stock you can then simply do :
SPY_stock = all_stocks_close['SPY']

Concatenating data frames pandas

I would like to historical close prices from the yfinance module and create a data frame with a column with these closing prices for each of the tickers stored in the Holdings list. I can do everything except creating that data frame at the end. Can someone please help?:
Holdings = ['RL', 'AMC', 'BYND', 'BRK-B',
'BBY', 'AYX', 'AAPL', 'KO',
'FB', 'RACE', 'INTC', 'PFE',
'CRM', 'WFC', 'JPM', 'GOOG']
Hist_Holdings = []
for symbol in Holdings:
Ticker = yf.Ticker(symbol)
Hist = Ticker.history(period = "6mo", interval = "1d")
Hist = Hist['Close']
Hist.columns = [symbol]
Hist_Holdings.append(Hist)

The desired data frame format is not known, but the following code will concatenate the stocks you want to get with spaces. It is fast and returns the data in a data frame format. The code below specifies only the closing price.
import yfinance as yf
import datetime
now_ = datetime.datetime.today()
start = datetime.datetime(now_.year, now_.month - 6, now_.day + 1)
end = datetime.datetime(now_.year, now_.month, now_.day - 1)
Holdings = 'RL AMC BYND BRK-B BBY AYX AAPL KO FB RACE INTC PFE CRM WFC JPM GOOG'
data = yf.download(Holdings, start=start, end=end)['Close']
AAPL AMC AYX BBY BRK-B BYND CRM FB GOOG INTC JPM KO PFE RACE RL WFC
Date
2020-06-12 84.699997 5.89 141.130005 77.760002 181.210007 144.740005 175.110001 228.580002 1413.180054 59.330002 99.870003 45.599998 32.020874 167.889999 74.769997 27.969999
2020-06-15 85.747498 5.80 142.940002 80.010002 181.550003 154.000000 178.610001 232.500000 1419.849976 60.099998 101.250000 46.299999 31.650854 169.699997 73.739998 28.209999
2020-06-16 88.019997 5.56 145.690002 83.470001 182.300003 151.940002 180.479996 235.649994 1442.719971 60.400002 102.059998 46.770000 31.688805 169.690002 76.419998 28.520000
2020-06-17 87.897499 5.42 150.990005 83.239998 180.860001 156.339996 181.399994 235.529999 1451.119995 60.490002 99.480003 46.580002 31.840607 169.809998 74.480003 27.450001
2020-06-18 87.932503 5.63 160.779999 82.300003 180.729996 158.199997 187.660004 235.940002 1435.959961 60.080002 98.940002 46.990002 31.537003 168.580002 73.940002 27.549999

pandas_datareader.DataReader returns data for just one date

On using the pandas_datareader library to access the S&P 500 historical data, I manage to get the date of JUST the current date and not until the start date as mentioned below.
import pandas as pd
import pandas_datareader as web
import datetime
def test_run():
end_date = datetime.datetime.today()
start_date = datetime.date(end_date.year,2,28)
#DataReader method name is case sensitive
df=web.DataReader('^SPX','yahoo',start=start_date,end=end_date)
path_out = 'Data/'
df.to_csv(path_out+'SPY.csv')
if __name__ == "__main__":
test_run()
Date,High,Low,Open,Close,Volume,Adj Close
2020-03-13,2711.330078125,2492.3701171875,2569.989990234375,2711.02001953125,708668739,2711.02001953125
The output is in a csv file as:
enter image description here
I cant seem to figure out where the error lies.

The symbol/ticker is '^GSPC', not '^SPX'.
import pandas as pd
import datetime
import pandas_datareader.data as web
start=datetime.datetime(2019,3,13)
end=datetime.datetime(2020,3,13)
df=web.DataReader('^GSPC','yahoo',start,end)
print(df)
Result:
High Low ... Volume Adj Close
Date ...
2019-03-13 2821.239990 2799.780029 ... 3766150000 2810.919922
2019-03-14 2815.000000 2803.459961 ... 3469730000 2808.479980
2019-03-15 2830.729980 2810.790039 ... 5962730000 2822.479980
2019-03-18 2835.409912 2821.989990 ... 3552190000 2832.939941
2019-03-19 2852.419922 2823.270020 ... 3620220000 2832.570068
... ... ... ... ...
2020-03-09 2863.889893 2734.429932 ... 8423050000 2746.560059
2020-03-10 2882.590088 2734.000000 ... 7635960000 2882.229980
2020-03-11 2825.600098 2707.219971 ... 7374110000 2741.379883
2020-03-12 2660.949951 2478.860107 ... 8829380000 2480.639893
2020-03-13 2711.330078 2492.370117 ... 8258670000 2711.020020

Is there a proper way to produce a OHLCV pandas dataframe using ib api?

Here is the code that print out the data. However i don't see how to collect these data into a pandas dataframe. I used reqHistoricalData imported from ibapi (interactive broker) to request the data from TestApp class function which inherit EClient and EWrapper.
from ibapi.client import EClient
from ibapi.wrapper import EWrapper
from ibapi.contract import Contract
from ibapi.order import Order
from ibapi.ticktype import TickTypeEnum
import pandas as pd
import numpy as np
import os.path # To manage paths
import sys # To find out the script name (in argv[0])
from datetime import datetime
from time import sleep, strftime, localtime
from socket import error as SocketError
import errno
class TestApp(EWrapper, EClient):
def __init__(self):
EClient.__init__(self,self)
def error(self, reqId, errorCode, errorString):
print ('Error: ', reqId, errorCode, ' ', errorString)
def historicalData(self,reqId, bar):
print (bar.date, bar.open, bar.high, bar.low, bar.close, bar.volume)
def create_contract(symbol, sec_type, exch, prim_exch, curr):
contract = Contract()
contract.symbol = symbol
contract.secType = sec_type
contract.exchange = exch
contract.currency = curr
contract.primaryExchange = prim_exch
return contract
def create_order(order_type, quantity, action):
order = Order()
order.orderType = order_type
order.totalQuantity = quantity
order.action = action
return order
app = TestApp()
app.connect('127.0.0.1', 7497, 0)
contract = create_contract('AAPL', 'STK', 'SMART', 'NASDAQ', 'USD')
app.reqHistoricalData( reqId = 0,
contract = contract,
endDateTime = '',
durationStr = '1 Y',
barSizeSetting = '1 month',
whatToShow = 'TRADES',
useRTH = 1, # =1 for RTH data
formatDate = 1,
keepUpToDate = False,
chartOptions = []
)
app.run()
and the output is:
20181031 222.52 224.23 206.09 218.86 1752000
20181130 219.07 222.36 170.26 178.58 7249186
20181231 184.39 184.94 146.6 157.74 6851826
20190131 154.89 169.0 142.0 166.44 6383564
20190228 166.93 175.87 165.93 173.15 3478346
20190329 174.28 197.69 169.5 189.95 4956586
20190430 191.64 208.48 188.38 200.67 3812115
20190531 209.88 215.31 174.99 175.07 5642571
20190628 175.58 201.57 170.27 197.92 3592406
20190731 203.28 221.37 198.41 213.04 3418242
20190830 213.82 218.03 192.58 208.74 5078104
20190930 206.42 226.42 204.22 223.97 3768842
20191023 225.13 243.18 215.13 242.51 3253952
What i am looking for:
Open High Low Close Volume
Date
20181031 222.52 224.23 206.09 218.86 1752000
20181130 219.07 222.36 170.26 178.58 7249186
20181231 184.39 184.94 146.6 157.74 6851826
20190131 154.89 169.0 142.0 166.44 6383564
20190228 166.93 175.87 165.93 173.15 3478346
20190329 174.28 197.69 169.5 189.95 4956586
20190430 191.64 208.48 188.38 200.67 3812115
20190531 209.88 215.31 174.99 175.07 5642571
20190628 175.58 201.57 170.27 197.92 3592406
20190731 203.28 221.37 198.41 213.04 3418242
20190830 213.82 218.03 192.58 208.74 5078104
20190930 206.42 226.42 204.22 223.97 3768842
20191023 225.13 243.18 215.13 242.51 3253952

You could add a dataframe member to TestApp, and then add a row to it every time historicalData() is called:
...
self.cols = ['date', 'open', 'high', 'low', 'close', 'volume']
self.df = pd.DataFrame(columns=self.cols)
def historicalData(self, reqId, bar):
print (bar.date, bar.open, bar.high, bar.low, bar.close, bar.volume)
self.df.loc[len(self.df)] = [bar.date, bar.open, bar.high, bar.low, bar.close, bar.volume]
You would probably want to have a separate DataFrame for each reqId.

What I do is create a queue from the module queue.
So that goes something like...
def create_queue(self):
my_queue = queue.Queue()
self.my_hist_queue = my_queue
return my_queue
Then, when I define historicalData in the wrapper, I have it add it to the queue. It goes like...
def historicalData(self, reqId, bar):
print("HistoricalData. ", reqId,
"Date:", bar.date,
"Open:", bar.open,
"High:", bar.high,
"Low:", bar.low,
"Close:", bar.close,
"Volume:", bar.volume,
"Count:", bar.barCount,
"WAP:", bar.average)
self.my_hist_queue.put({'Request ID': reqId,
'Date': bar.date,
'Open': bar.open,
'High': bar.high,
'Low': bar.low,
'Close': bar.close,
'Volume': bar.volume,
'Count': bar.barCount,
'WAP': bar.average})
Then, it is relatively straight forward to iterate through the queue and put the historical data into a list of dictionaries. This way, pandas can easily convert it into a data frame. Here's how I do it...
def create_dataframe:
ticker_list = []
hist_storage = self.create_queue()
num_of_days = 5 #put however many days you want
data = self.reqHistoricalData(101, Contract, '', '{} D'.format(num_of_days), '1 day', "TRADES", 1, 1, False, [])
for i in range(number_of_days):
ticker_list.append(hist_storage.get())
df = pd.DataFrame(ticker_list)
print(df)
I hope this helps! Cheers!

For bigger data sets, use list of dicts. The bar object returning from the api can be translated into a dict, which can be appended to a list member of the api.
class IBApi(TestWrapper, TestClient): # the actual API 'app' = API Object we interact with when sending/receiving
def __init__(self):
TestWrapper.__init__(self) # requires the wrapper...
TestClient.__init__(self, wrapper=self)
###.....
self.histbars=[]
def historicalData(self, reqId:int, bar: BarData):
bardict={'reqid':reqId,'datetime':bar.date,'open':bar.open,'high':bar.high,'low':bar.low,'close':bar.close,'vol':bar.volume,'wap':bar.wap,'barcount':bar.barCount}
self.histbars.append(bardict)
This resulting list of dicts easily converts into a dataframe:
df =DataFrame.from_records(apclient.histbars)
df
reqid datetime open high low close vol wap barCount
0 1 20220519 09:30:00 191.50 193.80 189.60 191.58 16178 191.778 7890
1 1 20220519 09:45:00 191.64 194.30 190.21 192.98 12876 192.433 6090
2 1 20220519 10:00:00 192.97 194.74 191.88 192.33 12974 193.27 7835
3 1 20220519 10:15:00 192.39 193.75 191.77 192.79 8370 192.906 4372
4 1 20220519 10:30:00 192.71 194.07 191.29 191.69 7269 192.425 3774
5 1 20220519 10:45:00 191.72 193.19 191.01 192.26 6565 192.093 3167
6 1 20220519 11:00:00 192.24 193.99 191.80 193.44 6664 192.998 3023
7 1 20220519 11:15:00 193.37 193.85 192.58 192.97 5105 193.132 2278
8 1 20220519 11:30:00 193.04 194.99 192.99 194.63 5787 194.196 2703
9 1 20220519 11:45:00 194.60 194.97 193.90 194.80 7207 194.467 2949
10 1 20220519 12:00:00 194.82 195.29 194.50 194.67 4862 194.839 2169
11 1 20220519 12:15:00 194.66 195.15 194.04 194.59 5753 194.605 2638
12 1 20220519 12:30:00 194.61 194.75 192.92 192.92 3618 193.83 1723
13 1 20220519 12:45:00 192.90 193.39 192.38 193.20 3921 192.911 1739
14 1 20220519 13:00:00 193.20 193.32 191.69 191.98 3309 192.602 1844
15 1 20220519 13:15:00 191.97 192.28 191.45 191.98 4601 191.883 2407
16 1 20220519 13:30:00 192.00 192.62 191.55 192.02 4240 192.034 2031
...
(excuse the manual format)
To do: simple lookup dict from request id to ticker, and format the datetime to pandas-readable datetime.