KeyError: 'begins_at' - python

TICKERS = ['^GSPC','^IXIC','^GDAXI', '^FTSE']
ind_data = pd.DataFrame()
for f in TICKERS:
ind_data [f] = wb.DataReader(f,data_source = 'robinhood',start = '1998-01-01') ['close_price'].values
KeyError
Traceback (most recent call last)
<ipython-input-813-4985fcf38db5> in <module>()
2 ind_data = pd.DataFrame()
3 for f in TICKERS:
----> 4 ind_data [f] = wb.DataReader(f,data_source = 'robinhood',start = '1998-01-01') ['close_price'].values
KeyError: 'begins_at'

I was using dash and I encountered the same problem.
this is a very helpful document of dataframe
https://media.readthedocs.org/pdf/pandas-datareader/latest/pandas-datareader.pdf
I still couldn't get stock price from robinhood to work, but I successfully displayed my chart with other API in my case, It was quandl.
Few things I got from reading the doc given above.
each broker offers different length of data. eg. robinhood only offers 1 years. ( this doesn't seem to be a problem tho. even if you asked for more than 1 year, i will just give you 1 year data)
Symbols used in each broker is different.
eg. morningstar (no longer available) used aapl for Apple stock. quandl used WIKI/AAPL.
I am sure that there are more exception like those I give above, but that's all I know for now because I only took a look at the doc less than 10 mins.

I was having the same problem with all the ticker symbols that contained dashes and caret.
Also if the ticker symbol is not supported by Robinhood, than you will get the same 'begins_at' massage.
Here is an example the ticker symbol IMIO is not supported by Robinhood and will give the same massage.
import pandas_datareader.data as web
web.DataReader("IMIO", 'robinhood')

Related

Downloading Shares Gives "JSONDecodeError: Expecting value: line 1 column 1 (char 0)"

I am downloading shared from Finance Yahoo. There is this error prompts:
JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I have checked all the similar questions and applied but all in vain, that's why asking this question again specifying my problem.
I am downloading shares details of the top 100 current stocks of Nasdaq using this repo, that is basically for the prediction of stock shares based on financial analysis. there is this stage of downloading shares and saving them as Pandas DataFrame. The code is:
shares = []
tickers_done = []
for ticker in tqdm(tickers):
if ticker in tickers_done:
continue
d = requests.get(f"https://query1.finance.yahoo.com/ws/fundamentals-timeseries/v1/finance/timeseries/{ticker}?symbol={ticker}&padTimeSeries=true&type=annualPreferredSharesNumber,annualOrdinarySharesNumber&merge=false&period1=0&period2=2013490868")
if not d.ok:
time.sleep(300)
d = requests.get(f"https://query1.finance.yahoo.com/ws/fundamentals-timeseries/v1/finance/timeseries/{ticker}?symbol={ticker}&padTimeSeries=true&type=annualPreferredSharesNumber,annualOrdinarySharesNumber&merge=false&period1=0&period2=2013490868")
ctn = d.json()['timeseries']['result']
dct = dict()
for n in ctn:
type = n['meta']['type'][0]
dct[type] = dict()
if type in n:
for o in n[type]:
if o is not None:
dct[type][o['asOfDate']] = o['reportedValue']['raw']
df = pd.DataFrame.from_dict(dct)
df['symbol'] = ticker
shares.append(df)
tickers_done.append(ticker)
time.sleep(1)
# save dataframe
df = pd.concat(shares)
df['date'] = df.index
df.to_csv(f"data/{project}/shares.csv", index=False)
And the error screen shot is:
And I have checked each ticker requests status_code as:
for x in tickers:
d = requests.get(f"https://query1.finance.yahoo.com/ws/fundamentals-timeseries/v1/finance/timeseries/{x}?symbol={x}&padTimeSeries=true&type=annualPreferredSharesNumber,annualOrdinarySharesNumber&merge=false&period1=0&period2=2013490868")
print(d.status_code)
The results are all 403. Your kind help will be highly appreciable. Thanks!
But searching the link https://query1.finance.yahoo.com/ws/fundamentals-timeseries/v1/finance/timeseries/AAPL?symbol=AAPL&padTimeSeries=true&type=annualPreferredSharesNumber,annualOrdinarySharesNumber&merge=false&period1=0&period2=2013490868in chrome by putting one of the tickers, like, AAPL, it gives some data, like,
The error
Per your error trace, the below line is throwing an error:
ctn = d.json()['timeseries']['result']
The error is trying to tell you that the data in d is not JSON-formatted:
JSONDecodeError: Expecting value: line 1 column 1 (char 0)
So basically the very first character (line 1, column 1) is not the expected { that starts JSON.
Add a print/breakpoint prior to the line above and see for yourself what data is in d (and whether or not it contains valid JSON).
The cause
As you point out towards the end:
The results are all 403. Your kind help will be highly appreciable. Thanks!
Since the requests.get calls are being rejected by the server, the responses don't contain valid JSON.
Find out why the server is rejecting your calls (403 suggests you don't have access to the requested resource); start with the most basic call that returns [any] data and then add bits until you either get a working query, or an error.

Cryptocurrency Price Tracker using Python

I'm trying to build a cryptocurrency price tracker in Python (see code below). I'm working with Python 3.10.1 in Visual Studio Code.
import pandas_datareader.data as web
import datetime as dt
currency = 'EUR'
metric = 'Close'
crypto = ['BTC','ETH']
colnames = []
first = True
start = dt.datetime(2020,1,1)
end = dt.datetime.now()
for ticker in crypto:
data = web.DataReader(f'{crypto}-{currency}', 'yahoo', start, end)
if first:
combined = data[[metric]].copy()
colnames.append(ticker)
combined.columns = colnames
first = False
else:
combined = combined.join(data[metric])
colnames.append(ticker)
combined.columns = colnames
When I execute this code, I get the following error notification:
RemoteDataError: No data fetched for symbol ['BTC', 'ETH']-EUR using YahooDailyReader
When I change the variable crypto to only pull the prices for BTC the code works, but the output looks like this:
Date
B
T
C
2020-01-01
6417.781738
6417.781738
6417.781738
2020-01-02
6252.938477
6252.938477
6252.938477
2020-01-03
6581.735840
6581.735840
6581.735840
In the scenario of only pulling BTC, the variable colnames looks like this: colnames = ['B','T', 'C']. I suspect, there's something wrong with that variable and it's potentially the reason why my code fails when I try to pull the data for multiple cryptocurrencies but I can't quite figure it out and solve my problem.

Python For Loop giving RANDOM errors on each run, IndexError or ValueError

I'm new to python. I put together this code to pull daily Options data from yfinance for multiple stock symbols for all possible expiration dates for each stock symbol (each symbol can have different expirations dates). So I created two For Loops, first loop picks the stock, second loop picks the expiration date for that selected symbol. The code looks like this
pip install yfinance --upgrade --no-cache-dir
import yfinance as yf
import pandas as pd
# List of tickers
tickers = ["AAPL","ABBV","ABT","ACN","ADBE","ADI","ADP","AEP","AGG","ALL","AMAT","AMD","AMGN","AMT","AMZN","APD","ARKF","ARKG","ARKK","ARKQ","ARKW","ARKX","XOP"]
# Loop to call put and call values for all expirations for all tickers in the list
put_call_combined = []
for X in tickers:
ticker = X
DateArray = yf.Ticker(ticker).options
for Y in DateArray:
strikeChoice = Y
opt = yf.Ticker(ticker).option_chain(strikeChoice)
calls = opt.calls
puts = opt.puts
put_call_combined.append([calls.lastTradeDate.max().date(),ticker,strikeChoice,puts['openInterest'].sum(),puts['volume'].sum(),calls['openInterest'].sum(),calls['volume'].sum()])
ArrayStore = None
#Final Output
df = pd.DataFrame(data=put_call_combined, columns=["dataset_day","ticker", "expiry","putOI","putVolume","callOI","callVolume"])
df
My problem is; on every run I'm getting random errors, when I look at the final DF, I can see the loop was broken at different symbols. Sometimes it fails due to an IndexError
IndexError Traceback (most recent call last)
<ipython-input-26-de7016fd3a37> in <module>
6
7 ticker = X
----> 8 DateArray = yf.Ticker(ticker).options
9
10 for X in DateArray:
~\anaconda3\lib\site-packages\yfinance\ticker.py in options(self)
193 def options(self):
194 if not self._expirations:
--> 195 self._download_options()
196 return tuple(self._expirations.keys())
~\anaconda3\lib\site-packages\yfinance\ticker.py in _download_options(self, date, proxy)
59 self._expirations[_datetime.datetime.utcfromtimestamp(
60 exp).strftime('%Y-%m-%d')] = exp
---> 61 return r['optionChain']['result'][0]['options'][0]
62 return {}
63
IndexError: list index out of range
And sometime it's a ValueError;
ValueError Traceback (most recent call last)
<ipython-input-25-0a07edf80d9a> in <module>
9 for x in DateArray:
10 strikeChoice = x
---> 11 opt = yf.Ticker(ticker).option_chain(strikeChoice)
12 calls = opt.calls
13 puts = opt.puts
~\anaconda3\lib\site-packages\yfinance\ticker.py in option_chain(self, date, proxy, tz)
92 self._download_options()
93 if date not in self._expirations:
---> 94 raise ValueError(
95 "Expiration `%s` cannot be found. "
96 "Available expiration are: [%s]" % (
ValueError: Expiration `2021-10-15` cannot be found. Available expiration are: [2021-09-17, 2022-01-21, 2023-01-20]
If I reduce the number of symbols, for instance 4 or 5 symbols at a time, I can get the output for all of these symbols, but when the list becomes too many, then the errors start kicking in RANDOMLY. Anyone has any idea why this might be happening? Am I pushing the limits of Yahoo Finance API?
Thanks
MT
I'm getting a similar issue when I try to download a large number (100 or more) of historical stock prices using yf.download. I thought I might try using a for loop with yf.Ticker to get the prices one at a time, and put a try-except block inside the for loop to deal with exceptions. Hopefully that will at least allow the for loop to complete and I'll get the rest of the data.

YFinance - tickerData.info not working for some stocks

import yfinance as yf
#define the ticker symbol
tickerSymbol = "AFT.NZ"
#get data on this ticker
tickerData = yf.Ticker(tickerSymbol)
print(tickerData.info)
This doesn't seem to work. IndexError: list index out of range
Replace "AFT.NZ" with "MSFT" or "FPH.NZ" and it works fine. Going to the Yahoo website, can't see why it wouldn't have data on it.
What's more confusing, is that replacing print(tickerData.info) with tickerDf = tickerData.history(period='max') does print some of the data.
I need the info because I want the full name of the company along with the currency the shares are traded in. Which is why just having the price data isn't the solution.
The AFT.NZ is just an example, most others on the NZX50 seem to have the same problem.
There's a merge request from williamsiuhang to fix this that's currently 9 days old.
https://github.com/ranaroussi/yfinance/pull/371/commits/7e137357296a1df177399d26543e889848efc021
I just made the change manually myself, go into base.py (in your_py_dir\Lib\site-packages\yfinance) and change line 286:
old line:
self._institutional_holders = holders[1]
new line:
self._institutional_holders = holders[1] if len(holders) > 1 else []
I've had the same problem, and see there's a lot of posts on github with the same error.
I've fixed the error with a try & except in the base.py file for yfinance
Line 282
# holders
try:
url = "{}/{}/holders".format(self._scrape_url, self.ticker)
holders = _pd.read_html(url)
self._major_holders = holders[0]
self._institutional_holders = holders[1]
if 'Date Reported' in self._institutional_holders:
self._institutional_holders['Date Reported'] = _pd.to_datetime(
self._institutional_holders['Date Reported'])
if '% Out' in self._institutional_holders:
self._institutional_holders['% Out'] = self._institutional_holders[
' % Out'].str.replace('%', '').astype(float)/100
except:
print("institutional_holders error")
Not a great solution, but makes it run for me. I'm not a great programmer, so I hope the problem get fixed in a more delicate way by the developer(s).

Alpha Vantage stockinfo only collects 4 dfs properly formatted, not 6

I can get 4 tickers of stockinfo from Alpha Vantage before the rest of the DataFrames are not getting the stockinfo I ask for. So my resulting concatenated df gets interpreted as Nonetype (because the 4 first dfs are formatted differently than the last 2). This is not my problem. The fact that I only get 4 of my requests is... If I can fix that - the resulting concatenated df will be intact.
My code
import pandas as pd
import datetime
import requests
from alpha_vantage.timeseries import TimeSeries
import time
tickers = []
def alvan_csv(stocklist):
api_key = 'demo' # For use with Alpha Vantage stock-info retrieval.
for ticker in stocklist:
#data=requests.get('https://www.alphavantage.co/query?function=TIME_SERIES_DAILY_ADJUSTED&symbol=%s&apikey={}'.format(api_key) %(ticker))
df = pd.read_csv('https://www.alphavantage.co/query?function=TIME_SERIES_DAILY_ADJUSTED&datatype=csv&symbol=%s&apikey={}'.format(api_key) %(ticker))#, index_col = 0) &outputsize=full
df['ticker'] = ticker
tickers.append(df)
# concatenate all the dfs
df = pd.concat(tickers)
print('\ndata before json parsing for === %s ===\n%s' %(ticker,df))
df['adj_close'] = df['adjusted_close']
del df['adjusted_close']
df['date'] = df['timestamp']
del df['timestamp']
df = df[['date','ticker','adj_close','volume','dividend_amount','split_coefficient','open','high','low']] #
df=df.sort_values(['ticker','date'], inplace=True)
time.sleep(20.3)
print('\ndata after col reshaping for === %s ===\n%s' %(ticker,df))
return df
if __name__ == '__main__':
stocklist = ['vws.co','nflx','mmm','abt','msft','aapl']
df = alvan_csv(stocklist)
NB. Please note that to use the Alpha Vantage API, you need a free API-Key which you may optain here: https://www.alphavantage.co/support/#api-key
Replace the demo API Key with your API Key to make this code work.
Any ideas as to get this to work?
Apparently Alpha Vantage has a pretty low fair usage allowance, where they measure no of queries pr. minute. So in effekt only the first 4 stocks are allowed at full speed. The rest of the stocks need to pause before downloading for not violating their fair-usage policy.
I have now introduced a pause between my stock-queries. At the moment I get approx 55% of my stocks, if I pause for 10 sec. between calls, and 100% if I pause for 15 seconds.
I will be testing exactly how low the pause can be set to allow for 100% of stocks to come through.
I must say compared to the super high-speed train we had at finance.yahoo.com, this strikes me as steam-train. Really really slow downloads. To get my 500 worth of tickers it takes me 2½ hours. But I guess beggars can't be choosers. This is a free service and I will manage with this.

Categories