I have been using pandas_datareader to retrieve stock prices from the IEX and Robinhood APIs for some time without any hitches. However, recently (early this week), my code started failing.
Here's the code and the error:
IEX API
import pandas_datareader.data as web
import datetime as dt
end = dt.datetime.today()
start = end - relativedelta(months=20)
data = web.DataReader('MSFT', 'iex', start, end)
And the error:
web.DataReader('MSFT', 'iex', start, end)
Traceback (most recent call last):
File "<ipython-input-88-2781d4515d5c>", line 1, in <module>
web.DataReader('MSFT', 'iex', start, end)
File "/anaconda3/lib/python3.7/site-packages/pandas_datareader/data.py", line 322, in DataReader
session=session).read()
File "/anaconda3/lib/python3.7/site-packages/pandas_datareader/iex/daily.py", line 91, in read
self._get_params(self.symbols))
File "/anaconda3/lib/python3.7/site-packages/pandas_datareader/base.py", line 84, in _read_one_data
out = self._read_url_as_StringIO(url, params=params)
File "/anaconda3/lib/python3.7/site-packages/pandas_datareader/base.py", line 95, in _read_url_as_StringIO
response = self._get_response(url, params=params)
File "/anaconda3/lib/python3.7/site-packages/pandas_datareader/base.py", line 155, in _get_response
raise RemoteDataError(msg)
RemoteDataError: Unable to read URL: https://api.iextrading.com/1.0/stock/market/batch?symbols=MSFT&types=chart&range=2y
Response Text:
b'Forbidden'
Robinhood API
import pandas_datareader.data as web
import datetime as dt
end = dt.datetime.today()
start = end - relativedelta(months=20)
data = web.DataReader('MSFT', 'robinhood', start, end)
And the error:
web.DataReader('MSFT', 'robinhood', start, end)
Traceback (most recent call last):
File "<ipython-input-90-0a0fc1b0d4ce>", line 1, in <module>
web.DataReader('MSFT', 'robinhood', start, end)
File "/anaconda3/lib/python3.7/site-packages/pandas_datareader/data.py", line 391, in DataReader
session=session).read()
File "/anaconda3/lib/python3.7/site-packages/pandas_datareader/base.py", line 77, in read
return self._read_one_data(self.url, self.params)
File "/anaconda3/lib/python3.7/site-packages/pandas_datareader/base.py", line 86, in _read_one_data
out = self._get_response(url, params=params).json()
File "/anaconda3/lib/python3.7/site-packages/pandas_datareader/base.py", line 136, in _get_response
last_response_text = response.text.encode(response.encoding)
TypeError: encode() argument 1 must be str, not None
The exact code was working fine until recently. I wonder what changed, and what I can do to correct it.
Most APIs listed on the pandas_datareader documentation have been deprecated and are no longer effective through the pandas_datareader framework (Yahoo, Google, etc).
However, Tiingo works just fine, at least for now.
To use Tiingo API, you'll first need to sign up for a free account (or paid if you want premium features). You can do that here. After activating your Tiingo account, you'll receive an API key which you can use to retrieve stock historical data with pandas_datareader (find details here).
From https://iextrading.com/developer/docs/#stocks
"On June 1, 2019, IEX Group removed all non-IEX data, and certain functionality, according to the schedule. IEX Cloud, a non-Exchange platform, will continue to provide access to third-party data sources."
Looks like you'll need access and secret keys to call their API. I'm looking into this for my python app stock_quote that hit this issue.
You can use the efficient Python library for Yahoo Finance, yfinance (GitHub link).
You can also find example codes here and here.
Related
I'm working on a project that needs to find the top wikipedia pages from today, and it uses the code found on the GitHub for pageviewapi, except it substitutes in today's date. The original code on the GitHub is this:
import pageviewapi
pageviewapi.top('fr.wikipedia', 2015, 11, 14, access='all-access')
My code looks like this:
from datetime import datetime
currentDay = datetime.now().day
currentMonth = datetime.now().month
currentYear = datetime.now().year
import pageviewapi
toppages = pageviewapi.top('fr.wikipedia', currentYear, currentMonth, currentDay, access='all-access')
print(toppages)
This just returns a long error (shown below). What is the issue with using this code, and why doesn't a result show up?
Error: Traceback (most recent call last):
File "main.py", line 54, in
toppages = pageviewapi.top('fr.wikipedia', currentYear, currentMonth, currentDay, access='all-access')
File "/home/runner/wikipedia-game/venv/lib/python3.8/site-packages/pageviewapi/client.py", line 94, in top
return api(TOP_ENDPOINT, args)
File "/home/runner/wikipedia-game/venv/lib/python3.8/site-packages/pageviewapi/client.py", line 154, in api
response.raise_for_status()
File "/home/runner/wikipedia-game/venv/lib/python3.8/site-packages/requests/models.py", line 960, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://wikimedia.org/api/rest_v1/metrics/pageviews/top/fr.wikipedia/all-access/2022/3/24
There are two problems - first you need to use 03 as the month, not 3 so use:
currentMonth = str(datetime.now().month).zfill(2)
Secondly, you can't view today's pageviews as they don't exist yet - the best you can do is use yesterday's, so:
currentDay = datetime.now().day-1
Thank you in advanced for any insights you can offer to my current challenge. I am running Python 3.9.1 and the latest versions of PANDAS and TIINGO libraries.
The problem is a reliable API call has started failing for certain tickers. I assume the problem is with the API provider but find it odd that no one else is reporting the problems.
My (simplified) code:
# Loads historical stock data into files
import datetime
#from pandas_datareader import data
import pandas as pd
import io
import os
#import requests
from tiingo import TiingoClient
os.environ["TIINGO_API_KEY"] = 'MY API KEY'
client = TiingoClient()
ticker = 'PG'
last_load_date = '2020-08-19'
today = '2021-02-12'
#idx = client.get_dataframe(ticker, frequency='daily', startDate=last_load_date, endDate=today)
##history = client.get_dataframe(['GOOGL',],
## frequency='daily',
## metric_name='adjClose',
## startDate='2020-01-01',
## endDate='2021-05-31')
prices = client.get_ticker_price(ticker,
fmt='json',
startDate=last_load_date,
endDate=today,
frequency='daily')
The failed output:
Traceback (most recent call last):
File "V:/woodstock/jobs/tiingo_test.py", line 22, in <module>
prices = client.get_ticker_price(ticker,
File "C:\Users\wood\AppData\Local\Programs\Python\Python39\lib\site-packages\tiingo\api.py", line 241, in get_ticker_price
return response.json()
File "C:\Users\wood\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\models.py", line 900, in json
return complexjson.loads(self.text, **kwargs)
File "C:\Users\wood\AppData\Local\Programs\Python\Python39\lib\json\__init__.py", line 346, in loads
return _default_decoder.decode(s)
File "C:\Users\wood\AppData\Local\Programs\Python\Python39\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\wood\AppData\Local\Programs\Python\Python39\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I have tried both the direct JSON method (above) and the leveraging dataframes (preferred but commented out above). This works for a ticker like GE but fails for PG, TSLA and others.
I have contacted Tiingo support but appreciate any insights from the community.
THANK YOU!
Tiingo support answered my question very quickly and professionally. As a "free" customer, I am incredibly impressed by their service.
The answer, for anyone else facing this is that Tiingo has a constraint on the number of calls per hour AND total number of unique symbols requested. The API is failing due to the fact that I have reached my 500 ticker limit and am requesting a ticker outside of that list. I will be upgrading my service to get around this issue.
I am having trouble using pandas-datareader to import data from Quandl. Here is the code that I have tried (with a real API key):
import pandas_datareader.data as pdr
from datetime import date
start=date(1970,1,1)
end=date.today()
ticker='F'
qkey=[My API Key]
pdr.QUANDL_API_KEY=qkey
QUANDL_API_KEY=qkey
pdrquandl=pdr.DataReader('WIKI/'+ticker,'quandl',start,end)
pdrquandl=pdr.DataReader('WIKI/'+ticker,'quandl',start,end,api_key=qkey)
I get the following error messages when I run this:
>>> pdrquandl=pdr.DataReader('WIKI/'+ticker,'quandl',start,end)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python37\lib\site-packages\pandas_datareader\data.py", line 379, in DataReader
session=session, api_key=access_key).read()
File "C:\Python37\lib\site-packages\pandas_datareader\quandl.py", line 54, in __init__
raise ValueError('The Quandl API key must be provided either '
ValueError: The Quandl API key must be provided either through the api_key variable or through the environmental variable QUANDL_API_KEY.
>>> pdrquandl=pdr.DataReader('WIKI/'+ticker,'quandl',start,end,api_key=qkey)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: DataReader() got an unexpected keyword argument 'api_key'
What am I missing? How do I provide the API key?
The source shows how the DataReader factory function passes it to the Quandl reader:
elif data_source == "quandl":
return QuandlReader(symbols=name, start=start, end=end,
retry_count=retry_count, pause=pause,
session=session, api_key=access_key).read()
So try passing it to DataReader with the access_key argument:
pdrquandl=pdr.DataReader('WIKI/'+ticker,'quandl',start,end,access_key=qkey)
A slightly different approach to quandl
import quandl
quandl.ApiConfig.api_key = "API-KEY"
aapl = quandl.get("WIKI/TICKER", start_date="YEAR-MONTH-DAY", end_date="YEAR-MONTH-DAY")
Configuration
MAC OS
VS Code
Python 3.x
Method
Run this code:
import pandas_datareader.data as pdr
api_key = [YOUR_API_KEY]
pdrquandl = pdr.DataReader("WIKI/TICKER", 'quandl', start_date, end_date, api_key)
I'd like to automate my manual trading strategies. However, for the beginning, I tried to reproduce Zipline's simple example of buying Apple stocks. I struggled running the algorithm with run_algorithm(). When I was trying to run the 'dual moving average cross', the exact same error came up. I also tried IPython and Terminal but still get that error. I couldn't find anything related to that either in this forum. I would very grateful for any hints. Thank you.
I'm using Python 3.6 on macOS and Zipline version 1.1.1.
That's the code:
import zipline as zl from zipline.api import order, record, symbol
def initialize(context):
pass
def handle_data(context, data):
order(symbol('AAPL'), 10)
record(AAPL=data.current(symbol('AAPL'), 'price'))
zl.run_algorithm(start='2015-1-1', end='2017-1-1', initialize=initialize, capital_base=10000)
That's the traceback:
Traceback (most recent call last): File
"/Users/SOL/Desktop/Python/backtest.py", line 13, in
zl.run_algorithm(start=2015-1-1, end=2017-1-1, initialize=initialize,
capital_base=10000) File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/zipline/utils/run_algo.py",
line 360, in run_algorithm environ=environ, File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/zipline/utils/run_algo.py",
line 132, in _run env = TradingEnvironment(asset_db_path=connstr,
environ=environ) File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/zipline/finance/trading.py",
line 99, in init self.bm_symbol, File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/zipline/data/loader.py",
line 173, in load_market_data environ, File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/zipline/data/loader.py",
line 287, in ensure_treasury_data if not has_data_for_dates(data,
first_date, last_date): File
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/zipline/data/loader.py",
line 87, in has_data_for_dates dts = series_or_df.index
AttributeError: 'NoneType' object has no attribute 'index'
FYI, I think the problem has to do with SSL certificates in python 3.6. Unfortunately the error message (especially if you ignore the stack trace) is not very helpful. For mac try:
sudo /Applications/Python\ 3.6/Install\ Certificatescommand
to reinstall the certificates
According to docs here, start and end are datetime objects, and not strings. So, You should define those as following:
from datetime import datetime
start = datetime(year=2015, month=1, day=1)
end = datetime(year=2017, month=1, day=1)
and then call
zl.run_algorithm(start=start, end=end, initialize=initialize, capital_base=10000)
so I have something very simple to ask but I can't get my head around it. I'm trying to import some stock data from yahoo finance, nothing fancy. I used to do this via:
import pandas.io.data as web
import pandas as pd
import datetime
start=datetime.datetime(2000,1,1)
end=datetime.date.today()
SLB=web.DataReader("SLB","yahoo",start,end)
Everything used to work lovely like this. However, I'm well aware of the changes that have taken place. The import method is now:
from pandas_datareader import data,wb
However, using this method, (and I'm still using the same start,end data etc), I receive:
SLB=wb.DataReader("SLB","yahoo",start,end)
Traceback (most recent call last):
File "<ipython-input-218-6633f366e2cb>", line 1, in <module>
SLB=wb.DataReader("SLB","yahoo",start,end)
AttributeError: module 'pandas_datareader.wb' has no attribute
'DataReader'
I've had a look online (obviously) for more information, and I've seen in some cases people have put:
SLB=data.DataReader('SLB','yahoo',start,end)
which leads to an even longer error altogether regarding issues connecting to the URL.
If anyone has come across this before, or can see errors in my code, It would be great if you could shed some light!
cheers
p.s.
the longer error message I got using:
SLB=data.DataReader("SLB","yahoo",start,end)
is:
SLB=data.DataReader("SLB","yahoo",start,end)
Traceback (most recent call last):
File "<ipython-input-226-0cd57c0d4cbe>", line 1, in <module>
SLB=data.DataReader("SLB","yahoo",start,end)
File "C:\Users\User\Anaconda3\lib\site-
packages\pandas_datareader\data.py", line 116, in DataReader
retry_count=retry_count, pause=pause,
File "C:\Users\User\Anaconda3\lib\site-
packages\pandas_datareader\yahoo\daily.py", line 76, in read
""" read one data from specified URL """
File "C:\Users\User\Anaconda3\lib\site-
packages\pandas_datareader\base.py", line 155, in read
if isinstance(self.symbols, (compat.string_types, int)):
File "C:\Users\User\Anaconda3\lib\site-
packages\pandas_datareader\base.py", line 74, in _read_one_data
out = self._read_url_as_StringIO(url, params=params)
File "C:\Users\User\Anaconda3\lib\site-
packages\pandas_datareader\base.py", line 85, in _read_url_as_StringIO
response = self._get_response(url, params=params)
File "C:\Users\User\Anaconda3\lib\site-
packages\pandas_datareader\base.py", line 120, in _get_response
raise RemoteDataError('Unable to read URL: {0}'.format(url))
RemoteDataError: Unable to read URL:
http://ichart.finance.yahoo.com/table.csv?
s=SLB&a=0&b=1&c=2000&d=5&e=16&f=2017&g=d&ignore=.csv
Hope this helps a little bit more.