We Keep Coding

Downloading Shares Gives "JSONDecodeError: Expecting value: line 1 column 1 (char 0)"

I am downloading shared from Finance Yahoo. There is this error prompts:
JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I have checked all the similar questions and applied but all in vain, that's why asking this question again specifying my problem.
I am downloading shares details of the top 100 current stocks of Nasdaq using this repo, that is basically for the prediction of stock shares based on financial analysis. there is this stage of downloading shares and saving them as Pandas DataFrame. The code is:
shares = []
tickers_done = []
for ticker in tqdm(tickers):
if ticker in tickers_done:
continue
d = requests.get(f"https://query1.finance.yahoo.com/ws/fundamentals-timeseries/v1/finance/timeseries/{ticker}?symbol={ticker}&padTimeSeries=true&type=annualPreferredSharesNumber,annualOrdinarySharesNumber&merge=false&period1=0&period2=2013490868")
if not d.ok:
time.sleep(300)
d = requests.get(f"https://query1.finance.yahoo.com/ws/fundamentals-timeseries/v1/finance/timeseries/{ticker}?symbol={ticker}&padTimeSeries=true&type=annualPreferredSharesNumber,annualOrdinarySharesNumber&merge=false&period1=0&period2=2013490868")
ctn = d.json()['timeseries']['result']
dct = dict()
for n in ctn:
type = n['meta']['type'][0]
dct[type] = dict()
if type in n:
for o in n[type]:
if o is not None:
dct[type][o['asOfDate']] = o['reportedValue']['raw']
df = pd.DataFrame.from_dict(dct)
df['symbol'] = ticker
shares.append(df)
tickers_done.append(ticker)
time.sleep(1)
# save dataframe
df = pd.concat(shares)
df['date'] = df.index
df.to_csv(f"data/{project}/shares.csv", index=False)
And the error screen shot is:
And I have checked each ticker requests status_code as:
for x in tickers:
d = requests.get(f"https://query1.finance.yahoo.com/ws/fundamentals-timeseries/v1/finance/timeseries/{x}?symbol={x}&padTimeSeries=true&type=annualPreferredSharesNumber,annualOrdinarySharesNumber&merge=false&period1=0&period2=2013490868")
print(d.status_code)
The results are all 403. Your kind help will be highly appreciable. Thanks!
But searching the link https://query1.finance.yahoo.com/ws/fundamentals-timeseries/v1/finance/timeseries/AAPL?symbol=AAPL&padTimeSeries=true&type=annualPreferredSharesNumber,annualOrdinarySharesNumber&merge=false&period1=0&period2=2013490868in chrome by putting one of the tickers, like, AAPL, it gives some data, like,

The error
Per your error trace, the below line is throwing an error:
ctn = d.json()['timeseries']['result']
The error is trying to tell you that the data in d is not JSON-formatted:
JSONDecodeError: Expecting value: line 1 column 1 (char 0)
So basically the very first character (line 1, column 1) is not the expected { that starts JSON.
Add a print/breakpoint prior to the line above and see for yourself what data is in d (and whether or not it contains valid JSON).
The cause
As you point out towards the end:
The results are all 403. Your kind help will be highly appreciable. Thanks!
Since the requests.get calls are being rejected by the server, the responses don't contain valid JSON.
Find out why the server is rejecting your calls (403 suggests you don't have access to the requested resource); start with the most basic call that returns [any] data and then add bits until you either get a working query, or an error.

how do I extract date string "Mar 11, 2019 • 3:26AM" from a paragraph and convert it to date time format (dd/mm/yy) in python

I have a paragraph that contains details like date and comments that I need to extract and make a separate column. The paragraph is in a column from which I am extracting the date is as follows:
'Story\nFAQ\nUpdates 2\nComments 35\nby Antaio Inc\nMar 11, 2019 • 3:26AM\n2 years ago\nThank you all for an amazing start!\nHi all,\nWe just want to thank you all for an awesome start! This is our first ever Indiegogo campaign and we are very grateful for your support that helped us achieve a successful campaign.\nIn the next little while, we will be dedicating our effort on production and shipping of the awesome A-Buds and A-Buds SE. We plan to ship them to you as promised in the coming month.\nWe will send out more updates as we are approaching the key production dates.\nStay tuned!\nBest regards,\nAntaio Team\nby Antaio Inc\nJan 31, 2019 • 5:15AM\nover 2 years ago\nPre-Production Update\nDear all,\nWe want to take this opportunity to thank all of you for being our early backers. You guys rock! :)\nAs you may have noticed, the A-Buds are already in production stage, which means we have already completed all development and testing, and are now working on pre-production. Not only will you receive fully tested and certified awesome A-Buds after the campaign, we are also giving you the promise to deliver them on time! We are truly excited to have these awesome true Bluetooth 5.0 earbuds in your hands. We are sure you will love them!\nSo here is a quick sneak peek:\nMore to come. Stay tuned! :)\nFrom: Antaio Team\nRead More'
This kind of paragraph is present in each row of the dataset in a particular column called 'Project_Updates_Description'. I am trying to extract the first date in each entry
The code I'm using so far is:
for i in df['Project_Updates_Description']:
if type(i) == str:
print(count)
word = i.split('\n',7)
count+=1
if len(word) > 5:
print(word[5])
df['Date'] = word[5]
The issue I have right now is that when I extract the date from the paragraph I'm getting it as string I need it as dd/mm/yyyy format I tried the methods like strptime it didn't work it is appending as string and when i try to append it in new 'Date' column I keep getting the same date for all entry. Could someone tell me were I am going wrong?

Assuming you have a dataframe with a column entitled 'Project_Updates_Description' which contains the example text and you want to extract the first date and generate a datetime stamp from this information you can do the following:
import pandas as pd
import numpy as np
def findDate(txin):
schptrn = '^\w+ \d{1,2}, \d{4,4}'
lines = txin.split('\n')
for line in lines:
#print(line)
data = re.findall(schptrn, line)[0]
if data:
#print(data)
return pd.to_datetime(data)
return np.nan
df['date'] = df.apply(lambda row: findDate(row['Project_Updates_Description']), axis = 1)

How to match asset price data from a csv file to another csv file with relevant news by date

I am researching the impact of news article sentiment related to a financial instrument and its potenatial effect on its instruments's price. I have tried to get the timestamp of each news item, truncate it to minute data (ie remove second and microsecond components) and get the base shareprice of an instrument at that time, and at several itervals after that time, in our case t+2. However, program created twoM to the file, but does not return any calculated price changes
Previously, I used Reuters Eikon and its functions to conduct the research, described in the article below.
https://developers.refinitiv.com/article/introduction-news-sentiment-analysis-eikon-data-apis-python-example
However, instead of using data available from Eikon, I would like to use my own csv news file with my own price data from another csv file. I am trying to match the
excel_file = 'C:\\Users\\Artur\\PycharmProjects\\JRA\\sentimenteikonexcel.xlsx'
df = pd.read_excel(excel_file)
sentiment = df.Sentiment
print(sentiment)
start = df['GMT'].min().replace(hour=0,minute=0,second=0,microsecond=0).strftime('%Y/%m/%d')
end = df['GMT'].max().replace(hour=0,minute=0,second=0,microsecond=0).strftime('%Y/%m/%d')
spot_data = 'C:\\Users\\Artur\\Desktop\\stocksss.csv'
spot_price_10 = pd.read_csv(spot_data)
print(spot_price_10)
df['twoM'] = np.nan
for idx, newsDate in enumerate(df['GMT'].values):
sTime = df['GMT'][idx]
sTime = sTime.replace(second=0, microsecond=0)
try:
t0 = spot_price_10.iloc[spot_price_10.index.get_loc(sTime),2]
df['twoM'][idx] = ((spot_price_10.iloc[spot_price_10.index.get_loc((sTime + datetime.timedelta(minutes=10))),3]/(t0)-1)*100)
except:
pass
print(df)
However, the programm is not able to return the twoM price change values

I assume that you got a warning because you are trying to make changes on views. As soon as you have 2 [] (one for the column, one for the row) you can only read. You must use loc or iloc to write a value:
...
try:
t0 = spot_price_10.iloc[spot_price_10.index.get_loc(sTime),2]
df.loc[idx,'twoM'] = ((spot_price_10.iloc[spot_price_10.index.get_loc((sTime + datetime.timedelta(minutes=10))),3]/(t0)-1)*100)
except:
pass
...

dataframe.empty gives "True" result, which should be "False" imo

With the help of Pandas I try to search for an ID in a column in a CSV file. I want the code to check if the dataframe is empty.
When I try the standalone code in Python3 it works fine. However, incorporated in the main code, the value that is returned is always "True". This is the strangest thing I have encountered so far.
def lookupID(ID):
ID_df = df[df['ID'] == ID]
ID_test = ID_df.empty
return ID_test
df = pd.read_csv(mainFile, usecols=range(0,19))
for container in containers:
ID = container.a["data-id"]
if not lookupID(ID):
code
else:
code
I hope someone knows what the hell is up with this one.
Some example rows with the headers included:
ID,merk,soort,prijs,bouwjaar,km-stand,transmissie,brandstof,inhoud,vermogen,plaatsnaam,distance,datum_begin,datum,prijs_oud,carroserie,kleur,titel,URL
124471366,BMW,2 Serie,19900,2015,25089,Handgeschakeld,Benzine,1499,100,Nieuwerkerk Ad Ijssel (ZH),116240,08/07/2019,08/07/2019,19900,MPV,Grijs,Active Tourer 218i High Executive Sport Line
124538604,BMW,1-serie,8900,2006,66914,Automaat,Benzine,1995,110,Heerlen (LI),134876,08/07/2019,08/07/2019,8900,Hatchback,Blauw,120i Anniversary / Automaat / Navigatie
124538344,BMW,3-serie,2900,2012,108390,Automaat,Diesel,1995,121,Haarlem (NH),131073,08/07/2019,08/07/2019,2900,Sedan,Grijs,320D Sedan High Exe-LEER-FINANCIAL LEASE VA 269| - PMND

How to see what get_fundamentals is adding to universe

I have a simple python code with searches the market for companies with specific financial criteria.
Code:
num_stocks = 2
fundamental_df = get_fundamentals(
query(
fundamentals.valuation.market_cap,
fundamentals.income_statement.net_income
)
.filter(fundamentals.income_statement.net_income > 1)
.order_by(fundamentals.valuation.market_cap)
.desc())
.limit(num_stocks)
)
update_universe(fundamental_df.columns.values)
context.stocks = [stock for stock in fundamental_df]
for stock in context.stocks:
print stock
Im want to see if it is working/what companies it is querying.
I tried having my algo print what whats inside context.stocks but its not work.
Could someone help me out,
Thanks
Mike