We Keep Coding

how do I extract date string "Mar 11, 2019 • 3:26AM" from a paragraph and convert it to date time format (dd/mm/yy) in python

I have a paragraph that contains details like date and comments that I need to extract and make a separate column. The paragraph is in a column from which I am extracting the date is as follows:
'Story\nFAQ\nUpdates 2\nComments 35\nby Antaio Inc\nMar 11, 2019 • 3:26AM\n2 years ago\nThank you all for an amazing start!\nHi all,\nWe just want to thank you all for an awesome start! This is our first ever Indiegogo campaign and we are very grateful for your support that helped us achieve a successful campaign.\nIn the next little while, we will be dedicating our effort on production and shipping of the awesome A-Buds and A-Buds SE. We plan to ship them to you as promised in the coming month.\nWe will send out more updates as we are approaching the key production dates.\nStay tuned!\nBest regards,\nAntaio Team\nby Antaio Inc\nJan 31, 2019 • 5:15AM\nover 2 years ago\nPre-Production Update\nDear all,\nWe want to take this opportunity to thank all of you for being our early backers. You guys rock! :)\nAs you may have noticed, the A-Buds are already in production stage, which means we have already completed all development and testing, and are now working on pre-production. Not only will you receive fully tested and certified awesome A-Buds after the campaign, we are also giving you the promise to deliver them on time! We are truly excited to have these awesome true Bluetooth 5.0 earbuds in your hands. We are sure you will love them!\nSo here is a quick sneak peek:\nMore to come. Stay tuned! :)\nFrom: Antaio Team\nRead More'
This kind of paragraph is present in each row of the dataset in a particular column called 'Project_Updates_Description'. I am trying to extract the first date in each entry
The code I'm using so far is:
for i in df['Project_Updates_Description']:
if type(i) == str:
print(count)
word = i.split('\n',7)
count+=1
if len(word) > 5:
print(word[5])
df['Date'] = word[5]
The issue I have right now is that when I extract the date from the paragraph I'm getting it as string I need it as dd/mm/yyyy format I tried the methods like strptime it didn't work it is appending as string and when i try to append it in new 'Date' column I keep getting the same date for all entry. Could someone tell me were I am going wrong?

Assuming you have a dataframe with a column entitled 'Project_Updates_Description' which contains the example text and you want to extract the first date and generate a datetime stamp from this information you can do the following:
import pandas as pd
import numpy as np
def findDate(txin):
schptrn = '^\w+ \d{1,2}, \d{4,4}'
lines = txin.split('\n')
for line in lines:
#print(line)
data = re.findall(schptrn, line)[0]
if data:
#print(data)
return pd.to_datetime(data)
return np.nan
df['date'] = df.apply(lambda row: findDate(row['Project_Updates_Description']), axis = 1)

YFinance - tickerData.info not working for some stocks

import yfinance as yf
#define the ticker symbol
tickerSymbol = "AFT.NZ"
#get data on this ticker
tickerData = yf.Ticker(tickerSymbol)
print(tickerData.info)
This doesn't seem to work. IndexError: list index out of range
Replace "AFT.NZ" with "MSFT" or "FPH.NZ" and it works fine. Going to the Yahoo website, can't see why it wouldn't have data on it.
What's more confusing, is that replacing print(tickerData.info) with tickerDf = tickerData.history(period='max') does print some of the data.
I need the info because I want the full name of the company along with the currency the shares are traded in. Which is why just having the price data isn't the solution.
The AFT.NZ is just an example, most others on the NZX50 seem to have the same problem.

There's a merge request from williamsiuhang to fix this that's currently 9 days old.
https://github.com/ranaroussi/yfinance/pull/371/commits/7e137357296a1df177399d26543e889848efc021
I just made the change manually myself, go into base.py (in your_py_dir\Lib\site-packages\yfinance) and change line 286:
old line:
self._institutional_holders = holders[1]
new line:
self._institutional_holders = holders[1] if len(holders) > 1 else []

I've had the same problem, and see there's a lot of posts on github with the same error.
I've fixed the error with a try & except in the base.py file for yfinance
Line 282
# holders
try:
url = "{}/{}/holders".format(self._scrape_url, self.ticker)
holders = _pd.read_html(url)
self._major_holders = holders[0]
self._institutional_holders = holders[1]
if 'Date Reported' in self._institutional_holders:
self._institutional_holders['Date Reported'] = _pd.to_datetime(
self._institutional_holders['Date Reported'])
if '% Out' in self._institutional_holders:
self._institutional_holders['% Out'] = self._institutional_holders[
' % Out'].str.replace('%', '').astype(float)/100
except:
print("institutional_holders error")
Not a great solution, but makes it run for me. I'm not a great programmer, so I hope the problem get fixed in a more delicate way by the developer(s).

dataframe.empty gives "True" result, which should be "False" imo

With the help of Pandas I try to search for an ID in a column in a CSV file. I want the code to check if the dataframe is empty.
When I try the standalone code in Python3 it works fine. However, incorporated in the main code, the value that is returned is always "True". This is the strangest thing I have encountered so far.
def lookupID(ID):
ID_df = df[df['ID'] == ID]
ID_test = ID_df.empty
return ID_test
df = pd.read_csv(mainFile, usecols=range(0,19))
for container in containers:
ID = container.a["data-id"]
if not lookupID(ID):
code
else:
code
I hope someone knows what the hell is up with this one.
Some example rows with the headers included:
ID,merk,soort,prijs,bouwjaar,km-stand,transmissie,brandstof,inhoud,vermogen,plaatsnaam,distance,datum_begin,datum,prijs_oud,carroserie,kleur,titel,URL
124471366,BMW,2 Serie,19900,2015,25089,Handgeschakeld,Benzine,1499,100,Nieuwerkerk Ad Ijssel (ZH),116240,08/07/2019,08/07/2019,19900,MPV,Grijs,Active Tourer 218i High Executive Sport Line
124538604,BMW,1-serie,8900,2006,66914,Automaat,Benzine,1995,110,Heerlen (LI),134876,08/07/2019,08/07/2019,8900,Hatchback,Blauw,120i Anniversary / Automaat / Navigatie
124538344,BMW,3-serie,2900,2012,108390,Automaat,Diesel,1995,121,Haarlem (NH),131073,08/07/2019,08/07/2019,2900,Sedan,Grijs,320D Sedan High Exe-LEER-FINANCIAL LEASE VA 269| - PMND

Downloading data from yahoo finance with a dash or dot in the ticker

I have this code to download data from yahoo:
#gets data from yahoo finance
stocks = list(newmerge.index)
start = dt.datetime(2012,1,1)
end = dt.datetime.today()
yahoodata = pdr.get_data_yahoo(stocks,start,end)
cleanData = yahoodata.loc['Adj Close']
dataFrame = pd.DataFrame(cleanData, columns=stocks)
It works fine but I noticed a problem recently, it doesn't download data for stocks "BRK.B" , and "BR.B" .
I have a list of all the stocks called "stocks" , and here's what I've done, but it still doesn't show data for stocks w/ dot in them:
def stocksdot(stocks):
stocks_dash = str(stocks).replace('.','-')
stockslist = stocks_dash.split(',')
return stockslist
stocksdot(stocks)
My expected output would be to download all stocks, even those with a dot in them. Any ideas how to circumvent?

Your problem is Yahoo Finance doesn't use the "." notation to track shares of different classes. So, "BRK.B" and "BR.B" are actually "BRKB" and "BRB".
Using the Yahoo Finance python SDK I made a little script to test whether or not Yahoo Finance could find information about a stock with the ticker "BRK.B" or "BR.B".
from yahoo_finance import Share
stock = Share('BRK.B')
print(stock.get_price())
This results is:
>>>> None
Stock tickers with a dot in them are used as a shorthand for a type or class of a specific stock. You can learn more here.
To circumvent it looks like you can remove the ".". For example when I use "BRKB" instead of "BRK.B" I get the result:
>>>> 173.05
Which is the current price of Berkshire Hathaway class B stock.
To replace the "." programatically use Python's .replace() method.
for stock in stocks:
stock = stock.replace(".", "") # Replaces all "." with "" in the string
# stock

Your problem is Yahoo Finance doesn't use the "." notation to track shares of different classes. So, "BRK.B" and "BR.B" are actually "BRKB" and "BRB". --- My comment: Now "BRK.B" and "BR.B" are actually "BRK-B" and "BR-B".

Is there a way to automatically get general info of many stocks like P/E ratio, Yield, and so on?

I know some ways to get daily stock prices and volumes in R or python, but just wondering whether these is a way (using either R or python) to get more info about stocks such as P/E ratio, company website, Yield and so on, preferably not just current value, but also historical values.
Thanks.

Historical is going to be difficult. The quantmod package for R has getQuote which together with yahooQF will be all you need to get current values.
require("quantmod")
getQuote("GS", what = yahooQF(c("Market Capitalization", "Earnings/Share",
"P/E Ratio", "Book Value", "EBITDA", "52-week Range")))
Trade Time Market Capitalization Earnings/Share P/E Ratio Book Value EBITDA 52-week Range
GS 2012-06-21 04:00:00 47.870B 6.764 14.27 134.476 0 84.27 - 139.25
Also, try
getQuote("GS", what=yahooQF())
which will give you a menu of choices for what fields to request.
You can get recent financial statements from Google Finance with getFinancials
There is also the FinancialInstrument package which has several update_instruments.* functions to download metadata about instruments (stocks in this case). For example, here's what the yahoo one does
require("FinancialInstrument")
stock("GS", currency("USD")) # define the stock
#[1] "GS"
update_instruments.yahoo("GS") #update with yahoo
#[1] "GS"
getInstrument("GS")
#primary_id :"GS"
#currency :"USD"
#multiplier :1
#tick_size :0.01
#identifiers : list()
#type :"stock"
#name :"Goldman Sachs Gro"
#exchange :"NYSE"
#market.cap :"47.870B"
#avg.volume :5480530
#EPS :6.76
#EPS.current.year.est:11.4
#EPS.next.year.est :12.9
#book.value :134
#EBITDA :0
#range.52wk :"84.27 - 139.25"
#defined.by :"yahoo"
#updated : POSIXct, format: "2012-06-21 19:31:11"
If you have an InteractiveBrokers account, you can use the outstanding IBrokers package to get lots of info about lots of instruments. Also, if you have an IB account you'll want to look at my twsInstrument package which has a lot of convenience functions.

Just to answer the website part of my question:
str <- paste("http://investing.money.msn.com/investments/company-report?symbol=", ticker, sep = "")
page <- paste(readLines(url(str, open = "rt")), collapse = "\n")
match <- regexpr("Website", page, perl = TRUE)
if (attr(match, "match.length") > 0) {
site <- substring(page, attr(match, "capture.start"), attr(match, "capture.start") + attr(match, "capture.length") - 1)
site <- strsplit(site, "/")[[1]][1]
}