I am trying to extract data from Bloomberg.
I need data for multiple fields with diferent currencies.
I can't get what I need from this answer https://github.com/alpha-xone/xbbg
Can anyone help, please?. for a "specifc time" and on a "period of time"?
I tried the following code but it didn't work.
blp.bdh(
tickers='TPXDDVD Index,SCTOGAA LN Equity,VAPEJSI ID Equity', flds=['PX_LAST', 'FUND_NET_ASSET_VAL', 'FUND_TOTAL_ASSETS'],
start_date='2018-10-01', end_date='2019-11-01', FX=['JPY','GBp','USD']
)
It is best to group similar securities together when pulling in historic data. In the OP's question, 'TPXDDVD Index' is a calculated total return index. Hence it will not have the same fields available as the other two, which as their tickers suggest are quoted funds.
Taking just the two quoted funds, SCTOGAA LN Equity and VAPEJSI ID Equity, we can determine the default currency in which each field is denominated. This being Bloomberg, naming conventions are organic and may not be obvious! The two value fields are FUND_NET_ASSET_VAL and FUND_TOTAL_ASSETS, and the default currency might be different for each.
We can use the bdp() function to pull back these currencies as follows (NB tickers are in a python list:
from xbbg import blp
tickers = ['SCTOGAA LN Equity','VAPEJSI ID Equity']
df = blp.bdp(tickers,['NAV_CRNCY','FUND_TOTAL_ASSETS_CRNCY'])
print(df)
With the result:
nav_crncy fund_total_assets_crncy
SCTOGAA LN Equity GBp GBP
VAPEJSI ID Equity USD EUR
So for VAPEJSI the NAV and Total Assets are denominated in different currencies. And NB. GBp is not a typo by the data entry clerk, it means GBP pence.
You can override the currency with a single currency value applied to all fields in the function call.
from xbbg import blp
fields = ['FUND_NET_ASSET_VAL','FUND_TOTAL_ASSETS']
df = blp.bdh('SCTOGAA LN Equity',fields,
start_date='2018-10-01', end_date='2019-11-01')
print(df.tail(2))
df = blp.bdh('SCTOGAA LN Equity',fields,
start_date='2018-10-01', end_date='2019-11-01', Currency='USD')
print(df.tail(2))
which returns:
SCTOGAA LN Equity
FUND_NET_ASSET_VAL FUND_TOTAL_ASSETS
2019-10-31 70.81 1755.65
2019-11-01 70.99 1756.51
SCTOGAA LN Equity
FUND_NET_ASSET_VAL FUND_TOTAL_ASSETS
2019-10-31 0.91607 2271.28527
2019-11-01 0.91840 2272.40325
The asset value in GBP pence, has been converted to USD dollars. As an aside if you had put Currency='USd' instead you would have the price in US cents. You have to love case-sensitivity ...
Related
I have a dataframe (df1) that looks like this;
title
score
id
timestamp
Stock_name
Biocryst ($BCRX) continues to remain undervalued
120
mfuz84
2021-01-28 21:32:10
...and then continues with 44000 something more rows. I have another dataframe (df2) that looks like this;
Company name
Symbol
BioCryst Pharmaceuticals, Inc.
BCRX
GameStop
GME
Apple Inc.
AAPL
...containing all nasdaq and NYSE listed stocks. What I want to do now however, is to add the symbol of the stock to the column "Stock_name" in df1. In order to do this, I want to match the df1[title] with the df2[Symbol] and then based on what symbol has a match in the title, add the corresponding stock name (df2[Company name]) to the df1[Stock_name] column. If there is more than one stock name in the title, I want to use the first one mentioned.
Is there any easy way to do this?
I tried with this little dataset and it's working, let me know if you have some problems
df1 = pd.DataFrame({"title" : ["Biocryst ($BCRX) continues to remain undervalued", "AAPL is good, buy it"], 'score' : [120,420] , 'Stock_name' : ["",""] })
df2 = pd.DataFrame({'Company name' : ['BioCryst Pharmaceuticals, Inc.','GameStop','Apple Inc.'], 'Symbol' : ["BCRX","GME","AAPL"]})
df1
title score Stock_name
0 Biocryst ($BCRX) continues to remain undervalued 120
1 AAPL is good, buy it 420
df2
Company name Symbol
0 BioCryst Pharmaceuticals, Inc. BCRX
1 GameStop GME
2 Apple Inc. AAPL
for j in range(0,len(df2)):
for i in range(0,len(df1)):
if df2['Symbol'][j] in df1['title'][i]:
df1['Stock_name'][i] = df2['Symbol'][j]
df1
title score Stock_name
0 Biocryst ($BCRX) continues to remain undervalued 120 BCRX
1 AAPL is good, buy it 420 AAPL
First, I think you should create a dictionary based on df2.
symbol_lookup = dict(zip(df2['Symbol'],df2['Company name']))
Then you need a function that will parse the title column. If you can rely on stock symbols being preceded by a dollar sign, you can use the following:
def find_name(input_string):
for symbol in input_string.split('$'):
#if the first four characters form
#a stock symbol, return the name
if symbol_lookup.get(symbol[:4]):
return symbol_lookup.get(symbol[:4])
#otherwise check the first three characters
if symbol_lookup.get(symbol[:3]):
return symbol_lookup.get(symbol[:3])
You could also write a function based on expecting the symbols to be in parentheses. If you can't rely on either, it would be more complicated.
Finally, you can apply your function to the title column:
df1['Stock_name'] = df1['title'].apply(find_name)
In my use of the InvestPy package, I am able to get stock ticker data easily, using the in-built function 'get_stock_historical_data'.
But not having the same luck in trying to get Index data of Nifty50, for example. A quick look at all the Indian tickers available from the function <investpy.get_stocks(country='india')> reveals nothing related to the Index.
Is their a way to get it using the package? My alternative is web-scraping stuff for index. Couldn't find any thing relevant in the official documentation here.
There are a few methods that investpy has implemented to gather information regarding indices. Unfortunately, I could not find any function that returns the performances of each individual member of the index, but you can, for example, get the historical data regarding an index:
df = investpy.get_index_historical_data(index="Nifty 50",
country="India",
from_date='01/01/2018',
to_date='01/01/2019')
Open High Low Close Volume Currency
Date
2018-01-01 10531.70 10537.85 10423.10 10435.55 134532000 INR
2018-01-02 10477.55 10495.20 10404.65 10442.20 158092000 INR
2018-01-03 10482.65 10503.60 10429.55 10443.20 172516992 INR
2018-01-04 10469.40 10513.00 10441.45 10504.80 180256992 INR
2018-01-05 10534.25 10566.10 10520.10 10558.85 186470000 INR
... ... ... ... ... ... ...
2018-12-26 10635.45 10747.50 10534.55 10729.85 271943 INR
2018-12-27 10817.90 10834.20 10764.45 10779.80 470160 INR
2018-12-28 10820.95 10893.60 10817.15 10859.90 253087 INR
2018-12-31 10913.20 10923.55 10853.20 10862.55 186495 INR
2019-01-01 10881.70 10923.60 10807.10 10910.10 159405 INR
[247 rows x 6 columns]
More on Index Data Retrieval is found in the documentation here.
You can also retrieve a list of all stocks in a certain country:
df = investpy.get_stocks_list(country="india") # returns 'list'
df = investpy.get_stocks(country="india") # returns 'pandas.DataFrame' (shown below)
country name ... currency symbol
0 india Aditya Birla Capital ... INR ADTB
1 india Hubtown ... INR HUBT
2 india 3i Infotech ... INR TIIN
3 india 3M India ... INR TMIN
4 india ABB India ... INR ABB
... ... ... ... ... ...
1706 india G K P Printing ... INR GKPP
1707 india Vivid Mercantile ... INR VIVD
1708 india White Organic ... INR WHIE
1709 india Parshva Enterprises ... INR PAHV
1710 india SK International Export ... INR SKIN
[1711 rows x 6 columns]
Because investpy doesn't offer something to get all stocks from a certain index, you will have to sort through them yourself. Fortunately, investpy does give you functions that allow you to get the recent, historical, etc. data about a specific index (mentioned above). If you are looking for data on each of those stocks, you could:
Gather symbols of each stock in the Nifty 50 index as a list
Set up a loop that gets the historical data for each stock
Perform operations on data such as writing to CSV, etc.
Edit
You mentioned not being able to find data for MSCI Emerging Markets, which is a world index. Unfortunately, I could not specify world as a country. I do not know what the reasoning behind this is, but I did dive into the source code to find out what was happening:
It turns out that the world indices do exist inside of the investpy/resources/indices.csv, but they are filtered out in index_countries_as_list() because the world country does not exist inside of the variable INDEX_COUNTRIES in investpy/utils/constants.py. You may want to submit an issue here. Here are a few things I was able to do that verify that MSCIEF exists, but again, I am not sure there's a way to get data regarding this index.
investpy.indices.get_indices_dict(country=None, columns=None, as_json=False)
{'country': 'world', 'name': 'MSCI Emerging Markets', 'full_name': 'MSCI Emerging Markets', 'symbol': 'MSCIEF', 'currency': 'USD', 'class': 'other_indices', 'market': 'global_indices'}
and:
investpy.indices.search_indices(by="name", value="MSCI Emerging Markets")
country name ... class market
0 world MSCI Emerging Markets ... other_indices global_indices
[1 rows x 7 columns]
Regarding the part where MSCI Emerging Markets (MSCIEF) is not found:
It can be found via investpy's SearchObj interface. Here's how to do using the wrapper package tessa:
from tessa import search, price_history
r = search("MSCIEF")
# Find the object that fits what you're looking for
prices, currency = price_history('{"id_": 101764, "name": "MSCI Emerging Markets", "symbol": "MSCIEF", "country": "worldFlag", "tag": "/indices/msci-emerging-markets", "pair_type": "indices", "exchange": "Global Indexes"}', "searchobj")
I try to download key financial ratios from yahoo finance via the FundamentalAnalysis library. It's pretty easy for single I have a df with tickers and names:
Ticker Company
0 A Agilent Technologies Inc.
1 AA ALCOA CORPORATION
2 AAC AAC Holdings Inc
3 AAL AMERICAN AIRLINES GROUP INC
4 AAME Atlantic American Corp.
I then tried to use a for-loop to download the ratios for every ticker with fa.ratios().
for i in range (3):
i = 0
i = i + 1
Ratios = fa.ratios(tickers["Ticker"][i])
So basically it shall download all ratios for one ticker and the second and so on. I also tried to change the df into a list, but it didn't work as well. If I put them in a list manually like:
Symbol = ["TSLA" , "AAPL" , "MSFT"]
it works somehow. But as I want to work with Data from 1000+ Tickers I don't want to type all of them manually into a list.
Maybe this question has already been answered elsewhere, in that case sorry, but I've not been able to find a thread that helps me. Any ideas?
You can get symbols using
symbols = df['Ticker'].to_list()
and then you could use for-loop without range()
ratios = dict()
for s in symbols:
ratios[s] = fa.ratios(s)
print(ratios)
Because some symbols may not give ratios so you should use try/except
Minimal working example. I use io.StringIO only to simulate file.
import FundamentalAnalysis as fa
import pandas as pd
import io
text='''Ticker Company
A Agilent Technologies Inc.
AA ALCOA CORPORATION
AAC AAC Holdings Inc
AAL AMERICAN AIRLINES GROUP INC
AAME Atlantic American Corp.'''
df = pd.read_csv(io.StringIO(text), sep='\s{2,}')
symbols = df['Ticker'].to_list()
#symbols = ["TSLA" , "AAPL" , "MSFT"]
print(symbols)
ratios = dict()
for s in symbols:
try:
ratios[s] = fa.ratios(s)
except Exception as ex:
print(s, ex)
for s, ratio in ratios.items():
print(s, ratio)
EDIT: it seems fa.ratios() returns DataFrames and if you will keep them on list then you can concatenate all DataFrames to one DataFrame
ratios = list() # list instead of dictionary
for s in symbols:
try:
ratios.append(fa.ratios(s)) # append to list
except Exception as ex:
print(s, ex)
df = pd.concat(ratios, axis=1) # convert list of DataFrames to one DataFrame
print(df.columns)
print(df)
Doc: pandas.concat()
I'd like to create a new column in a pandas dataframe from the results produced from a regular expression.
The result I'm expecting is:
In[1]: df
Out[1]:
valueProduct valueService totValue
0 $465580.99 $322532.34 $788113.33
My dataframe dtypes are:
df.dtypes
Contracting Office Name object
Contracting Office Region object
PIID object
PIID Agency ID object
Major Program object
Description of Requirement object
Referenced IDV PIID object
Completion Date datetime64[ns]
Prepared By object
Funding Office Name object
Funding Agency ID object
Funding Agency Name object
Funding Office ID object
Effective Date datetime64[ns]
Fiscal Year int64
Ultimate Contract Value float64
Count int64
The column titled "Description of Requirements" in row 1 has a long string value of the following (similar string values in this column through out the dataset):
STEWARDSHIP ADD ADDITIONAL VOLUME AND ROAD WORK CHANGES SILVER SLIDE STEWARDSHIP PROJECT - ALLEGHENY NATIONAL FOREST VALUE OF PRODUCT = $465580.99 VALUE OF SERVICE = $322532.34 TOTAL VALUE OF CONTRACT = $788113.33
I want to successfully write a regex to extract 3 items from this string but only produce the dollar value in new columns:
VALUE OF PRODUCT = $465580.99
VALUE OF SERVICE = $322532.34
TOTAL VALUE OF CONTRACT = $788113.33
Here's the code to do this assuming the string in the dataframe were a simple string value outside of a dataframe:
text = "STEWARDSHIP ADD ADDITIONAL VOLUME AND ROAD WORK CHANGES SILVER SLIDE STEWARDSHIP PROJECT - ALLEGHENY NATIONAL FOREST VALUE OF PRODUCT = $465580.99 VALUE OF SERVICE = $322532.34 TOTAL VALUE OF CONTRACT = $788113.33"
pattern = re.compile('(VALUE OF PRODUCT).{1,3}\$\d*\.\d*', re.IGNORECASE)
getPattern = re.search(pattern, text)
print (getPattern.group())
Which would produce:
VALUE OF PRODUCT = $465580.99
I can repeat this action for the other two items.
Now, sense I'm working in a dataframe I tried to do something like the following:
def valProduct(row):
pattern = re.compile('(VALUE OF PRODUCT).{1,3}\$\d*\.\d*', re.IGNORECASE)
findPattern = re.search(pattern, row['Description of Requirement'])
return findPatter
df['valueProduct'] = df.apply(lambda row: valProduct(row), axis=1)
In[2]: sf[['valueProduct']][:1]
Out[2]: None
This produces a new column but its empty, but should show at the very least:
VALUE OF PRODUCT = $465580.99
Any help is greatly appreciated!
import re
text = "STEWARDSHIP ADD ADDITIONAL VOLUME AND ROAD WORK CHANGES SILVER SLIDE STEWARDSHIP PROJECT - ALLEGHENY NATIONAL FOREST VALUE OF PRODUCT = $465580.99 VALUE OF SERVICE = $322532.34 TOTAL VALUE OF CONTRACT = $788113.33"
re.findall(r'value.+?\d\b',text, re.I)
Output
['VALUE OF PRODUCT = $465580', 'VALUE OF SERVICE = $322532', 'VALUE OF CONTRACT = $788113']
I am using django_countries to show the countries list. Now, I have a requirement where I need to show currency according to country.
Norway - NOK, Europe & Afrika (besides UK) - EUR, UK - GBP, AMERICAS & ASIA - USDs.
Could this be achieved through django_countries project? or are there any other packages in python or django which I could use for this?
Any other solution is welcomed as well.
--------------------------- UPDATE -------------
The main emphasis is on this after getting lot of solutions:
Norway - NOK, Europe & Afrika (besides UK) - EUR, UK - GBP, AMERICAS & ASIA - USDs.
---------------------------- SOLUTION --------------------------------
My solution was quite simple, when I realized that I couldnt get any ISO format or a package to get what I want, I thought to write my own script. It is just a conditional based logic:
from incf.countryutils import transformations
def getCurrencyCode(self, countryCode):
continent = transformations.cca_to_ctn(countryCode)
# print continent
if str(countryCode) == 'NO':
return 'NOK'
if str(countryCode) == 'GB':
return 'GBP'
if (continent == 'Europe') or (continent == 'Africa'):
return 'EUR'
return 'USD'
Dont know whether this is efficient way or not, would like to hear some suggestions.
Thanks everyone!
There are several modules out there:
pycountry:
import pycountry
country = pycountry.countries.get(name='Norway')
currency = pycountry.currencies.get(numeric=country.numeric)
print currency.alpha_3
print currency.name
prints:
NOK
Norwegian Krone
py-moneyed
import moneyed
country_name = 'France'
for currency, data in moneyed.CURRENCIES.iteritems():
if country_name.upper() in data.countries:
print currency
break
prints EUR
python-money
import money
country_name = 'France'
for currency, data in money.CURRENCY.iteritems():
if country_name.upper() in data.countries:
print currency
break
prints EUR
pycountry is regularly updated, py-moneyed looks great and has more features than python-money, plus python-money is not maintained now.
Hope that helps.
django-countries just hands you a field to couple to your model (and a static bundle with flag icons). The field can hold a 2 character ISO from the list in countries.py which is convenient if this list is up-to-date (haven't checked) because it saves a lot of typing.
If you wish to create a model with verbose data that's easily achieved, e.g.
class Country(models.Model):
iso = CountryField()
currency = # m2m, fk, char or int field with pre-defined
# choices or whatever suits you
>> obj = Country.objects.create(iso='NZ', currency='NZD')
>> obj.iso.code
u'NZ'
>> obj.get_iso_display()
u'New Zealand'
>> obj.currency
u'NZD'
An example script of preloading data, which could later be exported to create a fixture which is a nicer way of managing sample data.
from django_countries.countries import COUNTRIES
for key in dict(COUNTRIES).keys():
Country.objects.create(iso=key)
I have just released country-currencies, a module that gives you a mapping of country codes to currencies.
>>> from country_currencies import get_by_country
>>> get_by_country('US')
('USD',)
>>> get_by_country('ZW')
('USD', 'ZAR', 'BWP', 'GBP', 'EUR')