I have a lot of different table (and other unstructured data in an excel sheet) .. I need to create a dataframe out of range 'A3:D20' from 'Sheet2' of Excel sheet 'data'.
All examples that I come across drilldown up to sheet level, but not how to pick it from an exact range.
import openpyxl
import pandas as pd
wb = openpyxl.load_workbook('data.xlsx')
sheet = wb.get_sheet_by_name('Sheet2')
range = ['A3':'D20'] #<-- how to specify this?
spots = pd.DataFrame(sheet.range) #what should be the exact syntax for this?
print (spots)
Once I get this, I plan to look up data in column A and find its corresponding value in column B.
Edit 1: I realised that openpyxl takes too long, and so have changed that to pandas.read_excel('data.xlsx','Sheet2') instead, and it is much faster at that stage at least.
Edit 2: For the time being, I have put my data in just one sheet and:
removed all other info
added column names,
applied index_col on my leftmost column
then used wb.loc[]
Use the following arguments from pandas read_excel documentation:
skiprows : list-like
Rows to skip at the beginning (0-indexed)
nrows: int, default None
Number of rows to parse.
parse_cols : int or list, default None
If None then parse all columns,
If int then indicates last column to be parsed
If list of ints then indicates list of column numbers to be parsed
If string then indicates comma separated list of column names and column ranges (e.g. “A:E” or “A,C,E:F”)
I imagine the call will look like:
df = read_excel(filename, 'Sheet2', skiprows = 2, nrows=18, parse_cols = 'A:D')
EDIT:
in later version of pandas parse_cols has been renamed to usecols so the above call should be rewritten as:
df = read_excel(filename, 'Sheet2', skiprows = 2, nrows=18, usecols= 'A:D')
One way to do this is to use the openpyxl module.
Here's an example:
from openpyxl import load_workbook
wb = load_workbook(filename='data.xlsx',
read_only=True)
ws = wb['Sheet2']
# Read the cell values into a list of lists
data_rows = []
for row in ws['A3':'D20']:
data_cols = []
for cell in row:
data_cols.append(cell.value)
data_rows.append(data_cols)
# Transform into dataframe
import pandas as pd
df = pd.DataFrame(data_rows)
my answer with pandas O.25 tested and worked well
pd.read_excel('resultat-elections-2012.xls', sheet_name = 'France entière T1T2', skiprows = 2, nrows= 5, usecols = 'A:H')
pd.read_excel('resultat-elections-2012.xls', index_col = None, skiprows= 2, nrows= 5, sheet_name='France entière T1T2', usecols=range(0,8))
So :
i need data after two first lines ; selected desired lines (5) and col A to H.
Be carefull #shane answer's need to be improved and updated with the new parameters of Pandas
Related
I want to read from excel sheet a specific cell: h6. So I try it like this:
import pandas as pd
excel_file = './docs/fruit.xlsx'
df = pd.read_excel(excel_file,'Overzicht')
sheet = df.active
x1 = sheet['H6'].value
print(x1)
But then I get this error:
File "C:\Python310\lib\site-packages\pandas\core\generic.py", line 5575, in __getattr__
return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute 'active'
So my questiion is: How to read specif cell from sheet from excelsheet?
Thank you
Oke, I tried with openpyxl:
import openpyxl
path = "./docs/fruit.xlsx"
wb_obj = openpyxl.load_workbook(path)
sheet_obj = wb_obj.active
cell_obj = sheet_obj.cell(row = 6, column = 9)
print(cell_obj.value)
But then the formula is printed. Like this:
=(H6*1000)/F6/G6
and not the value: 93
You can do this using openpyxl directly or pandas (which internally uses openpyxl behind the scene)...
Using Openpyxl
You will need to use data_only=True when you open the file. Also, make sure you know the row and column number. To read the data in H6, row would be 6 and 8 would be H
import openpyxl
path = "./docs/Schoolfruit.xlsx"
wb_obj = openpyxl.load_workbook(path, data_only=True)
sheet_obj = wb_obj.active ## Or use sheet_obj = wb_obj['Sheet1'] if you know sheet name
val = sheet_obj.cell(row = 6, column = 8).value
print(val)
Using Pandas
The other option is to use pandas read_excel() which will read the whole sheet into a dataframe. You can use iloc() or at() to read the specific cell. Note that this is probably the less optimal solution if you need to read just one cell...
Another point to note here is that, once you have read the data into a dataframe, the row 1 will be considered as the header and the first row would now be 0. So the row number would be 4 instead of 6. Similarly, the first column would now be 0 and not 1, which would change the position to [4,7]
import pandas as pd
path = "./docs/Schoolfruit.xlsx"
df = pd.read_excel(path, 'Sheet1')
print(df.iloc[4,7])
I found a solution and hope, it works for you.
import pandas as pd
excel_file = './docs/Schoolfruit.xlsx'
df = pd.read_excel(excel_file, sheet_name='active' ,header=None, skiprows=1)
print(df[7][4])
7: Hth column
4: 6th row (skipped first row and index is began from 0)
I have an excel sheet:
31-12-2019 31-01-2020 28-02-2020 *(which btw is formatted as: 31-Dec-19, 31-Jan-20, etc. not sure if relevant)*
1 -0,36% 0,12% -0,09%
2 -0,18% 0,06% -0,07%
3 0,05% 0,04% 0,14%
To be clear, the problem is not in reading the file, but the issue below.
I want to read this file with pandas in python and have the dates in the header as strings. So that later i can to refer to any column with something like df['31-12-2019'].
When I read the excel now, I get a keyerror message, because the formats of the dates in the header are changed. I read it like this now:
curve = pd.read_excel("Monthly curves.xlsx", sheet_name = "swap", skiprows = 1, index_col = 0)
I receive the error when selecting for instance column 31-12-2019: "Keyerror: '31-12-2019'. Any help would be much appreciated!
Also, the first column does not have a header, how can I name it myself as 'years'?
It worked when I used this:
import pandas as pnd
file = 'excelfile.xlsx'
df = pnd.read_excel(file,sheet_name=0,index_col=0)
df.head()
I don't know about naming the headers though...
I worked around my problem by reading the file as follows:
curve = pd.read_excel("Monthly Curves.xlsx", sheet_name = "swap", index_col = 0, skiprows = 2, header = None)
Then to select for instance the 91th column I used .loc (because .ix is deprecated), and I did that in the following way:
M12 = curve.loc[:, 91]
Hope that helps others as well!
I have existing excel document and want to update M column according to A column. And I want to start from second row to maintain first row 'header'.
Here is my code;
import openpyxl
wb = openpyxl.load_workbook('D:\Documents\Desktop\deneme/formula.xlsx')
ws=wb['Sheet1']
for i, cellObj in enumerate(ws['M'], 1):
cellObj.value = '=_xlfn.ISOWEEKNUM(A2)'.format(i)
wb.save('D:\Documents\Desktop\deneme/formula.xlsx')
When I run that code;
-first row 'header' changes.
-all columns in excel "ISOWEEKNUM(A2)", but I want it to change according to row number (A3,A4,A5... "ISOWEEKNUM(A3), ISOWEEKNUM(A4), ISOWEEKNUM(A5)....")
Edit:
I handled right now the ISOWEEKNUM issue with below code. I changed A2 to A2:A5.
import openpyxl
wb = openpyxl.load_workbook('D:\Documents\Desktop\deneme/formula.xlsx')
ws=wb['Sheet1']
for i, cellObj in enumerate(ws['M'], 1):
cellObj.value = '=_xlfn.ISOWEEKNUM(A2:A5)'.format(i)
wb.save('D:\Documents\Desktop\deneme/formula.xlsx')
But still starts from first row.
Here is an answer using pandas.
Let us consider the following spreadsheet:
First import pandas:
import pandas as pd
Then load the third sheet of your excel workbook into a dataframe called df:
df=pd.read_excel('D:\Documents\Desktop\deneme/formula.xlsx', sheet_name='Sheet3')
Update column 'column_to_update' using column 'deneme'. The line below converts the dates in the 'deneme' column from strings to datetime objects and then returns the week of the year associated with each of those dates.
df['Column_to_update'] = pd.to_datetime(df['deneme']).dt.week
You can then save your dataframe to a new excel document:
df.to_excel('./newspreadsheet.xlsx', index=False)
Here is the result:
You can see that the values in 'column_to_update' got updated from 1, 2 and 3 to 12, 12 and 18.
I would like to convert an excel file to a pandas dataframe. All the sheets name have spaces in the name, for instances, ' part 1 of 22, part 2 of 22, and so on. In addition the first column is the same for all the sheets.
I would like to convert this excel file to a unique dataframe. However I dont know what happen with the name in python. I mean I was hable to import them, but i do not know the name of the data frame.
The sheets are imported but i do not know the name of them. After this i would like to use another 'for' and use a pd.merge() in order to create a unique dataframe
for sheet_name in Matrix.sheet_names:
sheet_name = pd.read_excel(Matrix, sheet_name)
print(sheet_name.info())
Using only the code snippet you have shown, each sheet (each DataFrame) will be assigned to the variable sheet_name. Thus, this variable is overwritten on each iteration and you will only have the last sheet as a DataFrame assigned to that variable.
To achieve what you want to do you have to store each sheet, loaded as a DataFrame, somewhere, a list for example. You can then merge or concatenate them, depending on your needs.
Try this:
all_my_sheets = []
for sheet_name in Matrix.sheet_names:
sheet_name = pd.read_excel(Matrix, sheet_name)
all_my_sheets.append(sheet_name)
Or, even better, using list comprehension:
all_my_sheets = [pd.read_excel(Matrix, sheet_name) for sheet_name in Matrix.sheet_names]
You can then concatenate them into one DataFrame like this:
final_df = pd.concat(all_my_sheets, sort=False)
You might consider using the openpyxl package:
from openpyxl import load_workbook
import pandas as pd
wb = load_workbook(filename=file_path, read_only=True)
all_my_sheets = wb.sheetnames
# Assuming your sheets have the same headers and footers
n = 1
for ws in all_my_sheets:
records = []
for row in ws._cells_by_row(min_col=1,
min_row=n,
max_col=ws.max_column,
max_row=n):
rec = [cell.value for cell in row]
records.append(rec)
# Make sure you don't duplicate the header
n = 2
# ------------------------------
# Set the column names
records = records[header_row-1:]
header = records.pop(0)
# Create your df
df = pd.DataFrame(records, columns=header)
It may be easiest to call read_excel() once, and save the contents into a list.
So, the first step would look like this:
dfs = pd.read_excel(["Sheet 1", "Sheet 2", "Sheet 3"])
Note that the sheet names you use in the list should be the same as those in the excel file. Then, if you wanted to vertically concatenate these sheets, you would just call:
final_df = pd.concat(dfs, axis=1)
Note that this solution would result in a final_df that includes column headers from all three sheets. So, ideally they would be the same. It sounds like you want to merge the information, which would be done differently; we can't help you with the merge without more information.
I hope this helps!
I am trying to figure out a way to read data from a specific column from a certain cell range and store it into a array using pandas.
For example my Excel sheet consists of :
test | p
Food| Price
Chicken| 8.54
Beef |6.73
Vegetables| 3.2
Total Price |18.47
Note: there is a an empty space on the first row for a reason.
Note: | indicates cell separation.
I am trying to get the price values which start from Row B3 to row B5 and store them into an array via [8.54,6.73,3.2].
So far the code I have is:
import pandas as pd
xl_workbook = pd.ExcelFile("readme.xlsx") # Load the excel workbook
df = xl_workbook.parse("Sheet1") # Parse the sheet into a dataframe
x1_list = df['p'].tolist() # Cast the desired column into a python list
print(x1_list)
Which then results to [nan, u'price',8.54,6.73,3.2]
If I just wanted to read the values 8.54, 6.73, and 3.2, to result in [8.54,6.73,3.2] how would I do this?
Is there a way to grab a certain column of a certain cell range?
As written, you could use read_excel in Pandas. This assumes you have consistent formatting.
import pandas as pd
# define the file name and "sheet name"
fn = 'Book1.xlsx'
sn = 'Sheet1'
data = pd.read_excel(fn, sheetname=sn, index_col=0, skiprows=1, header=0, skip_footer=1)