Related
Magicians out there....
I need your help with the best approaches for the below use case.
I have an excel sheet whith lakhs of rows of data and I need to filter it based on some criteria and need to create new multiple tiles.
I am in no mood to do it manually hence started working out on a python script that will do the job for me. however, I am facing a few challenges in achieving the end goal. The challenges are the "Color Formatting" and "comment" added to the cell.
Let's recreate the scenario. I have attached a sample excel sheet for your reference here.
it includes "Indian Cars" data with 4 headers called (Brand, Model, Fuel Type & Transmission Type). I need to filter the data based on "Brand" and create a new excel file (workbook) with the Brand name as the excel file name.
Approach 1:-
First I started with loading an excelsheet into a data frame with Pandas and then filtered the data and exported it, that was quite fast and easy and I loved it. However, I am losing cell colors and added note to the cell (Model & Fuel type)
Note: I tried styling the pandas, however, for some reason, it's not working for me.
Approach 2:-
I though of using Openpyxl & Xlsxwriter, however, the issue is I am unable to filter data and keep comments added to the header.
Approach 3:-
Logically, I can create a copy of my existing sheet and delete the unwanted rows from it and save it with desired name, that should do the job for me. Unfortunately, I am unable to figure out how to achieve it in python.
Kindly share your thoughts on this and help me with right approach... and If I can get a sample code or full code... that would just make my day... :D
This should do the trick. You can change the colors of the headers.
Code for custom styling of the excel added.
import pandas as pd
# function to style the dataframe with some conditons (simple condition for an example you can change or add conditions with multiple rows)
def style_df(row):
values = pd.Series(data=False, index=row.index)
if not pd.isna(row['Transmission Type']):
if row['Transmission Type'].strip() == 'Manual':
return ['background-color : gray; color: red' for _ in values]
elif row['Transmission Type'].strip() == 'Manual, Automatic':
return ['background-color : lightblue; color: green' for _ in values]
return ['' for _ in values]
page = pd.read_excel("Cars_in_india.xlsx", 'Cars in India')
# creating an excel file for each brand
for brand in page.Brand.unique():
writer = pd.ExcelWriter(brand+".xlsx", engine = 'xlsxwriter')
workbook = writer.book
border_fmt = workbook.add_format({'bottom':1, 'top':1, 'left':1, 'right':1})
dataframe = page[page.Brand == brand].copy()
dataframe = dataframe.style.apply(style_df, axis=1)
dataframe.to_excel(writer, index=False, sheet_name=brand)
# dynamic columns sizes
for column in page[page.Brand == brand]:
column_width = max(page[page.Brand == brand][column].astype(str).map(len).max(), len(column))
col_idx = page[page.Brand == brand].columns.get_loc(column)
writer.sheets[brand].set_column(col_idx, col_idx, column_width)
worksheet = writer.sheets[brand]
#applying style to the header columns
worksheet.write(0, 1, "Model", workbook.add_format({'fg_color': '#00FF00'}))
worksheet.write(0, 2, "Fuel Type", workbook.add_format({'fg_color': '#FFA500'}))
# applying borders to the table
worksheet.conditional_format(xlsxwriter.utility.xl_range(0, 0, len(page[page.Brand == brand]), len(page[page.Brand == brand].columns)-1), {'type': 'no_errors', 'format': border_fmt})
writer.save()
You can use openpyxl to read the coments and then write the comments when creating the excel. But you used a type of comment not compatible with the current version of excel that openpyxl uses (you will see the same error in the google cloud editor). Then, the only option is to change the type of the comment or rewrite them in the python code.
Example code:
from openpyxl import load_workbook
wb = load_workbook("Cars_in_india.xlsx")
ws = wb["Cars in India"]
_, comment, comment2, _ = list(ws.rows)[0]
# then after this code:
# worksheet.write(0, 1, "Model", workbook.add_format({'fg_color': '#00FF00'}))
# worksheet.write(0, 2, "Fuel Type", workbook.add_format({'fg_color': '#FFA500'}))
# you can add:
worksheet.write_comment('B1', comment.text)
worksheet.write_comment('C1', comment2.text)
I want the text in an Excel cell to align with the top, rather then with the bottom, when writing a file from python using openpyxl
This works for me. I am using openpyxl v2.5.8.
new_cell.alignment=Alignment(horizontal='general',
vertical='top',
text_rotation=0,
wrap_text=False,
shrink_to_fit=False,
indent=0)
For more info: https://openpyxl.readthedocs.io/en/stable/styles.html?highlight=cell%20alignment
Try this:
sheet["A1"].alignment.vertical = "top"
Use two loops to apply the alignment format to each cell.
al = Alignment(horizontal="left", vertical="top")
for row in sheet['A1':'A2']:
for cell in row:
cell.alignment = al
This idea comes from https://groups.google.com/forum/#!topic/openpyxl-users/GDrfknwrYEM
You set the cell style.alignment.vertical to a desired value. For example, to make the cell A1 to vertically align up:
# ws is worksheet
myCell = ws.cell('A1')
myCell.style.alignment.vertical = Alignment.VERTICAL_TOP
Find more details on the class reference page here.
I am trying to create an Excel workbook where I can auto-set, or auto-adjust the widths of the columns before saving the workbook.
I have been reading the Python-Excel tutorial in hopes of finding some functions in xlwt that emulate xlrd ones (such as sheet_names(), cellname(row, col), cell_type, cell_value, and so on...) For example, suppose I have the following:
from xlwt import Workbook
wb = Workbook()
sh1 = wb.add_sheet('sheet1' , cell_overwrite_ok = True)
sh2 = wb.get_sheet(0)
wb.get_sheet(0) is similar to the rb.sheet_by_index(0) function offered in xlrd, except that the former allows you to modify the contents (provided the user has set cell_overwrite_ok = True)
Assuming xlwt DOES offer the functions I am looking for, I was planning on going through every worksheet again, but this time keeping track of the content that takes the most space for a particular column, and set the column width based on that. Of course, I can also keep track of the max width for a specific column as I write to the sheet, but I feel like it would be cleaner to set the widths after all the data has been already written.
Does anyone know if I can do this? If not, what do you recommend doing in order to adjust the column widths?
I just implemented a wrapper class that tracks the widths of items as you enter them. It seems to work pretty well.
import arial10
class FitSheetWrapper(object):
"""Try to fit columns to max size of any entry.
To use, wrap this around a worksheet returned from the
workbook's add_sheet method, like follows:
sheet = FitSheetWrapper(book.add_sheet(sheet_name))
The worksheet interface remains the same: this is a drop-in wrapper
for auto-sizing columns.
"""
def __init__(self, sheet):
self.sheet = sheet
self.widths = dict()
def write(self, r, c, label='', *args, **kwargs):
self.sheet.write(r, c, label, *args, **kwargs)
width = arial10.fitwidth(label)
if width > self.widths.get(c, 0):
self.widths[c] = width
self.sheet.col(c).width = width
def __getattr__(self, attr):
return getattr(self.sheet, attr)
All the magic is in John Yeung's arial10 module. This has good widths for Arial 10, which is the default Excel font. If you want to write worksheets using other fonts, you'll need to change the fitwidth function, ideally taking into account the style argument passed to FitSheetWrapper.write.
If one is not interested in using another class (FitSheetWrapper), then this can be implemented using WorkSheet column Method.
work = xlwt.WorkBook()
sheet = work.add_sheet('Sheet1')
for row_index in range(0,max_row):
for column_index in range(0,max_col) :
cwidth = sheet.col(column_index).width
if (len(column_data)*367) > cwidth:
sheet.col(column_index).width = (len(column_data)*367) #(Modify column width to match biggest data in that column)
sheet.write(row_index,column_index,column_data,style)
Default value of width is 2962 units and excel points it to as 8.11 units. Hence i am multiplying 367 to length of data.
This is adapted from Kevins FitSheetWrapper.
There is no automatic facility for this in xlwt. You have to follow the general pattern you describe, of keeping track of the max width as you're writing, and setting the column width at the end, sometime after you've seen all the data but before you've saved the workbook.
Note that this is the cleanest and most efficient approach available when dealing with Excel files. If your notion of "after the data has already been written" means after you've already committed the cell values ("writing") but before actually saving the workbook, then the method described above is doing exactly this. If what you mean is after you've already saved the workbook, you want to read it again to get the max widths, and then save it again with new column widths, this will be much slower, and will involve using both xlwt and xlrd (and possibly xlutils as well). Also note that when you are using the genuine Microsoft Excel, there is no notion of "updating" a file. It may seem like that from a user point of view, but what is happening behind the scenes is that every time you do a save, Excel blows away the existing file and writes a brand new one from scratch.
FitSheetWrapper should have a little modify with xlwt3 in 3.3.4
line 19:
change:
width = arial10.fitwidth(label)
to:
width = int(arial10.fitwidth(label))
reason:
\Python\3.3.3\Lib\site-packages\xlwt3\biffrecords.py
1624 def __init__(self, first_col, last_col, width, xf_index, options):
1625 self._rec_data = pack('<6H', first_col, last_col, width, xf_index, options, 0)
width must be integer.
This may be a little late, but I created a method that does this for the whole
sheet at once. It's quick and gets the job done. The extra cushion param. is only needed if you think that the 256 calculation won't be accurate (if you have longer text fields).
from xlrd import *
from xlwt import *
def autoAdjustColumns(workbook, path, writerSheet, writerSheet_index, extraCushion):
readerSheet = open_workbook(path).sheet_by_index(writerSheet_index)
for row in range(readerSheet.nrows):
for column in range(readerSheet.ncols):
thisCell = readerSheet.cell(row, column)
neededWidth = int((1 + len(str(thisCell.value))) * 256)
if writerSheet.col(column).width < neededWidth:
writerSheet.col(column).width = neededWidth + extraCushion
workbook.save(path)
i use this method:
wb = Workbook()
ws = wb.add_sheet('Sheet1')
columnwidth = {}
row = 0
for rowdata in data:
column = 0
for colomndata in rowdata:
if column in columnwidth:
if len(colomndata) > columnwidth[column]:
columnwidth[column] = len(colomndata)
else:
columnwidth[column] = len(colomndata)
ws.write(row, column, colomndata, style0)
column = column + 1
row = row + 1
for column, widthvalue in columnwidth.items():
ws.col(column).width = (widthvalue + 4) * 367
I need advice on setting styles in Openpyxl.
I see that the NumberFormat of a cell can be set, but I also require setting of font colors and attributes (bold etc). There is a style.py class but it seems I can't set the style attribute of a cell, and I don't really want to start tinkering with the openpyxl source code.
Has anyone found a solution to this?
As of openpyxl version 1.5.7, I have successfully applied the following worksheet style options...
from openpyxl.reader.excel import load_workbook
from openpyxl.workbook import Workbook
from openpyxl.styles import Color, Fill
from openpyxl.cell import Cell
# Load the workbook...
book = load_workbook('foo.xlsx')
# define ws here, in this case I pick the first worksheet in the workbook...
# NOTE: openpyxl has other ways to select a specific worksheet (i.e. by name
# via book.get_sheet_by_name('someWorksheetName'))
ws = book.worksheets[0]
## ws is a openpypxl worksheet object
_cell = ws.cell('C1')
# Font properties
_cell.style.font.color.index = Color.GREEN
_cell.style.font.name = 'Arial'
_cell.style.font.size = 8
_cell.style.font.bold = True
_cell.style.alignment.wrap_text = True
# Cell background color
_cell.style.fill.fill_type = Fill.FILL_SOLID
_cell.style.fill.start_color.index = Color.DARKRED
# You should only modify column dimensions after you have written a cell in
# the column. Perfect world: write column dimensions once per column
#
ws.column_dimensions["C"].width = 60.0
FYI, you can find the names of the colors in openpyxl/style.py... I sometimes I patch in extra colors from the X11 color names
class Color(HashableObject):
"""Named colors for use in styles."""
BLACK = 'FF000000'
WHITE = 'FFFFFFFF'
RED = 'FFFF0000'
DARKRED = 'FF800000'
BLUE = 'FF0000FF'
DARKBLUE = 'FF000080'
GREEN = 'FF00FF00'
DARKGREEN = 'FF008000'
YELLOW = 'FFFFFF00'
DARKYELLOW = 'FF808000'
For openpyxl version 2.4.1 and above use below code to set font color:
from openpyxl.styles import Font
from openpyxl.styles.colors import Color
ws1['A1'].font = Font(color = "FF0000")
hex codes for various colors can be found at:
http://dmcritchie.mvps.org/excel/colors.htm
As of openpyxl 2.0, styles are immutable.
If you have a cell, you can (e.g.) set bold text by:
cell.style = cell.style.copy(font=cell.style.font.copy(bold=True))
Yes, this is annoying.
As of openpyxl 2.0, setting cell styles is done by creating new style objects and by assigning them to properties of a cell.
There are several style objects: Font, PatternFill, Border, and Alignment. See the doc.
To change a style property of a cell, first you either have to copy the existing style object from the cell and change the value of the property or you have to create a new style object with the desired settings. Then, assign the new style object to the cell.
Example of setting the font to bold and italic of cell A1:
from openpyxl import Workbook
from openpyxl.styles import Font
# Create workbook
wb = Workbook()
# Select active sheet
ws = wb.active()
# Select cell A1
cell = ws['A1']
# Make the text of the cell bold and italic
cell.font = cell.font.copy(bold=True, italic=True)
This seems like a feature that has changed a few times. I am using openpyxl 2.5.0, and I was able to set the strike-through option this way:
new_font = copy(cell.font)
new_font.strike = True
cell.font = new_font
It seems like earlier versions (1.9 to 2.4?) had a copy method on the font that is now deprecated and raises a warning:
cell.font = cell.font.copy(strike=True)
Versions up to 1.8 had mutable fonts, so you could just do this:
cell.font.strike=True
That now raises an error.
New 2021 Updated Way of Changing FONT in OpenPyXl:
sheet.cell.font = Font(size=23, underline='single', color='FFBB00', bold=True, italic=True)
Full Code:
import openpyxl # Connect the library
from openpyxl import Workbook
from openpyxl.styles import PatternFill # Connect cell styles
from openpyxl.workbook import Workbook
from openpyxl.styles import Font, Fill # Connect styles for text
from openpyxl.styles import colors # Connect colors for text and cells
wb = openpyxl.Workbook() # Create book
work_sheet = wb.create_sheet(title='Testsheet') # Created a sheet with a name and made it active
work_sheet['A1'] = 'Test text'
work_sheet_a1 = work_sheet['A5'] # Created a variable that contains cell A1 with the existing text
work_sheet_a1.font = Font(size=23, underline='single', color='FFBB00', bold=True,
italic=True) # We apply the following parameters to the text: size - 23, underline, color = FFBB00 (text color is specified in RGB), bold, oblique. If we do not need a bold font, we use the construction: bold = False. We act similarly if we do not need an oblique font: italic = False.
# Important:
# if necessary, the possibility of using standard colors is included in the styles, but the code in this case will look different:
work_sheet_a1.font = Font(size=23, underline='single', color=colors.RED, bold=True,
italic=True) # what color = colors.RED — color prescribed in styles
work_sheet_a1.fill = PatternFill(fill_type='solid', start_color='ff8327',
end_color='ff8327') # This code allows you to do design color cells
Like openpyxl doc said:
This is an open source project, maintained by volunteers in their spare time. This may well mean that particular features or functions that you would like are missing.
I checked openpyxl source code, found that:
Till openpyxl 1.8.x, styles are mutable. Their attribute can be assigned directly like this:
from openpyxl.workbook import Workbook
from openpyxl.style import Color
wb = Workbook()
ws = wb.active
ws['A1'].style.font.color.index = Color.RED
However from of openpyxl 1.9, styles are immutable.
Styles are shared between objects and once they have been assigned they cannot be changed. This stops unwanted side-effects such as changing the style for lots of cells when instead of only one.
To create a new style object, you can assign it directly, or copy one from an existing cell's style with new attributes, answer to the question as an example(forgive my Chinese English):
from openpyxl.styles import colors
from openpyxl.styles import Font, Color
from openpyxl import Workbook
wb = Workbook()
ws = wb.active
a1 = ws['A1']
d4 = ws['D4']
# create a new style with required attributes
ft_red = Font(color=colors.RED)
a1.font = ft_red
# you can also do it with function copy
ft_red_bold = ft_red.copy(bold=True)
# you can copy from a cell's style with required attributes
ft_red_sigle_underline = a1.font.copy(underline="single")
d4.font = ft_red_bold
# apply style to column E
col_e = ws.column_dimensions['E']
col_e.font = ft_red_sigle_underline
A cell' style contains these attributes: font, fill, border, alignment, protection and number_format. Check openpyxl.styles.
They are similar and should be created as an object, except number_format, its value is string type.
Some pre-defined number formats are available, number formats can also be defined in string type. Check openpyxl.styles.numbers.
from openpyxl.styles import numbers
# use pre-defined values
ws.cell['T49'].number_format = numbers.FORMAT_GENERAL
ws.cell(row=2, column=4).number_format = numbers.FORMAT_DATE_XLSX15
# use strings
ws.cell['T57'].number_format = 'General'
ws.cell(row=3, column=5).number_format = 'd-mmm-yy'
ws.cell['E5'].number_format = '0.00'
ws.cell['E50'].number_format = '0.00%'
ws.cell['E100'].number_format = '_ * #,##0_ ;_ * -#,##0_ ;_ * "-"??_ ;_ #_ '
As of openpyxl-1.7.0 you can do this too:
cell.style.fill.start_color.index = "FF124191"
I've got a couple of helper functions which set a style on a given cell - things like headers, footers etc.
You can define a common style then you can apply the same to any cell or range.
Define Style:
Apply on a cell.
This worked for me (font colour + bold font):
from openpyxl.styles import colors
from openpyxl.styles import Font, Color
from openpyxl import Workbook
wb = Workbook()
ws = wb['SheetName']
ws.cell(row_number,column_number).font = Font(color = "0000FF",bold = True)
I am trying to create an Excel workbook where I can auto-set, or auto-adjust the widths of the columns before saving the workbook.
I have been reading the Python-Excel tutorial in hopes of finding some functions in xlwt that emulate xlrd ones (such as sheet_names(), cellname(row, col), cell_type, cell_value, and so on...) For example, suppose I have the following:
from xlwt import Workbook
wb = Workbook()
sh1 = wb.add_sheet('sheet1' , cell_overwrite_ok = True)
sh2 = wb.get_sheet(0)
wb.get_sheet(0) is similar to the rb.sheet_by_index(0) function offered in xlrd, except that the former allows you to modify the contents (provided the user has set cell_overwrite_ok = True)
Assuming xlwt DOES offer the functions I am looking for, I was planning on going through every worksheet again, but this time keeping track of the content that takes the most space for a particular column, and set the column width based on that. Of course, I can also keep track of the max width for a specific column as I write to the sheet, but I feel like it would be cleaner to set the widths after all the data has been already written.
Does anyone know if I can do this? If not, what do you recommend doing in order to adjust the column widths?
I just implemented a wrapper class that tracks the widths of items as you enter them. It seems to work pretty well.
import arial10
class FitSheetWrapper(object):
"""Try to fit columns to max size of any entry.
To use, wrap this around a worksheet returned from the
workbook's add_sheet method, like follows:
sheet = FitSheetWrapper(book.add_sheet(sheet_name))
The worksheet interface remains the same: this is a drop-in wrapper
for auto-sizing columns.
"""
def __init__(self, sheet):
self.sheet = sheet
self.widths = dict()
def write(self, r, c, label='', *args, **kwargs):
self.sheet.write(r, c, label, *args, **kwargs)
width = arial10.fitwidth(label)
if width > self.widths.get(c, 0):
self.widths[c] = width
self.sheet.col(c).width = width
def __getattr__(self, attr):
return getattr(self.sheet, attr)
All the magic is in John Yeung's arial10 module. This has good widths for Arial 10, which is the default Excel font. If you want to write worksheets using other fonts, you'll need to change the fitwidth function, ideally taking into account the style argument passed to FitSheetWrapper.write.
If one is not interested in using another class (FitSheetWrapper), then this can be implemented using WorkSheet column Method.
work = xlwt.WorkBook()
sheet = work.add_sheet('Sheet1')
for row_index in range(0,max_row):
for column_index in range(0,max_col) :
cwidth = sheet.col(column_index).width
if (len(column_data)*367) > cwidth:
sheet.col(column_index).width = (len(column_data)*367) #(Modify column width to match biggest data in that column)
sheet.write(row_index,column_index,column_data,style)
Default value of width is 2962 units and excel points it to as 8.11 units. Hence i am multiplying 367 to length of data.
This is adapted from Kevins FitSheetWrapper.
There is no automatic facility for this in xlwt. You have to follow the general pattern you describe, of keeping track of the max width as you're writing, and setting the column width at the end, sometime after you've seen all the data but before you've saved the workbook.
Note that this is the cleanest and most efficient approach available when dealing with Excel files. If your notion of "after the data has already been written" means after you've already committed the cell values ("writing") but before actually saving the workbook, then the method described above is doing exactly this. If what you mean is after you've already saved the workbook, you want to read it again to get the max widths, and then save it again with new column widths, this will be much slower, and will involve using both xlwt and xlrd (and possibly xlutils as well). Also note that when you are using the genuine Microsoft Excel, there is no notion of "updating" a file. It may seem like that from a user point of view, but what is happening behind the scenes is that every time you do a save, Excel blows away the existing file and writes a brand new one from scratch.
FitSheetWrapper should have a little modify with xlwt3 in 3.3.4
line 19:
change:
width = arial10.fitwidth(label)
to:
width = int(arial10.fitwidth(label))
reason:
\Python\3.3.3\Lib\site-packages\xlwt3\biffrecords.py
1624 def __init__(self, first_col, last_col, width, xf_index, options):
1625 self._rec_data = pack('<6H', first_col, last_col, width, xf_index, options, 0)
width must be integer.
This may be a little late, but I created a method that does this for the whole
sheet at once. It's quick and gets the job done. The extra cushion param. is only needed if you think that the 256 calculation won't be accurate (if you have longer text fields).
from xlrd import *
from xlwt import *
def autoAdjustColumns(workbook, path, writerSheet, writerSheet_index, extraCushion):
readerSheet = open_workbook(path).sheet_by_index(writerSheet_index)
for row in range(readerSheet.nrows):
for column in range(readerSheet.ncols):
thisCell = readerSheet.cell(row, column)
neededWidth = int((1 + len(str(thisCell.value))) * 256)
if writerSheet.col(column).width < neededWidth:
writerSheet.col(column).width = neededWidth + extraCushion
workbook.save(path)
i use this method:
wb = Workbook()
ws = wb.add_sheet('Sheet1')
columnwidth = {}
row = 0
for rowdata in data:
column = 0
for colomndata in rowdata:
if column in columnwidth:
if len(colomndata) > columnwidth[column]:
columnwidth[column] = len(colomndata)
else:
columnwidth[column] = len(colomndata)
ws.write(row, column, colomndata, style0)
column = column + 1
row = row + 1
for column, widthvalue in columnwidth.items():
ws.col(column).width = (widthvalue + 4) * 367