I have a dataframe like as shown below
Date,cust,region,Abr,Number,
12/01/2010,Company_Name,Somecity,Chi,36,
12/02/2010,Company_Name,Someothercity,Nyc,156,
df = pd.read_clipboard(sep=',')
I would like to write this dataframe to a specific sheet (called temp_data) in the file output.xlsx
Therfore I tried the below
import pandas
from openpyxl import load_workbook
book = load_workbook('output.xlsx')
writer = pandas.ExcelWriter('output.xlsx', engine='openpyxl')
writer.book = book
writer.sheets = dict((ws.title, ws) for ws in book.worksheets)
I also tried the below
path = 'output.xlsx'
with pd.ExcelWriter(path) as writer:
writer.book = openpyxl.load_workbook(path)
final_df.to_excel(writer, sheet_name='temp_data',startrow=10)
writer.save()
But am not sure whether I am overcomplicating it. I get an error like as shown below. But I verifiedd in task manager, no excel file/task is running
BadZipFile: File is not a zip file
Moreover, I also lose my formatting of the output.xlsx file when I manage to write the file based on below suggestions. I already have a neatly formatted font,color file etc and just need to put the data inside.
Is there anyway to write the pandas dataframe to a specific sheet in an existing excel file? WITHOUT LOSING FORMATTING OF THE DESTIATION FILE
You need to just use to_excel from pandas dataframe.
Try below snippet:
df1.to_excel("output.xlsx",sheet_name='Sheet_name')
If there is existing data please try below snippet:
writer = pd.ExcelWriter('output.xlsx', engine='openpyxl')
# try to open an existing workbook
writer.book = load_workbook('output.xlsx')
df.to_excel(writer,index=False,header=False,startrow=len(reader)+1)
writer.save()
writer.close()
Are you restricted to using pandas or openpyxl?
Because if you're comfortable using other libraries, the easiest way is probably using win32com to puppet excel as if you were a user manually copying and pasting the information over.
import pandas as pd
import io
import win32com.client as win32
import os
csv_text = """Date,cust,region,Abr,Number
12/01/2010,Company_Name,Somecity,Chi,36
12/02/2010,Company_Name,Someothercity,Nyc,156"""
df = pd.read_csv(io.StringIO(csv_text),sep = ',')
temp_path = r"C:\Users\[User]\Desktop\temp.xlsx" #temporary location where to write this dataframe
df.to_excel(temp_path,index = False) #temporarily write this file to excel, change the output path as needed
excel = win32.Dispatch("Excel.Application")
excel.Visible = True #Switch these attributes to False if you'd prefer Excel to be invisible while excecuting this script
excel.ScreenUpdating = True
temp_wb = excel.Workbooks.Open(temp_path)
temp_ws = temp_wb.Sheets("Sheet1")
output_path = r"C:\Users\[User]\Desktop\output.xlsx" #Path to your output excel file
output_wb = excel.Workbooks.Open(output_path)
output_ws = output_wb.Sheets("Output_sheet")
temp_ws.Range('A1').CurrentRegion.Copy(Destination = output_ws.Range('A1')) # Feel free to modify the Cell where you'd like the data to be copied to
input('Check that output looks like you expected\n') # Added pause here to make sure script doesn't overwrite your file before you've looked at the output
temp_wb.Close()
output_wb.Close(True) #Close output workbook and save changes
excel.Quit() #Close excel
os.remove(temp_path) #Delete temporary excel file
Let me know if this achieves what you were after.
I spent all day on this (and a co-worker of mine spent even longer). Thankfully, it seems to work for my purposes - pasting a dataframe into an Excel sheet without changing any of the Excel source formatting. It requires the pywin32 package, which "drives" Excel as if it a user, using VBA.
import pandas as pd
from win32com import client
# Grab your source data any way you please - I'm defining it manually here:
df = pd.DataFrame([
['LOOK','','','','','','','',''],
['','MA!','','','','','','',''],
['','','I pasted','','','','','',''],
['','','','into','','','','',''],
['','','','','Excel','','','',''],
['','','','','','without','','',''],
['','','','','','','breaking','',''],
['','','','','','','','all the',''],
['','','','','','','','','FORMATTING!']
])
# Copy the df to clipboard, so we can later paste it as text.
df.to_clipboard(index=False, header=False)
excel_app = client.gencache.EnsureDispatch("Excel.Application") # Initialize instance
wb = excel_app.Workbooks.Open("Template.xlsx") # Load your (formatted) template workbook
ws = wb.Worksheets(1) # First worksheet becomes active - you could also refer to a sheet by name
ws.Range("A3").Select() # Only select a single cell using Excel nomenclature, otherwise this breaks
ws.PasteSpecial(Format='Unicode Text') # Paste as text
wb.SaveAs("Updated Template.xlsx") # Save our work
excel_app.Quit() # End the Excel instance
In general, when using the win32com approach, it's helpful to record yourself (with a macro) doing what you want to accomplish in Excel, then reading the generated macro code. Often this will give you excellent clues as to what commands you could invoke.
The solution to your problem exists here: How to save a new sheet in an existing excel file, using Pandas?
To add a new sheet from a df:
import pandas as pd
from openpyxl import load_workbook
import os
import numpy as np
os.chdir(r'C:\workdir')
path = 'output.xlsx'
book = load_workbook(path)
writer = pd.ExcelWriter(path, engine = 'openpyxl')
writer.book = book
### replace with your df ###
x = np.random.randn(100, 2)
df = pd.DataFrame(x)
df.to_excel(writer, sheet_name = 'x')
writer.save()
writer.close()
You can try xltpl.
Create a template file based on your output.xlsx file.
Render a file with your data.
from xltpl.writerx import BookWriterx
writer = BookWriterx('template.xlsx')
d = {'rows': df.values}
d['tpl_name'] = 'tpl_sheet'
d['sheet_name'] = 'temp_data'
writer.render_sheet(d)
d['tpl_name'] = 'other_sheet'
d['sheet_name'] = 'other'
writer.render_sheet(d)
writer.save('out.xls')
See examples.
I have a dataframe which I save it into an excel file at a certain location.
Currently I do this way:
df.to_excel(r'C:\Users\user_name\Downloads\test.xlsx')
Issue I am facing is when I insert the new dataframe it overwrites old ones. I want to append the new data. I tried several SOF answers but nothing seems to be working.
You can first read_excel, append and then write back to_excel:
filename = r'C:\Users\user_name\Downloads\test.xlsx'
existing = df.read_excel(filename)
output = existing.append(df)
output.to_excel(filename)
To check if the file exists before reading, you can use:
import os
filename = r'C:\Users\user_name\Downloads\test.xlsx'
if os.path.exists(filename):
existing = df.read_excel(filename)
output = existing.append(df)
else:
output = df
output.to_excel(filename)
One way to handle it is that you could read what is in Excel and combine it with your data frame, then overwrite the excel file/ generate it again basically.
Here's a sample of a similar question asked where a solution can be found with excel writer. Instead of overwriting the existing data, they just carefully set a startrow by reading the existing file for the startrow.
The last row/ start row can be found with the command writer.sheets[sheetname].max_row
append dataframe to excel with pandas
import pandas
from openpyxl import load_workbook
book = load_workbook('test.xlsx')
writer = pandas.ExcelWriter('test.xlsx', engine='openpyxl')
writer.book = book
writer.sheets = {ws.title: ws for ws in book.worksheets}
for sheetname in writer.sheets:
df1.to_excel(writer,sheet_name=sheetname, startrow=writer.sheets[sheetname].max_row, index = False,header= False)
writer.save()
My code works exactly like I would like it to by taking the data from the df and inserting it into the desired Excel file while skipping the appropriate rows. However, when I hit the .save() function other sheets that reference the data (mostly through pivots) seem to break even though they were not touched by the writer. I can insert the data into another Excel file, copy, and paste the exact same data where the python data puts it and the corresponding sheets do not break, but display the correct information. How do you stop other sheets from breaking when Python write to the file?
filename_in = 'File Location In'
filename_out = 'File Location Out'
sheet_name = 'Detail'
pos_detail_data_df.to_excel(filename_in, sheet_name=sheet_name, header = False, index = False)
df = pd.read_excel(filename_in, sheet_name=sheet_name)
book = load_workbook(filename_in)
writer = pd.ExcelWriter(filename_out, engine='openpyxl')
writer.book = book
writer.sheets = dict((ws.title, ws) for ws in book.worksheets)
writer.sheets
df.to_excel(writer, sheet_name, index=False, startrow = 2, header = False)
writer.save()
Edit:
The code was updated to reflect the assistance from below. However, now the process will simply remove everything from my filename_out and replace it with only the sheets from filename_in
I found an Excel file with a slicer so I took a look.
Sample file:
Site: https://www.contextures.com/excelpivottableslicers.html#download
Try:
import pandas as pd
from openpyxl import load_workbook
# sample Excel file with slicers.
# if required download and unzip and put in the folder with this script
sample_file = 'https://www.contextures.com/pivotsamples/regionsalesslicer.zip'
# set your filename_in, filename_out, and sheet_name
filename_in = 'regionsalesslicer.xlsx'
filename_out = 'regionsalesslicerUpdated.xlsx'
sheet_name = 'Sales Data'
# read in the Excel file with pd.read_excel rather than pd.ExcelFile
# just to play safe and avoid any BadZipFile: File is not a zip file errors
df = pd.read_excel(filename_in, sheet_name=sheet_name)
################## WHATEVER YOU WANT BELOW UNTIL LINE 37 ##################
# check the contents
print(df.head(2), '\n')
# make a change (or changes) to your df
# in the case just swap 'Carrot' for 'Orange' in the 'Product' column
df.loc[df['Product'] == 'Carrot', 'Product'] = 'Orange'
# check the contents after the change
print(df.head(2), '\n')
# as long as you have imported from the top two lines and read the file
# and not called ExcelWriter before this point all the other lines above
# are up to you.
################## WHATEVER YOU NEED ABOVE AFTER LINE 15 ##################
# from this point on try...
book = load_workbook(filename_in)
writer = pd.ExcelWriter(filename_out, engine='openpyxl')
writer.book = book
writer.sheets = dict((ws.title, ws) for ws in book.worksheets)
df.to_excel(writer, sheet_name, index=False)
writer.save()
In the resulting file (in the example code above we used filename_out = 'regionsalesslicerUpdated.xlsx'), the slicers still work.
Example:
Shows 'Orange'. Let's refresh the data...
Slicer/filter shows 'Orange'...
Exporting from pandas to Excel has not deleted any of the sheets etc...
We have successfully overwritten a dataframe to an existing sheet in Excel.
There is no way to do this if you are writing directly to the sheet unless you would like to pay for xlwings. A better (and easier to manage) solution is to change the way you are collecting your data from excel - Also, it won't break any dashboards or slicers you have. It will require some adjustments to your overall data pipeline and how you process it. Again, a one time thing that will pay dividends in the future.
Instead of writing directly to a sheet in the file, you can write to a separate file altogether.
df.to_excel(writer_path_to_seperate_sheet, sheet_name, index=False)
From excel you can now import this file (and every other file that you may write to the folder in the future) via power query.
Select either the file with your data, or preferably, the folder which will contain your file and all future files. Click combine and transform.
Once you complete this step, you can adjust your data set to your liking and load it. It will be a table by default (perfect for pivot tables and anything else). When new files are written to the folder, you simply click refresh on the table data set and wala. All slicers and other dashboard/pivots are left unaffected.
I am trying to use ExcelWriter to write/add some information into a workbook that contains multiple sheets.
First time when I use the function, I am creating the workbook with some data. In the second call, I would like to add some information into the workbook in different locations into all sheets.
def Out_Excel(file_name,C,col):
writer = pd.ExcelWriter(file_name,engine='xlsxwriter')
for tab in tabs: # tabs here is provided from a different function that I did not write here to keep it simple and clean
df = DataFrame(C) # the data is different for different sheets but I keep it simple in this case
df.to_excel(writer,sheet_name = tab, startcol = 0 + col, startrow = 0)
writer.save()
In the main code I call this function twice with different col to print out my data in different locations.
Out_Excel('test.xlsx',C,0)
Out_Excel('test.xlsx',D,10)
But the problem is that doing so the output is just the second call of the function as if the function overwrites the entire workbook. I guess I need to load the workbook that already exists in this case?
Any help?
Use load_book from openpyxl - see xlsxwriter and openpyxl docs:
import pandas as pd
from openpyxl import load_workbook
book = load_workbook('test.xlsx')
writer = pd.ExcelWriter('test.xlsx', engine='openpyxl')
writer.book = book
writer.sheets = dict((ws.title, ws) for ws in book.worksheets)
df.to_excel(writer, sheet_name='tab_name', other_params)
writer.save()
Pandas version 0.24.0 added the mode keyword, which allows you to append to excel workbooks without jumping through the hoops that we used to have to do. Just use mode='a' to append sheets to an existing workbook.
From the documentation:
with ExcelWriter('path_to_file.xlsx', mode='a') as writer:
df.to_excel(writer, sheet_name='Sheet3')
You could also try using the following method to create your Excel spreadsheet:
import pandas as pd
def generate_excel(csv_file, excel_loc, sheet_):
writer = pd.ExcelWriter(excel_loc)
data = pd.read_csv(csv_file, header=0, index_col=False)
data.to_excel(writer, sheet_name=sheet_, index=False)
writer.save()
return(writer.close())
Give this a try and let me know what you think.