We Keep Coding

How to loop across sheets in an Excel workbook

I am trying to learn Python and trying to generate a .sql file from a mapping given to me in an excel file. But I am able to go through the rows and columns of the first sheet but unable to move forward to the next sheets. I am using Python 3.7. Each sheet would contain two columns
This is what the excel sheet columns look like:
TABLE1 TABLE2
-----------------------------------
VERSION NULL
VALID_FROM USG_DT
USERNAME MIG
UD_GEPART_ID NULL
UD_SEPART_ID NULL
There can be multiple worksheets in one workbook.
Below is what I have tried so far. Please can someone point me out where I am going wrong in the code as I am not able to move to the next sheet:
wb = xl.load_workbook("C:\\New folder\\MAPPING.xlsx")
ws = wb.sheetnames
sheets = [wb.sheetnames]
statement_a = "INSERT /*+ APPEND */ INTO "
statement_b = "SELECT /*+ PARALLEL(64) */ "
for i, ws in enumerate(sheets):
stat_c = statement_b
stat_d = statement_b + ws[1+i] + " ( "
for sh in wb.worksheets:
sh = wb.active
for col_cells in sh.iter_cols(min_row= 2, min_col= 1, max_col=2):
for cell in col_cells:
stat_c = stat_c + str(cell.value) + ", "
stat_c = stat_c.rstrip(", ") +" FROM " + ws[1+i] + ";"
print(stat_c)
print("-------------------------------------------------")
for col_cellr in sh.iter_cols(max_row = 1,min_col= 2, max_col =2):
for cell in col_cellr:
stat_a = statement_a + cell.value + " ( "
for col_celli in sh.iter_cols(min_row= 2, min_col= 1, max_col=1):
for cell in col_celli:
stat_a = stat_a + str(cell.value) + ", "
stat_a = stat_a.rstrip(", ") + " )"
print("\n")
print(stat_a)
It does not provide me the correct output. For example it includes the second sheet name as the table name in select statement of the first worksheet columns. Please can someone help me by pointing out my mistake.
Expected o/p :
INSERT /*+ APPEND */ INTO TABLE1 (VERSION,VALID_FROM,USERNAME,UD_GEPART_ID, UD_SEPART_ID)
SELECT /*+ PARALLEL(64) */ NULL, USG_DT, MIG, NULL,NULL FROM TABLE2;
Similarly iterate over the other worksheets and generate statements as above.

sheetnames
Returns the list of the names of worksheets in this
workbook.
Type: list of strings
sheetnames returns a list. You should change
sheets = [wb.sheetnames]
to
sheets = wb.sheetnames
Also below is wrong. You are trying to iterate over each sheet but in the next step you lose sh by assigning it to something else.
for sh in wb.worksheets:
sh = wb.active

I think the following should help.
target_table, source_table, = ws.iter_rows(min_col=1, max_col=2, min_row=1 max_row=1, values_only)[0]
target_cols, source_cols, = ws.iter_cols(min_col=1, max_col=2, min_row=3, values)
target_cols.insert(0, "PARALLEL(64)")
stmt = """INSERT INTO {0} ({1})
SELECT {2} FROM {3}.format(target_table, ",".join(target_cols), ",".join(source_cols), source_table)
"""
This won't do everything and you'll probably want to adjust it but hopefully it does enough and if not gives you the basis for more specific questions.

Excel parser stuck on one row

So I was making a quick script to loop through a bunch of sheets in an excel file (22 to be exact) and what I wanted to do was the following:
Open the excel sheet and open the sheet named "All" which contained a list of names and then loop through each name and do the following
To loop through all the other 22 sheets in the same workbook and look through each one for the name, which I knew was in the 'B' column.
If the name were to be found, I wanted to take all the columns in that row containing the data for that name and these columns were from A-H
Then copy and paste them next to the original name (same row) in the 'All sheet' while leaving a bit of a space between the original name and the others.
I wanted to do this for all 22 sheets and for the 200+ names listed in the 'All' sheet, my code is as follows:
import openpyxl, pprint
columns = ['A','B','C','D','E','F','G','H']
k = 10
x = 0
def colnum_string(n):
string = ""
while n > 0:
n, remainder = divmod(n - 1, 26)
string = chr(65 + remainder) + string
return string
print("Opening Workbook...")
wb = openpyxl.load_workbook('FileName.xlsx')
sheet_complete = wb.get_sheet_by_name("All")
row_count_all = sheet_complete.max_row
for row in range(4, row_count_all+1):
k = 10
cell = 'B' + str(row)
print(cell)
name = sheet_complete[cell].value
for i in range(2, 23):
sheet = wb.get_sheet_by_name(str(1995 + i))
row_count = sheet.max_row
for row2 in range(2, row_count+1):
cell2 = 'B' + str(row2)
name2 = sheet[cell].value
if name == name2:
x = x + 1
for z in range(0,len(columns)):
k = k + 1
cell_data = sheet[columns[z] + str(row2)].value
cell_target = colnum_string(k) + str(row)
sheet_complete[cell_target] = cell_data
wb.save('Scimago Country Ranking.xlsx')
print("Completed " + str(x) + " Task(s)")
break
The problem is that it keeps looping with the first name only, so it goes through all the names but when it comes to copying and pasting the data, it just redoes the first name so in the end, I end up with all the names in the 'All' sheet and next to each one is the data for the first name repeated over and over. I can't see what's wrong with my code but forgive me if it's a silly mistake as I'm kind of a beginner in these excel parsing scripts. print statements were for testing reasons.
P.S I know I'm using a deprecated function and I will change that, I was just too lazy to do it since it seems to still work fine and if that's the problem then please let me know.

Copying an entire column using OpenPyXL in Python 3

I'm trying to copy an entire column over using OpenPyXL. Google seems to offer a lot of examples using ranges, but not for an entire column.
I have a workbook with a single worksheet with a load of dates in column A and column JX (A contains monthly dates, JX contains quarterly dates). I want the monthly dates column (in A:A) to be copied over to each worksheet ending in 'M' in my target workbook, and the quarterly dates column (in JX:JX) to the worksheets ending in 'Q'.
However, for some reason the last nested for loop, for src, dst in zip(ws_base[monthRange], ws_target['A:A']): is only copying the first cell, and nothing else. It looks like I'm identifying the correct column with my monthRange and quarterRange strings, but Python isn't looping through the whole column despite the fact that I've got two ranges defined.
Does anyone have any ideas?
# Load the target workbook
targetwb = openpyxl.load_workbook('pythonOutput.xlsx')
# Load the source workbook
wb_base = openpyxl.load_workbook('Baseline_IFRS9_' + reportingMonth+'.xlsx')
# Go to row 9 and find "Geography:" to identify the relevant
# month and quarter date columns
sentinel = u"Geography:"
ws_base = wb_base.active
found = 0
dateColumns = []
for column in ws_base:
for cell in column:
if cell.value == sentinel:
dateColumns.append(cell.column) #
found + 1
if found == 2:
break
ColumnM = dateColumns[0]
ColumnQ = dateColumns[1]
print('Monthly col is ' + ColumnM)
print('Quarterly col is ' + ColumnQ)
IndexM = int(openpyxl.utils.column_index_from_string(str(ColumnM)))
IndexQ = int(openpyxl.utils.column_index_from_string(str(ColumnQ)))
print('Monthly col index is ' + str(IndexM))
print('Quarterly col index is ' + str(IndexQ))
print('Proceeding to paste into our new workbook...')
sheetLoop = targetwb.get_sheet_names()
for sheets in sheetLoop:
if sheets.endswith('Q'):
ws_target = targetwb[sheets]
quarterRange = ColumnQ + ':' + ColumnQ
print('Copying and pasting quarterly dates into: ' + sheets)
for src, dst in zip(ws_base[quarterRange], ws_target['A:A']):
dst.value = src.value
elif sheets.endswith('M'):
ws_target = targetwb[sheets]
monthRange = ColumnM + ':' + ColumnM
print('Copying and pasting monthly dates into: ' + sheets)
for src, dst in zip(ws_base[monthRange], ws_target['A:A']):
dst.value = src.value
targetwb.save('pythonOutput.xlsx')
Here's a simpler form of my problem.
import openpyxl
wb1 = openpyxl.load_workbook('pythonInput.xlsx')
ws1 = wb1.active
wb2 = openpyxl.load_workbook('pythonOutput.xlsx')
ws2 = wb2.active
for src, dst in zip(ws1['A:A'], ws2['B:B']):
print( 'Printing from ' + str(src.column) + str(src.row) + ' to ' + str(dst.column) + str(dst.row))
dst.value = src.value
wb2.save('test.xlsx')
So the problem here is that the for loop only prints from A1 to B1. Shouldn't it be looping down across rows..?

When you load a new XLSX in a spreadsheet editor, you see lots and lots of empty cells in a grid. However, these empty cells are actually omitted from the file, and they will be only written once they have a non-empty value. You can see for yourself: XLSX is essentially a bunch of ZIP-compressed XMLs, which can be opened with any archive manager.
In a similar fashion, new cells in OpenPyXL are only created when you access them. The ws2['B:B'] range only contains one cell, B1, and zip stops when the shortest iterator is exhausted.
With this in mind, you can iterate through the source range and use explicit coordinates to save the values in correct cells:
import openpyxl
wb1 = openpyxl.load_workbook('pythonInput.xlsx')
ws1 = wb1.active
wb2 = openpyxl.load_workbook('pythonOutput.xlsx')
ws2 = wb2.active
for cell in ws1['A:A']:
print('Printing from ' + str(cell.column) + str(cell.row))
ws2.cell(row=cell.row, column=2, value=cell.value)
wb2.save('test.xlsx')

comparing one column in two spreadsheets in openpyxl

I am making an excel comparing program but I seem to be stuck. I want to compare two excel files in a spreadsheet. Here is my code:
import openpyxl
wb = openpyxl.load_workbook('C:\\Users\\Bill\\Desktop\\CK_Server_list_0.1.xlsx')
ws = wb.active
wb1 =
openpyxl.load_workbook('C:\\Users\\Bill\\Desktop\\CK_Server_list_0.2.xlsx')
ws1 = wb1.active
for x in ws.iter_cols(max_col=1):
for cell in x:
print(cell.value, cell.coordinate)
for row1 in ws1.iter_cols(min_col=1):
if row1[0].value != ws.cell(row=x, column=1).value:
print(str(row1[0].value) + ' is not equal to ' + str(ws.cell(row=x, column=1).value + ' ' + str(ws.cell(row=x, column=1).coordinate)))
And every time I run this it gives me an error saying that tuple() < int(). Can anyone fix this problem? Any help would be appreciated.

This error pops up because your variable x contains a tuple of cell objects at the time when the line if row1[0].value != ws.cell(row=x, column=1).value: gets executed. The input argument row requires an int value instead.
I think that a good approach for your problem would be to use for loops in combination with zip statements (more on zip here):
import openpyxl
wb = openpyxl.load_workbook('C:\\Users\\Bill\\Desktop\\CK_Server_list_0.1.xlsx')
ws = wb.active
wb1 = openpyxl.load_workbook('C:\\Users\\Bill\\Desktop\\CK_Server_list_0.2.xlsx')
ws1 = wb1.active
for (col, col_1) in zip(ws.iter_cols(), ws1.iter_cols()):
for (cell, cell_1) in zip(col, col_1):
if cell.value != cell_1.value:
print(str(cell.value) + ' is not equal to ' + str(cell_1.value) + ' ' + str(cell.coordinate))

Python csv, if check using header

I have a file daiy.csv which is updated on daily basis. The script fetches values out of it, checks against some values and then prints out some info based on those checks. My problem is that almost once a week, the column numbers in daily.csv either increase or decrease. Then I have to find the new column number for my desired values. I was wondering if there is a way I can use header values in my if checks so regardless the column numbers change, I can still run my script.
with open('daily.csv','rb')as f:
reader=csv.reader(f)
#next(reader, None) #Skipping the header
for row in reader:
#if row[3]=='M2' and (float(row[38]) > 60):
try:
if (row[3]=='M2' or row[3]=='M4' or row[3]=='M3') and (float(row[37]) > 60):
print row[1] + "/" + row[2] + "/" + row[3] + " : " + row[37]
if (row[3]=='M2' or row[3]=='M4' or row[3]=='M3') and (float(row[37]) < 70):
print row[1] + "/" + row[2] + "/" + row[3] + " : " + row[37]
except:
pass

This seems like a problem that pandas (http://pandas.pydata.org/) can easily help you solve. Assuming that all of your columns have unique headers you can just extract the relevant columns you care about. Let's say that your column names are 'A', 'B', and 'C' and you don't want to analyze column 'B'.
import pandas as pd
df = pd.read_csv('daily.csv')
columns_to_analyze = ['A', 'C']
df = df[columns_to_analyze]
Without knowing more specifics of how 'daily.csv' is formatted that's as much as I can confidently say...