How to run a python code for large inputs - python

In my code,
Samp_size
MSI
MRI
M_ASRS
a_h
d_h
a_v
d_v
max_hor_vel
max_ver_vel
These are randomly generated parameters above.Each can all take different numbers of values, lets say each has 2 different value.
I print them as dataframes to an excel file each in different sheets.(sheet1,sheet2 etc.)
So I have 2^10 different parameter set. How can I print the all the solutions for all this parameter set in order to an excel file?

It seems like you want to combine sheets into one single excel file, if so, the following post will solve your problem: Combine Multiple Excel sheets within Workbook into one sheet Python
The code from # And then append all the Workbooks into single Excel Workbook sheet will help.

Related

Opening an Excel File in Python Disables Dynamic Arrays

I have an excel workbook that uses functions like OFFSET, UNIQUE, and FILTER which spill into other cells. I'm using python to analyze and write some data to the workbook, but after doing so these formulas revert into normal arrays. This means they now take up a fixed number of cells (however many they took up before opening the file in python) instead of adjusting to fit all of the data. I can revert the change by selecting the formula and hitting enter, but there are many of these formulas it's more work to fix them than to just print the data to a text file and paste it into excel manually. Is there any way to prevent this behavior?
I've been using openpyxl to open and save the workbook, but after encountering this issue also tried xlsxwriter and the dataframe to excel function from pandas. Both of them had the same issue as openpyxl. For context I am on python 3.11 and using the most recent version of these modules. I believe this issue is on the Python side and not the Excel side, so I don't think changing Excel settings will help, but maybe there is something there I missed.
Example:
I've created an empty workbook with two sheets, one called 'main' and one called 'input'. The 'main' sheet will analyze data from the 'input' sheet which will be entered with openpyxl. The data will just be values in the first column.
In cell A1 of the 'main' sheet, enter =OFFSET(input!A1,0,0,COUNTA(input!A:A),1).
This formula will just show a copy of the data. Since there currently isn't any data it gives a #REF! error, so it only takes up one cell.
Now I'll run the following python code to add the numbers 0-9 into the first column of the input sheet:
from openpyxl import load_workbook
wb = load_workbook('workbook.xlsx')
ws = wb['input']
for i in range(10):
ws.append([i])
wb.save('workbook_2.xlsx')
When opening the new file, cell A1 on the 'main' sheet only has the first value, 0, instead of the range 0--9. When selecting the cell, you can see the formula is now {=OFFSET(input!A1,0,0,COUNTA(input!A:A),1)}. The curly brackets make it an array, so it wont spill. By hitting enter in the formula the array is removed and the sheet properly becomes the full range.
If I can get this simple example to work, then expanding it to the data I'm using shouldn't be a problem.

Merge particular sheets from multiple workbooks

I have a folder with 8 workbooks with multiple sheets. I want to rearrange columns from the sheet named RAW from all workbooks and combine all the RAW sheets into one sheet as Final_Raw.
I need a macro code to achieve this also can this be automated using python?
It is possible to do in VBA. You need to collect the data from the sheets. This means you declare all the sheets like:
Sub getdata()
Dim strLocation As String
Dim objWorkbookOne As Workbook
Dim wsData As Worksheet
Dim intFR, intLR As Integer
strLocation = "C:\Users\fredd\Documents\"
Set objWorkbookOne = Workbooks.Open(strLocation & "14082022194559_download_MEDEWERKER.xlsx")
Set wsData = ThisWorkbook.Sheets(1)
wsData.Activate
intFR = 1
intLR = objWorkbookOne.Worksheets("MEDEWERKER").Cells(Rows.Count, 1).End(xlUp).Row
For intFR = 1 To intLR
wsData.Cells(intFR, 1) = objWorkbookOne.Worksheets("MEDEWERKER").Cells(intFR, 1)
Next intFR
End Sub
In the code above we get data from a file named 14082022194559_download_MEDEWERKER.xlsx on location *C:\Users\fredd\Documents*. I made a variable of the location so it is easy to change if nessesary. The file is opened in objWorkbookOne (ofcourse you can do this for eight workbooks as well).
When the workbook is opened, we activate the sheet in which we want to 'paste' the data. Next the first row (intFR) and last Row (intLR) are defined in workbookone. With that FOR loop you can 'copy' the data to the masterfile.
I don't know exactly how your masterfile and other files are build up, so the I have to make assumptions. In this code (above) I copy one column to another column, but this is also possible with ranges.

How do I fill [] with sheet information from pandas and xlrd?

I have this:
dic_sheets = {}
for y in xl_files[]
dic_sheets.update({y:[]})
I want to populate the tables in the dictionary (dic_sheets) for each key(y) with the individual sheets inside of the excel document.
I do not know how many sheets are inside of the excel document; I don't have an index number to stop a range (x,y,z) loop.
Another way to put it: I want to dump x-number of excel files into the active directory and have each files sheets populate in a dictionary when I run the .py in CMD.
Can anyone help me achieve this goal?
xl_files contains "ExcelFile" data "pandas.io.excel.ExcelFile object at 0x0FF6B0D0
Edit: y represents individual excel files
Edit2: I need only the sheet names (or their unique index numbers) to populate, (i.e. 'sheet1', 'pivot2'). I'm not yet concerned with cells in the sheets.
Edit3: I already have the table ‘xl_files’ generated to contain every excel file in the cwd
I figured it out!
I had to use a for loop and the return function as an object, then combine it with another object of the array.append function and return function with a new array.
I'll try to word my questions better in the future, as I did not get a bite this round.

Use a value from a cell which is written with formulas

I have two columns in Excel. The first(column C) has cells with values, the second one(column B), I had used a script to extract some values from the first one with Excel formulas.
Now I want to use the values from the second column in another column and the script doesn't have any errors but gives me empty cells because the second column contains formulas.
Is it possible to paste values or to extract only the values from the second column?
Here is my code:
for i in range(0,len(listaunica)):
ws4.cell(row=i+1,column=3).value=listaunica[i]
for i in range(0,len(listaunica)):
ws4.cell(row=i+1,column=2).value='=iferror(find(".",C{0}),C{0})'.format(i+1)
Can someone help me with this?
I do not fully understand your situation, so I will explain some possibilities:
(1) You have an Excel workbook that was saved using Excel itself. In this case, column B should have both formulas and the results of those formulas, because Excel would have calculated them.
(2) You have an Excel workbook that was saved using some other method, such as being written by OpenPyXL, and has not (yet) been opened and saved by Excel. In this case, you most likely have either formulas or results stored in column B.
When you are reading using OpenPyXL, you have to choose whether you want formulas or results. This is controlled by the data_only parameter. Set this to True if you want just the results. If your workbook was saved in Excel, and thus has both formulas and results, then the way to read them both in OpenPyXL is to open the workbook twice, once with data_only=False and once with data_only=True. Cumbersome, but that is how OpenPyXL is designed.
If you have a workbook from scenario (2), and column B still looks like it has formulas, then most likely trying to open the workbook using data_only=True will just return zeros for column B. You won't be able to get the results from this workbook until you open it in Excel and then save it.
Try this
for i in range(0,len(listaunica)):
ws4.cell(row=i+1,column=3).value=listaunica[i]
for i in range(0,len(listaunica)):
ws4.cell(row=i+1,column=2).value='=iferror(find(".",C{0}),C{0})'.format(i+1)
ws4.cell(row=i+1,column=2).value = ws4.cell(row=i+1,column=2).value
For reference Does .Value = .Value act similar to Evaluate() function in VBA??

Use excel workbook like a function

I found this on a related subject - Excel function to make SQL-like queries on worksheet data?
But I was wondering if there is any way to use an excel workbook/file like a function in a separate workbook/file? So I have an excel workbook that has a control page - where I can input parameters 1,2,3. And based on those parameter, the outpage page will display the correlating data. I know I can duplictae the output page to show the outputs for all three parameters and link these to my other workbook.
However, is there any way to (in the other excel file) do something like FILENAME_otherfile.function(1).range(A1) or something? to extract the cell A1 with parameter 1 as an input? And in the same file also call FILENAME_otherfile.function(2).range(A1)?
There is a very inelegant way of doing it, but we can keep the interface as pretty as possible.
Open the workbook with a Application.Workbooks.Open()
Set its attribute to Hidden for aesthetic reasons.
Say you want to manipulate cells on "Sheet1" then Set InputRange and OutputRange (cells where you'll see the outputs) to the appropriate ranges in the opened workbook.
Change InputRange.Cells(m,n).Value
Get into some variable what happens, for e.g.
Dim MyAnswer as Double
MyAnswer = OutputRange.Cells(x,y)
Close the workbook, preferably not saving it (as your programming logic requires) and use the MyAnswer value as your 'function' output.

Categories