I'm exporting results of my script into Excel spreadsheet. Everything works fine, I put big sets of data into SpreadSheet, but sometimes an error occurs:
File "C:\Python26\lib\site-packages\win32com\client\dynamic.py", line 550, in __setattr__
self._oleobj_.Invoke(entry.dispid, 0, invoke_type, 0, value)
pywintypes.com_error: (-2147352567, 'Exception.', (0, None, None, None, 0, -2146777998), None)***
I suppose It's not a problem of input data format. I put several different types of data strings, ints, floats, lists and it works fine. When I run the sript for the second time it works fine - no error. What's going on?
PS. This is code that generates error, what's strange is that the error doesn't occur always. Say 30% of runs results in an error. :
import win32com.client
def Generate_Excel_Report():
Excel=win32com.client.Dispatch("Excel.Application")
Excel.Workbooks.Add(1)
Cells=Excel.ActiveWorkBook.ActiveSheet.Cells
for i in range(100):
Row=int(35+i)
for j in range(10):
Cells(int(Row),int(5+j)).Value="string"
for i in range(100):
Row=int(135+i)
for j in range(10):
Cells(int(Row),int(5+j)).Value=32.32 #float
Generate_Excel_Report()
The strangest for me is that when I run the script with the same code, the same input many times, then sometimes an error occurs, sometimes not.
This is most likely a synchronous COM access error. See my answer to Error while working with excel using python for details about why and a workaround.
I can't see why the file format/extension would make a difference. You'd be calling the same COM object either way. My experience with this error is that it's more or less random, but you can increase the chances of it happening by interacting with Excel while your script is running.
edit: It doesn't change a thing. Error occurs, but leff often. Once in 10 simulations while with .xlsx file once in 3 simulations. Please help
The problem was with the file I was opening. It was .xlsx , while I've saved it as .xls the problem disappeared. So beware, do not ever use COM interface with .xlsx or You'll get in trouble !
You should diseable excel interactivity while doing this.
import win32com.client
def Generate_Excel_Report():
Excel=win32com.client.Dispatch("Excel.Application")
#you won't see what happens (faster)
Excel.ScreenUpdating = False
#clics on the Excel window have no effect
#(set back to True before closing Excel)
Excel.Interactive = False
Excel.Workbooks.Add(1)
Cells=Excel.ActiveWorkBook.ActiveSheet.Cells
for i in range(100):
Row=int(35+i)
for j in range(10):
Cells(int(Row),int(5+j)).Value="string"
for i in range(100):
Row=int(135+i)
for j in range(10):
Cells(int(Row),int(5+j)).Value=32.32 #float
Excel.ScreenUpdating = True
Excel.Interactive = True
Generate_Excel_Report()
Also you could do that to increase your code performance :
#Construct data block
string_line = []
for i in range(10)
string_line.append("string")
string_block = []
for i in range(100)
string_block.append(string_line)
#Write data block in one call
ws = Excel.Workbooks.Sheets(1)
ws.Range(
ws.Cells(35, 5)
ws.Cells(135,15)
).Values = string block
I had the same error while using xlwings for interacting with Excel. xlwings also use win32com clients in the backend.
After some debugging, I realized that this error pops up whenever the code is executed and the excel file (containing data) is not in focus. In order to resolve the issue, I simply select the file which is being processed and run the code and it always works for me.
Related
Update acknowledging comment 1 and 4:
Indentation corrected, does compile in Geany, obvious redundancies stripped.
The first question is: why am I getting the args error when I try to run my code? The error message refers to a LO file.... (linux) /opt/libreoffice7.2/programs/pythonscript.py # line 915.
An excerpt from that file says...
def invoke(self, args, out, outindex ): # 910
log.debug( "PythonScript.invoke " + str( args ) ) # 911
try: # 912
if (self.args): # 913
args += self.args # 915
ret = self.func( *args ) # 915
My code is...
# -*- coding: UTF-8 -*-
from __future__ import unicode_literals
# Gets the current document
doc = XSCRIPTCONTEXT.getDocument()
ctx = XSCRIPTCONTEXT.getComponentContext()
sm = CTX.ServiceManager
import time
import datetime
import serial
import openpyxl
def PySerLO (args=None):
from datetime import date
from openpyxl import load_workbook
ser = serial.Serial('/dev/ttyUSB0') # ""ls /dev/ttyUSB* -l"" if unsure.
# Load the workbook and select the sheet...
wb = load_workbook('/media/mpa/UserFiles/LibreOffice/mpaNEW-spreadsheet.xlsx')
sheet = wb['Sheet1']
headings = ("Date", "Time", "Raw Data")
sheet.append(headings)
try:
while True:
# Get current date, Get current time, Read the serial port
today = date.today()
now = datetime.datetime.now().time()
data = ser.readline().decode()
if data != "":
row = (today, now, float((data))) # For some reason the first data point is garbage so don't do anything with this
row = (today, ("%s"%now), float((data)))
sheet.append(row)
ser.flush()
#Save the workbook to preserve data
wb.save('/media/mpa/UserFiles/LibreOffice/mpaNEW-spreadsheet.xlsx')
finally:
# Make sure the workbook is saved at end
wb.save('/media/mpa/UserFiles/LibreOffice/mpaNEW-spreadsheet.xlsx')
print('Data Finished')
I get this error on executing a macro executing command button...
I appreciate your patience and suggestions on improvement thus far.
The ultimate aim of this is to track and instantly plot scientific data, introduce my students to the power of LibreOffice - They download LO and I just pass them the fully contained spreadsheet. They then connect the hardware and they have an excellent resource on LO forever. Almost none of them have ever heard of LibreOffice (or Linux either!)
First of all, I do not think you should start by embedding the macro in the document. I don't think it is required for the reasons you seem to believe. Instead, put your code in My Macros (the LibreOffice user directory) where it is easier to develop. Then call it from a button or by going to Tools > Macros > Run Macro - no need to use APSO as it seems like that is causing you even more confusion. The only reason for embedding is for convenience, to make it easier to keep the document and a bit of code together. For complex code that needs to be given to multiple people, it's generally better to create a full-fledged extension.
Second, it looks like the error may be from M1.py line 22 - is this your code? I cannot tell where line 22 might be or where exactly the error is coming from. Most of us reading your question are looking for the code where the error message comes from, but we cannot find it because you did not provide the relevant file name or line number of your code. It is your job to simplify your code enough so that you can see where the error is coming from, and then give that information in the question.
The error mentions create_instance which is the call for creating an UNO service.
Also I am not comfortable with the idea of storing XSCRIPTCONTEXT values in global variables unless you're sure you know what you're doing (and it's pretty clear you don't. :) )
Also I have strong doubts about achieving success by combining different libraries - it looks like you want to use both openpyxl and the UNO interface in the same code, and I would not expect that to turn out well. Just stick with plain UNO because there are plenty of people who can help with that. (Or if you choose openpyxl then I will ignore your question and someone knowledgeable about that may be able to help.)
Finally, it seems like import uno is missing. That alone could explain the error, and it's hard to imagine how you have all this code yet didn't even start with that.
Before trying all of this complex code, get a simple example working. It looks like there are a lot of python-uno fundamentals you still need to figure out first, and it doesn't look like you have put much effort into working through a tutorial yet.
I'm reading a csv and writing it out as a compressed csv. I use the following code
inp = pd.read_csv(inp_path)
inp.to_csv(filename, compression='gzip',encoding='utf-8')
And it just works fine
I need to rename the index as rownum, and I use the following code
inp = pd.read_csv(inp_path)
inp.index.names = ['rownum']
inp.to_csv(filename, compression='gzip',encoding='utf-8')
this leads to an error while reading the written file
Compressed file ended before the end-of-stream marker was reached
I'm doing this for 4 files, but I run into this issue for just one file.
Is there something wrong with what I'm doing ? Or is this a possible data issue ?
OR
Is there another way I can do this rename that will help me bypass this problem ?
EDIT
As suggested in the comments. Tried the following code
inp = pd.read_csv(inp_path)
row_count = len(inp)
index_row = range(0, row_count)
inp.insert(0, "rownum", index_row)
inp.to_csv(filename, compression='gzip',encoding='utf-8', index=False)
Still running into the same error mentioned above while trying to read the file
So, I am facing a weird issue, In my code I have the follow snippet:
workbook = xlwt.Workbook()
sheet = workbook.add_sheet("Sheet 1")
for x in range(len(table)):
for y in range(len(table[x])):
sheet.write(x, y, table[x][y])
workbook.save("output.xls")
which is from this question's answer: How do I export a two dimensional list in Python to excel?
where table is a multidimentional array (4x12 in my test code) that I want to convert to excel. I know that the code reaches the save and attempts to do, because if I put a directory it cannot save to due to insufficient admin-rights it will give me an error telling me about it, But if I run the code with a normal directory... nothing happens.
I am not sure where this is going wrong
I'm having a bizarre issue trying to write an xlsx file in Python.
I'm using Python 2.7.x and xlsxwriter to write xlsx files.
Here's a code snippet for context:
workbook = xlsxwriter.Workbook('filename.xlsx')
worksheet = workbook.add_worksheet('worksheet_name')
worksheet.write_row('A1', make_header_row) // <---- ROW 1
... // initialize "fields" array
worksheet.write_row('A2', fields) // <---- ROW 2
So here's the problem: Row 1 gets written, no problem. Row 2 never gets written... unless I stick a import pdb; pdb.set_trace() right above the line where I write Row 2. Waiting ~5 seconds in the pdb and then hitting continue will result in a successfully written second row.
I've tried flushing the workbook right after write_row, making sure the file is closed... nothing works.
Thanks for any help you can provide!
I am reading in a CSV as a Spark DataFrame and performing machine learning operations upon it. I keep getting a Python serialization EOFError - any idea why? I thought it might be a memory issue - i.e. file exceeding available RAM - but drastically reducing the size of the DataFrame didn't prevent the EOF error.
Toy code and error below.
#set spark context
conf = SparkConf().setMaster("local").setAppName("MyApp")
sc = SparkContext(conf = conf)
sqlContext = SQLContext(sc)
#read in 500mb csv as DataFrame
df = sqlContext.read.format('com.databricks.spark.csv').options(header='true',
inferschema='true').load('myfile.csv')
#get dataframe into machine learning format
r_formula = RFormula(formula = "outcome ~ .")
mldf = r_formula.fit(df).transform(df)
#fit random forest model
rf = RandomForestClassifier(numTrees = 3, maxDepth = 2)
model = rf.fit(mldf)
result = model.transform(mldf).head()
Running the above code with spark-submit on a single node repeatedly throws the following error, even if the size of the DataFrame is reduced prior to fitting the model (e.g. tinydf = df.sample(False, 0.00001):
Traceback (most recent call last):
File "/home/hduser/spark1.6/python/lib/pyspark.zip/pyspark/daemon.py", line 157,
in manager
File "/home/hduser/spark1.6/python/lib/pyspark.zip/pyspark/daemon.py", line 61,
in worker
File "/home/hduser/spark1.6/python/lib/pyspark.zip/pyspark/worker.py", line 136,
in main if read_int(infile) == SpecialLengths.END_OF_STREAM:
File "/home/hduser/spark1.6/python/lib/pyspark.zip/pyspark/serializers.py", line 545,
in read_int
raise EOFError
EOFError
The error appears to happen in the pySpark read_int function. Code for which is as follows from spark site :
def read_int(stream):
length = stream.read(4)
if not length:
raise EOFError
return struct.unpack("!i", length)[0]
This would mean that when reading 4bytes from the stream, if 0 bytes are read, EOF error is raised. The python docs are here.
I have faced the same issues and don't know how to debug it. seems that it will cause executor thread stuck and never return anything.
Have you checked to see where in your code the EOError is arising?
My guess would be that it's coming as you attempt to define df with, since that's the only place in your code that the file is actually trying to be read.
df = sqlContext.read.format('com.databricks.spark.csv').options(header='true',
inferschema='true').load('myfile.csv')
At every point after this line, your code is working with the variable df, not the file itself, so it would seem likely that this line is generating the error.
A simple way to test if this is the case would be to comment out the rest of your code, and/or place a line like this right after the line above.
print(len(df))
Another way would be to use a try loop, like:
try:
df = sqlContext.read.format('com.databricks.spark.csv').options(header='true',
inferschema='true').load('myfile.csv')
except:
print("Failed to load file into df!")
If it turns out that that line is the one generating the EOFError, then you're never getting the dataframes in the first place, so attempting to reduce them won't make a difference.
If that is the line generating the error, two possibilities come to mind:
Your code is calling one or both of the .csv files earlier on, and isn't closing it prior to this line. If so, simply close it above your code here.
There's something wrong with the .csv files themselves. Try loading them outside of this code, and see if you can get them into memory properly in the first place, using something like csv.reader, and manipulate them in ways you'd expect.