We Keep Coding

python-win32com excel com model started generating errors

Over the last few days, I have been working on automating the generation of some pivot tables for a number of reports.
Boiled down to the minimum, the following code was working without issue:
import win32com.client
objExcelApp = win32com.client.gencache.EnsureDispatch('Excel.Application')
objExcelApp.Visible = 1
This would pop-up an instance of excel and I could continue working in Python. But suddenly, today my scripts are failing with the following:
>>>import win32com.client
>>> objExcelApp = win32com.client.gencache.EnsureDispatch('Excel.Application')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Program Files (x86)\Python37-32\lib\site-packages\win32com\client\gencache.py", line 534, in EnsureDispatch
mod = EnsureModule(tla[0], tla[1], tla[3], tla[4], bForDemand=bForDemand)
File "C:\Program Files (x86)\Python37-32\lib\site-packages\win32com\client\gencache.py", line 391, in EnsureModule
module = GetModuleForTypelib(typelibCLSID, lcid, major, minor)
File "C:\Program Files (x86)\Python37-32\lib\site-packages\win32com\client\gencache.py", line 266, in GetModuleForTypelib
AddModuleToCache(typelibCLSID, lcid, major, minor)
File "C:\Program Files (x86)\Python37-32\lib\site-packages\win32com\client\gencache.py", line 552, in AddModuleToCache
dict = mod.CLSIDToClassMap
AttributeError: module 'win32com.gen_py.00020813-0000-0000-C000-000000000046x0x1x9' has no attribute 'CLSIDToClassMap'
The code has not changed from yesterday to today. I have no idea what is happening!!!.
Another interesting kicker. if I do the same code in the same session again I get a different error:
>>> objExcelApp = win32com.client.gencache.EnsureDispatch('Excel.Application')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Program Files (x86)\Python37-32\lib\site-packages\win32com\client\gencache.py", line 534, in EnsureDispatch
mod = EnsureModule(tla[0], tla[1], tla[3], tla[4], bForDemand=bForDemand)
File "C:\Program Files (x86)\Python37-32\lib\site-packages\win32com\client\gencache.py", line 447, in EnsureModule
if module.MinorVersion != tlbAttributes[4] or genpy.makepy_version != module.makepy_version:
AttributeError: module 'win32com.gen_py.00020813-0000-0000-C000-000000000046x0x1x9' has no attribute 'MinorVersion'
>>>
So I jump over to a windows machine with a fresh windows install, install python37 and pip install pypiwin32. Run the very same lines and excel opens just like it did yesterday on my original machine.
I tried un-installing and re-installing with no luck. Any idea what is going on here?
NOTE:
Dynamic dispatch still works:
import win32com.client
objExcelApp = win32com.client.Dispatch("Excel.Application")
objExcelApp.Visible = 1
But I specifically need static dispatch as Pivot Tables won't work with a dynamically dispatched object (much later in my code):
objExcelPivotCache = objExcelWorkbook.PivotCaches().Create(SourceType=win32c.xlDatabase, SourceData=objExcelPivotSourceRange)

I had the same issue and I resolved it by following the instructions here: https://mail.python.org/pipermail/python-win32/2007-August/006147.html
Deleting the gen_py output directory and re-running makepy SUCCEEDS
and subsequently the test application runs OK again.
So the symptom is resolved, but any clues as to how this could have
happened. This is a VERY long running application (think 24x7 for
years) and I'm concerned that whatever caused this might occur again.
To find the output directory, run this in your python console / python session:
import win32com
print(win32com.__gen_path__)
Based on the exception message in your post, the directory you need to remove will be titled '00020813-0000-0000-C000-000000000046x0x1x9'. So delete this directory and re-run the code. And if you're nervous about deleting it (like I was) just cut the directory and paste it somewhere else.
💡Note that this directory is usually in your "TEMP" directory (copy-paste %TEMP%/gen_py in Windows File Explorer and you will arrive there directly).
I have no idea why this happens nor do I know how to prevent it from happening again, but the directions in the link I provided seemed to work for me.

A more straightforward solution was posted in a related question Issue in using win32com to access Excel file.
Basically, you just need to delete the folder C:\Users\<your username>\AppData\Local\Temp\gen_py and rerun your code.
💡TIP: You can also put in your Windows file explorer %TEMP%\gen_py to access it directly, and then delete its content.

Execute this command line in a powershell or cmd (NOT in Administrator mode => wouldn't work for me)
python -m win32com.client.makepy "Excel.Application"
It fixes all errors and you don't have to change your python code.
And keep using
win32com.client.gencache.EnsureDispatch("Excel.Application")
With gencache.EnsureDispatch you have access to the constants of the application loaded dynamically by makepy which must have the registered application (in our case Excel.Application).
If you have the same problem with Outlook, use "Outlook.Application" in above.
If still not working, reinstall pywin32 of your python distribution
<path to python root or venv>\pip.exe uninstall pywin32
<path to python root or venv>\pip.exe install pywin32

What has worked for me is:
excel = win32.gencache.EnsureDispatch('Excel.Application')
#change to =>
excel = win32.Dispatch('Excel.Application')

For me, it seems, the issue was that I have multiple processes that interact with Windows apps through win32com.
Since win32com creates the "gen_py" directory in win32api.GetTempPath() this can cause conflicts and the cache getting corrupted.
My solution is to set a custom location for "gen_py" for each process. A simple example:
from pathlib import Path
import win32com
gen_py_path = '/some/custom/location/gen_py'
Path(gen_py_path).mkdir(parents=True, exist_ok=True)
win32com.__gen_path__ = gen_py_path
# Any other imports/code that uses win32com
This way you don't have to delete the default "gen_py" folder and wonder what issues might arise. But if you still find you need to delete, you can just delete the custom folder and know you're deleting the cache just for that process.

To add to this discussion, for those receiving this error as part of an unsupervised automated process, you can fully automating the recovery process and allow any processes to continue unsupervised.
As answers by Ian and Qin mention, we need to delete the gen_py output directory and restart the process. As far as automation goes, there are two problems with this: the gen_py output directory is a temporary directory that can change, and currently running processes that rely on gen_py continue to fail even after it is regenerated.
To address these problems, we can dynamically look up the location, then we can completely nuke the whole process and restart it. I've tried deleting and re-importing win32com, but it seems a reference to the corrupted cache is still maintained, so the whole process needs to be restarted.
temp_data_dir = os.environ.get('LOCALAPPDATA'))
gen_py_dir = ''
for curr_path, dirs, files, in walk(temp_data_dir)
if 'gen_py' in dirs:
gen_py_dir = Path(curr_path).joinpath('gen_py')
shutil.rmtree(gen_py_dir)
execv(restart_args)
The restarted process should call win32com.client.gencache.EnsureDispatch('Excel.Application') again and a fresh copy of gen_py will now be utilized, then we can retry the failed Excel automation code.
The issue happens sporadically in a completely unreliable way that I can't replicate. My best guess is that the gen_py cache is sometimes corrupted somehow and just needs to be refreshed.

How to load DICOM files in Python?

I am trying to figure out the basics of importing DICOM files in Python using pydicom. While trying really simple code, I get following errors:
For code:
import dicom
filePath="C:\Python34\Lib\site-packages\dicom\testfiles"
ds=dicom.read_file(filePath[0])
I get error:
C:\Python34\python.exe C:/Users/041213/PycharmProjects/D/Deki.py
Traceback (most recent call last):
File "C:/Users/041213/PycharmProjects/D/Deki.py", line 4, in
ds=dicom.read_file(filePath[0])
File "C:\Python34\lib\site-packages\dicom\filereader.py", line 589, in read_file
fp = open(fp, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'C'
I am using Python 3.4, pydicom 0.9.9 and JetBrains PyCharm Community Edition 2016.3.2
If anyone can help me with this, or even just help me how to load a DICOM file in general, I would appreciate it a lot.

you're passing the first char of the string (C) instead of the full string. Just do:
ds=dicom.read_file(filePath)
next error you'll stumble into: use raw prefix or \t gets interpreted as a tabulation character:
filePath=r"C:\Python34\Lib\site-packages\dicom\testfiles"
^

Use and \ escape character to avoid issues with tab and other special characters. Also remember when you do filePath[0] on a string it returns the first character
filePath="C:\\Python34\\Lib\\site-packages\\dicom\\testfiles"
ds=dicom.read_file(filePath)

Error on Pytz - IndexError: list index out of range

I'm having this problem with ptyz:
>>> import pytz
>>> pytz.timezone('US/Mountain')
Traceback (most recent call last):
File "stdin", line 1, in module
File "/home/user/.local/lib/python2.6/site-packages/pytz-2013b-py2.6.egg/pytz/__init__.py", line 181, in timezone
_tzinfo_cache[zone] = build_tzinfo(zone, fp)
File "/home/user/.local/lib/python2.6/site-packages/pytz-2013b-py2.6.egg/pytz/tzfile.py", line 78, in build_tzinfo
while ttinfo[i][1]:
IndexError: list index out of range
I have two accounts in a server, and two different installations of pytz, though they are a copy of each other. In one of them, pytz works as a charm, but in the other one I have this error. It looks like an error inside ptyz, but it cannot be, because pytz works fine in the other account.
Has anyone handled this befor or have any idea of what may be causing this?
Update:
This error just happen when the script is called by a page, and that's the exact last message:
build_tzinfo(zone='US/Michigan', fp=<closed file '/home/user/.local/lib/python2...e-packages/pytz/zoneinfo/US/Michigan', mode 'rb'>)
and fp is a variable inside init.py that just receives it:
open(filename, 'rb')
It seems that the file is found, opened, but not really opened. I found at http://docs.python.org/2/tutorial/inputoutput.html that there are differences between linux and windows on the way they see binary files. My server runs Linux and, again, on the other account, with the same instalation, pytz works as a charm.

That's what happened:
After I updated my question, I noticed that the problem was with python opening a binary file. A friend told me to check the software I was using to send files to the server (Filezila), and there was three types of transfer: ASCII, Binary or Automatic. The software was set to Automatic. I send again the binary files in the binary transfer mode and it worked.

Is it possible to get writing access to raw devices using python with windows?

This is sort of a follow-up to this question. I want to know if you can access raw devices (i.e. \\.\PhysicalDriveN) in writing mode and if this should be the case, how.
Using Linux, write access can simply be achieved by using e.g. open("/dev/sdd", "w+") (provided that the script is running with root permissions). I assume that Mac OS behaves similar (with /dev/diskN as input file).
When trying the same command under Windows (with the corresponding path), it fails with the following error:
IOError: [Errno 22] invalid mode ('w+') or filename: '\\\\.\\PhysicalDrive3'
However, when trying to read from the PhysicalDrive, it does work (even the correct data is read). The shell is running with administrator permissions under Windows 7.
Is there any other way to accomplish this task using python while still keeping the script as platform-independent as possible?
Edit:
I looked a bit further into what methods python provides for file handling and stumbled across os.open. Opening the PhysicalDrive using os.open(drive_string, os.O_WRONLY|os.O_BINARY) returns no error. So far, so good. Now I have either the choice to write directly to this file-descriptor using os.write, or use os.fdopen to get a file-object and write to it in the regular way.
Sadly, none of these possibilities works. In the first case (os.write()), I get this:
>>> os.write(os.open("\\\\.\\PhysicalDrive3", os.O_WRONLY|os.O_BINARY), "test")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
OSError: [Errno 22] Invalid argument
In the second case, I can create a file object with write permissions, but the writing itself fails (well, after enforcing its execution using .flush()):
>>> g = os.fdopen(os.open("\\\\.\\PhysicalDrive3", os.O_WRONLY|os.O_BINARY), "wb")
>>> g.write("test")
>>> g.flush()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IOError: [Errno 22] Invalid argument

As eryksun and agf pointed out in the comments (but I didn't really get it at first), the solution is rather simple: you have to open the device in the rb+ mode, which opens the device for updating (as I have found out now..) without trying to replace it with a new file (which wouldn't work because the file is in fact a physical drive).
When writing, you have to write always a whole sector at a time (i.e. multiples of 512-byte), otherwise it fails.
In addition, the .seek() command can also jump only sector-wise. If you try to seek a position inside a sector (e.g. position 621), the file object will jump to the beginning of the sector where your requested position is (i.e. to the beginning of the second sector, byte 512).

Possibly in Win 7 you have to do something more extreme, such as locking the volume(s) for the disk beforehand with DeviceIoControl(hVol, FSCTL_LOCK_VOLUME, ...)
In Win 7 you don't have to do that; opening and writing with 'rb+' mode works fine.

WindowsError: priveledged instruction when saving a FreeImagePy Image in script, works in IDLE

I'm working on a program to do some image wrangling in Python for work. I'm using FreeImagePy because PIL doesn't support multi-page TIFFs. Whenever I try to save a file with it from my program I get this error message (or something similar depending on which way I try to save):
Error returned. TIFF FreeImage_Save: failed to open file C:/OCRtmp/ocr page0
Traceback (most recent call last):
File "C:\Python25\Projects\OCRPageUnzipper\PageUnzipper.py", line 102, in <mod
ule> OCRBox.convertToPages("C:/OCRtmp/ocr page",FIPY.FIF_TIFF)
File "C:\Python25\lib\site-packages\FreeImagePy\FreeImagePy\FreeImagePy.py", l
ine 2080, in convertToPages self.Save(FIF, dib, fileNameOut, flags)
File "C:\Python25\lib\site-packages\FreeImagePy\FreeImagePy\FreeImagePy.py", l
ine 187, in Save return self.__lib.Save(typ, bitmap, fileName, flags)
WindowsError: exception: priviledged instruction
When I try and do the same things from IDLE, it works fine.

Looks like a permission issues, make sure you don't have the file open in another application, and that you have write permissions to the file location your trying to write to.

That's what I thought too, but I figured it out a couple hours ago. Apparently if the directory/file I'm trying to write to doesn't exist, FreeImagePy isn't smart enough to create it (most of the time. Creating a new multipage image seems to work fine) but i guess running it within IDLE, IDLE figures it out and takes care of it or something. I managed to work around it by using os.mkdir to explicitly make sure things that I need exist.