hey guys, I need help considering win32com in Python:
I have a routine that opens a Workbook, creates a sheet and puts some data on it.
If everything runs fine the woorkbook is saved and closed - If not the python session is terminated but the woorkbook is left open. So the reference is lost. Now when restarting the code Excel prompts you with the msg "workbook still open do you want to re-open?".
So what I want is to suppress this msg. I found a solution that works for me when python terminates before writing to the sheet:
open_copys = self.xlApp.Workbooks.Count
if open_copys > 0:
""" Check if any copy is the desired one"""
for i in range(0, open_copys):
if(self.xlApp.Workbooks[i].FullName == self.file_path):
self.xlBook = self.xlApp.Workbooks[i]
else:
self.xlBook = self.xlApp.Workbooks.Open(self.file_path)
But if any changes were made on the EXCEL sheet this method is obsolet.
Anyone got an ides how to get back a reference to an open and changed worksheet from a new python session?
thx
I'm not familiar with Python but have done some Excel/Word COM code in other languages.
Excel's Application.DisplayAlerts property might help. Setting it to False suppresses most messages that Excel might normally show, and auto-chooses a default response, though I think there are some exceptions.
Looking at your existing code, I guess you'd insert this line before opening the workbook:
self.xlApp.DisplayAlerts = False
Have you tried to remove all references to your COM objects before terminating the Python interpreter ? You can force them to be garbage collected (using gc.collect()) to be really sure they are gone. This way the workbook shouldn't remain open in memory and you won't have the error message.
Try adding a "close()" method to your class, with something like the following, and call it before the end of your script.
import gc
...
def close(self):
del self.xlApp
if hasattr(self, 'xlBook'):
del self.xlBook
gc.collect()
You're going about this the wrong way. You do NOT want to let Python terminate, leaving orphaned Excel processes. This is especially important if you are going to install and run this code on other machines. Instead, find your errors and handle them - then you'll never have orphaned processes to deal with.
That said, there are a few things you can consider. You can choose either to instantiate a new Excel process each time (Dispatch) or work with an existing one (DispatchEx). This lets you do things like see what workbooks are open and to close them, or ensures that your process will not interfere with others. Also as Scott said, the Excel Application has some interesting properties, like suppressing errors for unattended running, that are worth learning.
Related
I wrote a python script that backs up my files while I'm sleeping at night. The program is designed to run whenever the computer is on and to automatically shut down the computer after the backups are done. My code looks like this:
from datetime import datetime
from os import system
from backup import backup
while True:
today = datetime.now()
# Perform backups on Monday at 3:00am.
if today.weekday() == 0 and today.hour == 3:
print('Starting backups...')
# Perform backups.
backup("C:\\Users\\Jeff Moorhead\\Desktop", "E:\\")
backup("C:\\Users\\Jeff Moorhead\\Desktop", "F:\\")
backup("C:\\Users\\Jeff Moorhead\\OneDrive\\Documents", "E:\\")
backup("C:\\Users\\Jeff Moorhead\\OneDrive\\Documents", "F:\\")
# Shutdown computer after backups finish.
system('shutdown /s /t 10')
break
else:
del today
continue
The backup function is from another file that I wrote to perform more customized backups on a case by by case basis. This code all works perfectly fine, but I'm wondering if the del statement
del today
is really necessary. I put it in thinking that it would prevent my memory from getting filled up by thousands of datetime objects, but then I read that Python uses garbage collection, similar to Java. Further, does the todayvariable automatically get replaced with each pass through the while loop? I know that the program works as intended with the del statement, but if it is unnecessary, then I would like to get rid of it if only for the sake of brevity! What are it's actual effects on memory?
I put it in thinking that it would prevent my memory from getting filled up by thousands of datetime objects
The del statement is not necessary, you may simply remove that block. Python will free the space from those local variables automatically.
... but then I read that Python uses garbage collection, similar to Java.
The above statement is misguided: this has nothing to do with the garbage collector, which exists to break up circular references. In CPython, the memory is released when the object reference count decreases to zero, and that would occur even if the garbage collector is disabled.
Further, does the today variable automatically get replaced with each pass through the while loop? I know that the program works as intended with the del statement, but if it is unnecessary, then I would like to get rid of it if only for the sake of brevity! What are it's actual effects on memory?
A new datetime object is created on each iteration of the loop.
The today name in scope will be rebound to the newly created datetime instance. The old datetime instance will be deleted because no reference exists on it (since the only existing reference is lost once you rebound the name today to a different object). Once again, I stress that this is just ref-counting and has nothing to do with gc.
On an unrelated note, your program will busy-loop and consume an entire CPU with this while loop. You should consider adding a call to time.sleep into the loop so the process will remain mostly idle. Or, better yet, schedule the task to run periodically using cron.
I have a piece of code that opens a workbook by means of the xlwings package and reads some stuff from it. When done, the workbook is closed. Sometimes, errors occur between the opening and the closing statements. This leaves the workbook open. When I attempt to run the code again, the file cannot be opened as it is already 'open'.
How can I deal with this?
Also, I cannot even open manually the workbook when this happens.
w1=app.books.open("WorkbookName.xlsx")
# Do stuff that can potentially give errors.
w1.close()
I expect to create some error handling so that if an exception is found along the way, the workbook is closed after it.
Use a try-finally construct:
w1=app.books.open("WorkbookName.xlsx")
try:
# Do stuff that can potentially give errors.
finally:
w1.close()
A finally clause is always executed. If the code in the try clause raises an unhandled exception, then the python interpreter will jump to the finally clause and execute it (and terminate itself only afterwards).
I have read that when file is opened using the below format
with open(filename) as f:
#My Code
f.close()
explicit closing of file is not required . Can someone explain why is it so ? Also if someone does explicitly close the file, will it have any undesirable effect ?
The mile-high overview is this: When you leave the nested block, Python automatically calls f.close() for you.
It doesn't matter whether you leave by just falling off the bottom, or calling break/continue/return to jump out of it, or raise an exception; no matter how you leave that block. It always knows you're leaving, so it always closes the file.*
One level down, you can think of it as mapping to the try:/finally: statement:
f = open(filename)
try:
# My Code
finally:
f.close()
One level down: How does it know to call close instead of something different?
Well, it doesn't really. It actually calls special methods __enter__ and __exit__:
f = open()
f.__enter__()
try:
# My Code
finally:
f.__exit__()
And the object returned by open (a file in Python 2, one of the wrappers in io in Python 3) has something like this in it:
def __exit__(self):
self.close()
It's actually a bit more complicated than that last version, which makes it easier to generate better error messages, and lets Python avoid "entering" a block that it doesn't know how to "exit".
To understand all the details, read PEP 343.
Also if someone does explicitly close the file, will it have any undesirable effect ?
In general, this is a bad thing to do.
However, file objects go out of their way to make it safe. It's an error to do anything to a closed file—except to close it again.
* Unless you leave by, say, pulling the power cord on the server in the middle of it executing your script. In that case, obviously, it never gets to run any code, much less the close. But an explicit close would hardly help you there.
Closing is not required because the with statement automatically takes care of that.
Within the with statement the __enter__ method on open(...) is called and as soon as you go out of that block the __exit__ method is called.
So closing it manually is just futile since the __exit__ method will take care of that automatically.
As for the f.close() after, it's not wrong but useless. It's already closed so it won't do anything.
Also see this blogpost for more info about the with statement: http://effbot.org/zone/python-with-statement.htm
I'm trying to use a unix named pipe to output statistics of a running service. I intend to provide a similar interface as /proc where one can see live stats by catting a file.
I'm using a code similar to this in my python code:
while True:
f = open('/tmp/readstatshere', 'w')
f.write('some interesting stats\n')
f.close()
/tmp/readstatshere is a named pipe created by mknod.
I then cat it to see the stats:
$ cat /tmp/readstatshere
some interesting stats
It works fine most of the time. However, if I cat the entry several times in quick successions, sometimes I get multiple lines of some interesting stats instead of one. Once or twice, it has even gone into an infinite loop printing that line forever until I killed it. The only fix that I've got so far is to put a delay of let's say 500ms after f.close() to prevent this issue.
I'd like to know why exactly this happens and if there is a better way of dealing with it.
Thanks in advance
A pipe is simply the wrong solution here. If you want to present a consistent snapshot of the internal state of your process, write that to a temporary file and then rename it to the "public" name. This will prevent all issues that can arise from other processes reading the state while you're updating it. Also, do NOT do that in a busy loop, but ideally in a thread that sleeps for at least one second between updates.
What about a UNIX socket instead of a pipe?
In this case, you can react on each connect by providing fresh data just in time.
The only downside is that you cannot cat the data; you'll have to create a new socket handle and connect() to the socket file.
MYSOCKETFILE = '/tmp/mysocket'
import socket
import os
try:
os.unlink(MYSOCKETFILE)
except OSError: pass
s = socket.socket(socket.AF_UNIX)
s.bind(MYSOCKETFILE)
s.listen(10)
while True:
s2, peeraddr = s.accept()
s2.send('These are my actual data')
s2.close()
Program querying this socket:
MYSOCKETFILE = '/tmp/mysocket'
import socket
import os
s = socket.socket(socket.AF_UNIX)
s.connect(MYSOCKETFILE)
while True:
d = s.recv(100)
if not d: break
print d
s.close()
I think you should use fuse.
it has python bindings, see http://pypi.python.org/pypi/fuse-python/
this allows you to compose answers to questions formulated as posix filesystem system calls
Don't write to an actual file. That's not what /proc does. Procfs presents a virtual (non-disk-backed) filesystem which produces the information you want on demand. You can do the same thing, but it'll be easier if it's not tied to the filesystem. Instead, just run a web service inside your Python program, and keep your statistics in memory. When a request comes in for the stats, formulate them into a nice string and return them. Most of the time you won't need to waste cycles updating a file which may not even be read before the next update.
You need to unlink the pipe after you issue the close. I think this is because there is a race condition where the pipe can be opened for reading again before cat finishes and it thus sees more data and reads it out, leading to multiples of "some interesting stats."
Basically you want something like:
while True:
os.mkfifo(the_pipe)
f = open(the_pipe, 'w')
f.write('some interesting stats')
f.close()
os.unlink(the_pipe)
Update 1: call to mkfifo
Update 2: as noted in the comments, there is a race condition in this code as well with multiple consumers.
Root priv can't be dropped in python even after seteuid. A bug?
EDIT Summary: I forgot to drop gid. The accepted answer may help you, though.
Hi. I can't drop the root privilege in python 3.2 on my linux. In fact, even after seteuid(1000), it can read root-owned 400-mode files. The euid is surely set to 1000!
I found after empty os.fork() call, the privileged access is correctly denied. (But it's only in the parent. The child can still read illegitimately.) Is it a bug in python, or is linux so?
Try the code below. Comment out one of the three lines at the bottom, and run as root.
Thanks beforehand.
#!/usr/bin/python3
# Python seteuid pitfall example.
# Run this __as__ the root.
# Here, access to root-owned files /etc/sudoers and /etc/group- are tried.
# Simple access to them *succeeds* even after seteuid(1000) which should fail.
# Three functions, stillRoot(), forkCase() and workAround() are defined.
# The first two seem wrong. In the last one, access fails, as desired.
# ***Comment out*** one of three lines at the bottom before execution.
# If your python is < 3.2, comment out the entire def of forkCase()
import os
def stillRoot():
"""Open succeeds, but it should fail."""
os.seteuid(1000)
open('/etc/sudoers').close()
def forkCase():
"""Child can still open it. Wow."""
# setresuid needs python 3.2
os.setresuid(1000, 1000, 0)
pid = os.fork()
if pid == 0:
# They're surely 1000, not 0!
print('uid: ', os.getuid(), 'euid: ', os.geteuid())
open('/etc/sudoers').close()
print('open succeeded in child.')
exit()
else:
print('child pid: ', pid)
open('/etc/group-').close()
print('parent succeeded to open.')
def workAround():
"""So, a dummy fork after seteuid is necessary?"""
os.seteuid(1000)
pid = os.fork()
if pid == 0:
exit(0)
else:
os.wait()
open('/etc/group-').close()
## Run one of them.
# stillRoot()
# forkCase()
# workAround()
Manipulating process credentials on Unix systems is tricky. I highly recommend gaining a thorough understanding of how the Real, Effective, and Saved-Set user ids are interrelated. It's very easy to screw up "dropping privileges".
As to your specific observations... I'm wondering if there's a simple cause you may have overlooked. Your code is preforming a inconsistent tests and you've neglected to specify the exact file permissions on your /etc/sudoers and /etc/group- files. Your could would be expected to behave exactly as you describe if /etc/sudoers has permissions mode=440, uid=root, gid=root (which are the default permissions on my system) and if /etc/group- has mode=400.
You're not modifying the process's GID so if /etc/sudoers is group-readable, that would explain why it's always readable. fork() does not modify process credentials. However, it could appear to do so in your example code since you're checking different files in the parent and child. If /etc/group- does not have group read permissions where /etc/sudoers does, that would explain the apparent problem.
If all you're trying to do is "drop privileges", use the following code:
os.setgid( NEW_GID )
os.setuid( NEW_UID )
Generally speaking, you'll only want to manipulate the effective user id if your process needs to toggle it's root permissions on and off over the life of the process. If you just need to do some setup operations with root permissions but will no longer require them after those setup operations are complete, just use the code above to irrevokably drop them.
Oh, and a useful debugging utility for process credential manipulation on Linux is to print the output of /proc/self/status, the Uid and Gid lines of this file display the real, effective, saved-set, and file ids held by the current process (in that order). The Python APIs can be used to retrieve the same information but you can consider the contents of this file as "truth data" and avoid any potential complications from Python's cross-platform APIs.