I'm getting an error in a program that is supposed to run for a long time that too many files are open. Is there any way I can keep track of which files are open so I can print that list out occasionally and see where the problem is?
To list all open files in a cross-platform manner, I would recommend psutil.
#!/usr/bin/env python
import psutil
for proc in psutil.process_iter():
print(proc.open_files())
The original question implicitly restricts the operation to the currently running process, which can be accessed through psutil's Process class.
proc = psutil.Process()
print(proc.open_files())
Lastly, you'll want to run the code using an account with the appropriate permissions to access this information or you may see AccessDenied errors.
I ended up wrapping the built-in file object at the entry point of my program. I found out that I wasn't closing my loggers.
import io
import sys
import builtins
import traceback
from functools import wraps
def opener(old_open):
#wraps(old_open)
def tracking_open(*args, **kw):
file = old_open(*args, **kw)
old_close = file.close
#wraps(old_close)
def close():
old_close()
open_files.remove(file)
file.close = close
file.stack = traceback.extract_stack()
open_files.add(file)
return file
return tracking_open
def print_open_files():
print(f'### {len(open_files)} OPEN FILES: [{", ".join(f.name for f in open_files)}]', file=sys.stderr)
for file in open_files:
print(f'Open file {file.name}:\n{"".join(traceback.format_list(file.stack))}', file=sys.stderr)
open_files = set()
io.open = opener(io.open)
builtins.open = opener(builtins.open)
On Linux, you can look at the contents of /proc/self/fd:
$ ls -l /proc/self/fd/
total 0
lrwx------ 1 foo users 64 Jan 7 15:15 0 -> /dev/pts/3
lrwx------ 1 foo users 64 Jan 7 15:15 1 -> /dev/pts/3
lrwx------ 1 foo users 64 Jan 7 15:15 2 -> /dev/pts/3
lr-x------ 1 foo users 64 Jan 7 15:15 3 -> /proc/9527/fd
Although the solutions above that wrap opens are useful for one's own code, I was debugging my client to a third party library including some c extension code, so I needed a more direct way. The following routine works under darwin, and (I hope) other unix-like environments:
def get_open_fds():
'''
return the number of open file descriptors for current process
.. warning: will only work on UNIX-like os-es.
'''
import subprocess
import os
pid = os.getpid()
procs = subprocess.check_output(
[ "lsof", '-w', '-Ff', "-p", str( pid ) ] )
nprocs = len(
filter(
lambda s: s and s[ 0 ] == 'f' and s[1: ].isdigit(),
procs.split( '\n' ) )
)
return nprocs
If anyone can extend to be portable to windows, I'd be grateful.
On Linux, you can use lsof to show all files opened by a process.
As said earlier, you can list fds on Linux in /proc/self/fd, here is a simple method to list them programmatically:
import os
import sys
import errno
def list_fds():
"""List process currently open FDs and their target """
if not sys.platform.startswith('linux'):
raise NotImplementedError('Unsupported platform: %s' % sys.platform)
ret = {}
base = '/proc/self/fd'
for num in os.listdir(base):
path = None
try:
path = os.readlink(os.path.join(base, num))
except OSError as err:
# Last FD is always the "listdir" one (which may be closed)
if err.errno != errno.ENOENT:
raise
ret[int(num)] = path
return ret
On Windows, you can use Process Explorer to show all file handles owned by a process.
There are some limitations to the accepted response, in that it does not seem to count pipes. I had a python script that opened many sub-processes, and was failing to properly close standard input, output, and error pipes, which were used for communication. If I use the accepted response, it will fail to count these open pipes as open files, but (at least in Linux) they are open files and count toward the open file limit. The lsof -p solution suggested by sumid and shunc works in this situation, because it also shows you the open pipes.
Get a list of all open files. handle.exe is part of Microsoft's Sysinternals Suite. An alternative is the psutil Python module, but I find 'handle' will print out more files in use.
Here is what I made. Kludgy code warning.
#!/bin/python3
# coding: utf-8
"""Build set of files that are in-use by processes.
Requires 'handle.exe' from Microsoft SysInternals Suite.
This seems to give a more complete list than using the psutil module.
"""
from collections import OrderedDict
import os
import re
import subprocess
# Path to handle executable
handle = "E:/Installers and ZIPs/Utility/Sysinternalssuite/handle.exe"
# Get output string from 'handle'
handle_str = subprocess.check_output([handle]).decode(encoding='ASCII')
""" Build list of lists.
1. Split string output, using '-' * 78 as section breaks.
2. Ignore first section, because it is executable version info.
3. Turn list of strings into a list of lists, ignoring first item (it's empty).
"""
work_list = [x.splitlines()[1:] for x in handle_str.split(sep='-' * 78)[1:]]
""" Build OrderedDict of pid information.
pid_dict['pid_num'] = ['pid_name','open_file_1','open_file_2', ...]
"""
pid_dict = OrderedDict()
re1 = re.compile("(.*?\.exe) pid: ([0-9]+)") # pid name, pid number
re2 = re.compile(".*File.*\s\s\s(.*)") # File name
for x_list in work_list:
key = ''
file_values = []
m1 = re1.match(x_list[0])
if m1:
key = m1.group(2)
# file_values.append(m1.group(1)) # pid name first item in list
for y_strings in x_list:
m2 = re2.match(y_strings)
if m2:
file_values.append(m2.group(1))
pid_dict[key] = file_values
# Make a set of all the open files
values = []
for v in pid_dict.values():
values.extend(v)
files_open = sorted(set(values))
txt_file = os.path.join(os.getenv('TEMP'), 'lsof_handle_files')
with open(txt_file, 'w') as fd:
for a in sorted(files_open):
fd.write(a + '\n')
subprocess.call(['notepad', txt_file])
os.remove(txt_file)
I'd guess that you are leaking file descriptors. You probably want to look through your code to make sure that you are closing all of the files that you open.
You can use the following script. It builds on Claudiu's answer. It addresses some of the issues and adds additional features:
Prints a stack trace of where the file was opened
Prints on program exit
Keyword argument support
Here's the code and a link to the gist, which is possibly more up to date.
"""
Collect stacktraces of where files are opened, and prints them out before the
program exits.
Example
========
monitor.py
----------
from filemonitor import FileMonitor
FileMonitor().patch()
f = open('/bin/ls')
# end of monitor.py
$ python monitor.py
----------------------------------------------------------------------------
path = /bin/ls
> File "monitor.py", line 3, in <module>
> f = open('/bin/ls')
----------------------------------------------------------------------------
Solution modified from:
https://stackoverflow.com/questions/2023608/check-what-files-are-open-in-python
"""
from __future__ import print_function
import __builtin__
import traceback
import atexit
import textwrap
class FileMonitor(object):
def __init__(self, print_only_open=True):
self.openfiles = []
self.oldfile = __builtin__.file
self.oldopen = __builtin__.open
self.do_print_only_open = print_only_open
self.in_use = False
class File(self.oldfile):
def __init__(this, *args, **kwargs):
path = args[0]
self.oldfile.__init__(this, *args, **kwargs)
if self.in_use:
return
self.in_use = True
self.openfiles.append((this, path, this._stack_trace()))
self.in_use = False
def close(this):
self.oldfile.close(this)
def _stack_trace(this):
try:
raise RuntimeError()
except RuntimeError as e:
stack = traceback.extract_stack()[:-2]
return traceback.format_list(stack)
self.File = File
def patch(self):
__builtin__.file = self.File
__builtin__.open = self.File
atexit.register(self.exit_handler)
def unpatch(self):
__builtin__.file = self.oldfile
__builtin__.open = self.oldopen
def exit_handler(self):
indent = ' > '
terminal_width = 80
for file, path, trace in self.openfiles:
if file.closed and self.do_print_only_open:
continue
print("-" * terminal_width)
print(" {} = {}".format('path', path))
lines = ''.join(trace).splitlines()
_updated_lines = []
for l in lines:
ul = textwrap.fill(l,
initial_indent=indent,
subsequent_indent=indent,
width=terminal_width)
_updated_lines.append(ul)
lines = _updated_lines
print('\n'.join(lines))
print("-" * terminal_width)
print()
Related
I my application, i have below requests:
1. There has one thread will regularly record some logs in file. The log file will be rollovered in certain interval. for keeping the log files small.
2. There has another thread also will regularly to process these log files. ex: Move the log files to other place, parse the log's content to generate some log reports.
But, there has a condition is the second thread can not process the log file that's using to record the log. in code side, the pseudocode similars like below:
#code in second thread to process the log files
for logFile in os.listdir(logFolder):
if not file_is_open(logFile) or file_is_use(logFile):
ProcessLogFile(logFile) # move log file to other place, and generate log report....
So, how do i check is a file is already open or is used by other process?
I did some research in internet. And have some results:
try:
myfile = open(filename, "r+") # or "a+", whatever you need
except IOError:
print "Could not open file! Please close Excel!"
I tried this code, but it doesn't work, no matter i use "r+" or "a+" flag
try:
os.remove(filename) # try to remove it directly
except OSError as e:
if e.errno == errno.ENOENT: # file doesn't exist
break
This code can work, but it can not reach my request, since i don't want to delete the file to check if it is open.
An issue with trying to find out if a file is being used by another process is the possibility of a race condition. You could check a file, decide that it is not in use, then just before you open it another process (or thread) leaps in and grabs it (or even deletes it).
Ok, let's say you decide to live with that possibility and hope it does not occur. To check files in use by other processes is operating system dependant.
On Linux it is fairly easy, just iterate through the PIDs in /proc. Here is a generator that iterates over files in use for a specific PID:
def iterate_fds(pid):
dir = '/proc/'+str(pid)+'/fd'
if not os.access(dir,os.R_OK|os.X_OK): return
for fds in os.listdir(dir):
for fd in fds:
full_name = os.path.join(dir, fd)
try:
file = os.readlink(full_name)
if file == '/dev/null' or \
re.match(r'pipe:\[\d+\]',file) or \
re.match(r'socket:\[\d+\]',file):
file = None
except OSError as err:
if err.errno == 2:
file = None
else:
raise(err)
yield (fd,file)
On Windows it is not quite so straightforward, the APIs are not published. There is a sysinternals tool (handle.exe) that can be used, but I recommend the PyPi module psutil, which is portable (i.e., it runs on Linux as well, and probably on other OS):
import psutil
for proc in psutil.process_iter():
try:
# this returns the list of opened files by the current process
flist = proc.open_files()
if flist:
print(proc.pid,proc.name)
for nt in flist:
print("\t",nt.path)
# This catches a race condition where a process ends
# before we can examine its files
except psutil.NoSuchProcess as err:
print("****",err)
I like Daniel's answer, but for Windows users, I realized that it's safer and simpler to rename the file to the name it already has. That solves the problems brought up in the comments to his answer. Here's the code:
import os
f = 'C:/test.xlsx'
if os.path.exists(f):
try:
os.rename(f, f)
print 'Access on file "' + f +'" is available!'
except OSError as e:
print 'Access-error on file "' + f + '"! \n' + str(e)
You can check if a file has a handle on it using the next function (remember to pass the full path to that file):
import psutil
def has_handle(fpath):
for proc in psutil.process_iter():
try:
for item in proc.open_files():
if fpath == item.path:
return True
except Exception:
pass
return False
I know I'm late to the party but I also had this problem and I used the lsof command to solve it (which I think is new from the approaches mentioned above). With lsof we can basically check for the processes that are using this particular file.
Here is how I did it:
from subprocess import check_output,Popen, PIPE
try:
lsout=Popen(['lsof',filename],stdout=PIPE, shell=False)
check_output(["grep",filename], stdin=lsout.stdout, shell=False)
except:
#check_output will throw an exception here if it won't find any process using that file
just write your log processing code in the except part and you are good to go.
Instead on using os.remove() you may use the following workaround on Windows:
import os
file = "D:\\temp\\test.pdf"
if os.path.exists(file):
try:
os.rename(file,file+"_")
print "Access on file \"" + str(file) +"\" is available!"
os.rename(file+"_",file)
except OSError as e:
message = "Access-error on file \"" + str(file) + "\"!!! \n" + str(e)
print message
You can use inotify to watch for activity in file system. You can watch for file close events, indicating that a roll-over has happened. You should also add additional condition on file-size. Make sure you filter out file close events from the second thread.
A slightly more polished version of one of the answers from above.
from pathlib import Path
def is_file_in_use(file_path):
path = Path(file_path)
if not path.exists():
raise FileNotFoundError
try:
path.rename(path)
except PermissionError:
return True
else:
return False
On Windows, you can also directly retrieve the information by leveraging on the NTDLL/KERNEL32 Windows API. The following code returns a list of PIDs, in case the file is still opened/used by a process (including your own, if you have an open handle on the file):
import ctypes
from ctypes import wintypes
path = r"C:\temp\test.txt"
# -----------------------------------------------------------------------------
# generic strings and constants
# -----------------------------------------------------------------------------
ntdll = ctypes.WinDLL('ntdll')
kernel32 = ctypes.WinDLL('kernel32', use_last_error=True)
NTSTATUS = wintypes.LONG
INVALID_HANDLE_VALUE = wintypes.HANDLE(-1).value
FILE_READ_ATTRIBUTES = 0x80
FILE_SHARE_READ = 1
OPEN_EXISTING = 3
FILE_FLAG_BACKUP_SEMANTICS = 0x02000000
FILE_INFORMATION_CLASS = wintypes.ULONG
FileProcessIdsUsingFileInformation = 47
LPSECURITY_ATTRIBUTES = wintypes.LPVOID
ULONG_PTR = wintypes.WPARAM
# -----------------------------------------------------------------------------
# create handle on concerned file with dwDesiredAccess == FILE_READ_ATTRIBUTES
# -----------------------------------------------------------------------------
kernel32.CreateFileW.restype = wintypes.HANDLE
kernel32.CreateFileW.argtypes = (
wintypes.LPCWSTR, # In lpFileName
wintypes.DWORD, # In dwDesiredAccess
wintypes.DWORD, # In dwShareMode
LPSECURITY_ATTRIBUTES, # In_opt lpSecurityAttributes
wintypes.DWORD, # In dwCreationDisposition
wintypes.DWORD, # In dwFlagsAndAttributes
wintypes.HANDLE) # In_opt hTemplateFile
hFile = kernel32.CreateFileW(
path, FILE_READ_ATTRIBUTES, FILE_SHARE_READ, None, OPEN_EXISTING,
FILE_FLAG_BACKUP_SEMANTICS, None)
if hFile == INVALID_HANDLE_VALUE:
raise ctypes.WinError(ctypes.get_last_error())
# -----------------------------------------------------------------------------
# prepare data types for system call
# -----------------------------------------------------------------------------
class IO_STATUS_BLOCK(ctypes.Structure):
class _STATUS(ctypes.Union):
_fields_ = (('Status', NTSTATUS),
('Pointer', wintypes.LPVOID))
_anonymous_ = '_Status',
_fields_ = (('_Status', _STATUS),
('Information', ULONG_PTR))
iosb = IO_STATUS_BLOCK()
class FILE_PROCESS_IDS_USING_FILE_INFORMATION(ctypes.Structure):
_fields_ = (('NumberOfProcessIdsInList', wintypes.LARGE_INTEGER),
('ProcessIdList', wintypes.LARGE_INTEGER * 64))
info = FILE_PROCESS_IDS_USING_FILE_INFORMATION()
PIO_STATUS_BLOCK = ctypes.POINTER(IO_STATUS_BLOCK)
ntdll.NtQueryInformationFile.restype = NTSTATUS
ntdll.NtQueryInformationFile.argtypes = (
wintypes.HANDLE, # In FileHandle
PIO_STATUS_BLOCK, # Out IoStatusBlock
wintypes.LPVOID, # Out FileInformation
wintypes.ULONG, # In Length
FILE_INFORMATION_CLASS) # In FileInformationClass
# -----------------------------------------------------------------------------
# system call to retrieve list of PIDs currently using the file
# -----------------------------------------------------------------------------
status = ntdll.NtQueryInformationFile(hFile, ctypes.byref(iosb),
ctypes.byref(info),
ctypes.sizeof(info),
FileProcessIdsUsingFileInformation)
pidList = info.ProcessIdList[0:info.NumberOfProcessIdsInList]
print(pidList)
I provided one solution. please see the following code.
def isFileinUsed(ifile):
widlcard = "/proc/*/fd/*"
lfds = glob.glob(widlcard)
for fds in lfds:
try:
file = os.readlink(fds)
if file == ifile:
return True
except OSError as err:
if err.errno == 2:
file = None
else:
raise(err)
return False
You can use this function to check if a file is in used.
Note:
This solution only can be used for Linux system.
I would like to automatically generate some sort of log of all the database changes that are made via the Django shell in the production environment.
We use schema and data migration scripts to alter the production database and they are version controlled. Therefore if we introduce a bug, it's easy to track it back. But if a developer in the team changes the database via the Django shell which then introduces an issue, at the moment we can only hope that they remember what they did or/and we can find their commands in the Python shell history.
Example. Let's imagine that the following code was executed by a developer in the team via the Python shell:
>>> tm = TeamMembership.objects.get(person=alice)
>>> tm.end_date = date(2022,1,1)
>>> tm.save()
It changes a team membership object in the database. I would like to log this somehow.
I'm aware that there are a bunch of Django packages related to audit logging, but I'm only interested in the changes that are triggered from the Django shell, and I want to log the Python code that updated the data.
So the questions I have in mind:
I can log the statements from IPython but how do I know which one touched the database?
I can listen to the pre_save signal for all model to know if data changes, but how do I know if the source was from the Python shell? How do I know what was the original Python statement?
This solution logs all commands in the session if any database changes were made.
How to detect database changes
Wrap execute_sql of SQLInsertCompiler, SQLUpdateCompiler and SQLDeleteCompiler.
SQLDeleteCompiler.execute_sql returns a cursor wrapper.
from django.db.models.sql.compiler import SQLInsertCompiler, SQLUpdateCompiler, SQLDeleteCompiler
changed = False
def check_changed(func):
def _func(*args, **kwargs):
nonlocal changed
result = func(*args, **kwargs)
if not changed and result:
changed = not hasattr(result, 'cursor') or bool(result.cursor.rowcount)
return result
return _func
SQLInsertCompiler.execute_sql = check_changed(SQLInsertCompiler.execute_sql)
SQLUpdateCompiler.execute_sql = check_changed(SQLUpdateCompiler.execute_sql)
SQLDeleteCompiler.execute_sql = check_changed(SQLDeleteCompiler.execute_sql)
How to log commands made via the Django shell
atexit.register() an exit handler that does readline.write_history_file().
import atexit
import readline
def exit_handler():
filename = 'history.py'
readline.write_history_file(filename)
atexit.register(exit_handler)
IPython
Check whether IPython was used by comparing HistoryAccessor.get_last_session_id().
import atexit
import io
import readline
ipython_last_session_id = None
try:
from IPython.core.history import HistoryAccessor
except ImportError:
pass
else:
ha = HistoryAccessor()
ipython_last_session_id = ha.get_last_session_id()
def exit_handler():
filename = 'history.py'
if ipython_last_session_id and ipython_last_session_id != ha.get_last_session_id():
cmds = '\n'.join(cmd for _, _, cmd in ha.get_range(ha.get_last_session_id()))
with io.open(filename, 'a', encoding='utf-8') as f:
f.write(cmds)
f.write('\n')
else:
readline.write_history_file(filename)
atexit.register(exit_handler)
Put it all together
Add the following in manage.py before execute_from_command_line(sys.argv).
if sys.argv[1] == 'shell':
import atexit
import io
import readline
from django.db.models.sql.compiler import SQLInsertCompiler, SQLUpdateCompiler, SQLDeleteCompiler
changed = False
def check_changed(func):
def _func(*args, **kwargs):
nonlocal changed
result = func(*args, **kwargs)
if not changed and result:
changed = not hasattr(result, 'cursor') or bool(result.cursor.rowcount)
return result
return _func
SQLInsertCompiler.execute_sql = check_changed(SQLInsertCompiler.execute_sql)
SQLUpdateCompiler.execute_sql = check_changed(SQLUpdateCompiler.execute_sql)
SQLDeleteCompiler.execute_sql = check_changed(SQLDeleteCompiler.execute_sql)
ipython_last_session_id = None
try:
from IPython.core.history import HistoryAccessor
except ImportError:
pass
else:
ha = HistoryAccessor()
ipython_last_session_id = ha.get_last_session_id()
def exit_handler():
if changed:
filename = 'history.py'
if ipython_last_session_id and ipython_last_session_id != ha.get_last_session_id():
cmds = '\n'.join(cmd for _, _, cmd in ha.get_range(ha.get_last_session_id()))
with io.open(filename, 'a', encoding='utf-8') as f:
f.write(cmds)
f.write('\n')
else:
readline.write_history_file(filename)
atexit.register(exit_handler)
I would consider something like this:
Wrapping each python session with some sort initialisation code using e.g.
PYTHONSTARTUP environment variable
https://docs.python.org/3/using/cmdline.html#envvar-PYTHONSTARTUP
In the file where PYTHONSTARTUP points to registering Exit handler using atexit
https://docs.python.org/3/library/atexit.html
These two things should allow you to use some lower level APIs of
django-reversion to wrap the whole terminal session with
https://django-reversion.readthedocs.io/en/stable/api.html#creating-revisions (something like this but calling __enter__ and __exit__ of that context manager directly in your startup and atexit code). Unfortunately I don't know the details but it should be doable.
In atexit / revision end calling the code to list the additional lines
of the terminal session and storing them somewhere else in the database with a reference to the specific revision.
See:
https://docs.python.org/3/library/readline.html#readline.get_history_length
https://docs.python.org/3/library/readline.html#readline.get_history_item
Basically, the idea is that you could call get_history_length twice: at the beginning and end of the terminal session. That will allow you to get relevant lines of where the change took place using get_history_item. You may end up having more lines of history than what you actually need but at least there is enough context to see what's going on.
Based on the answer of aaron and the implementation of the built-in IPython magic %logstart, this is the solution we came up with in the end.
All commands of the last IPython session are logged in a history file if any of the commands triggered a database write through the Django ORM.
Here's an excerpt of the generated history file:
❯ cat ~/.python_shell_write_history
# Thu, 27 Jan 2022 16:20:28
#
# New Django shell session started
#
# Thu, 27 Jan 2022 16:20:28
from people.models import *
# Thu, 27 Jan 2022 16:20:28
p = Person.objects.first()
# Thu, 27 Jan 2022 16:20:28
p
#[Out]# <Person: Test Albero Jose Maria>
# Thu, 27 Jan 2022 16:20:28
p.email
#[Out]# 'test-albero-jose-maria#gmail.com'
# Thu, 27 Jan 2022 16:20:28
p.save()
Here's our manage.py now:
#!/usr/bin/env python
import os
import sys
def shell_audit(logfname: str) -> None:
"""If any of the Python shell commands changed the Django database during the
session, capture all commands in a logfile for future analysis."""
import atexit
from django.db.models.sql.compiler import (
SQLDeleteCompiler,
SQLInsertCompiler,
SQLUpdateCompiler,
)
changed = False
def check_changed(func):
def _func(*args, **kwargs):
nonlocal changed
result = func(*args, **kwargs)
if not changed and result:
changed = not hasattr(result, "cursor") or bool(result.cursor.rowcount)
return result
return _func
SQLInsertCompiler.execute_sql = check_changed(SQLInsertCompiler.execute_sql)
SQLUpdateCompiler.execute_sql = check_changed(SQLUpdateCompiler.execute_sql)
SQLDeleteCompiler.execute_sql = check_changed(SQLDeleteCompiler.execute_sql)
def exit_handler():
if not changed:
return None
from IPython.core import getipython
shell = getipython.get_ipython()
if not shell:
return None
logger = shell.logger
# Logic borrowed from %logstart (IPython.core.magics.logging)
loghead = ""
log_session_head = "#\n# New Django shell session started\n#\n"
logmode = "append"
log_output = True
timestamp = True
log_raw_input = False
logger.logstart(logfname, loghead, logmode, log_output, timestamp, log_raw_input)
log_write = logger.log_write
input_hist = shell.history_manager.input_hist_parsed
output_hist = shell.history_manager.output_hist_reprs
log_write(log_session_head)
for n in range(1, len(input_hist)):
log_write(input_hist[n].rstrip() + "\n")
if n in output_hist:
log_write(output_hist[n], kind="output")
atexit.register(exit_handler)
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings")
try:
from django.core.management import execute_from_command_line
except ImportError:
# The above import may fail for some other reason. Ensure that the
# issue is really that Django is missing to avoid masking other
# exceptions on Python 2.
try:
import django # noqa: F401
except ImportError:
raise ImportError(
"Couldn't import Django. Are you sure it's installed and "
"available on your PYTHONPATH environment variable? Did you "
"forget to activate a virtual environment?"
)
raise
if sys.argv[1] == "shell":
logfname = os.path.expanduser("~/.python_shell_write_history")
shell_audit(logfname)
execute_from_command_line(sys.argv)
You could use django's receiver annotation.
For example, if you want to detect any call of the save method, you could do:
from django.db.models.signals import post_save
from django.dispatch import receiver
import logging
#receiver(post_save)
def logg_save(sender, instance, **kwargs):
logging.debug("whatever you want to log")
some more documentation for the signals
I have a program that outputs many calculations and results to the console through the print statement. I want to write some code to export (or save) all the contents of the console to a simple text file.
I searched StackOverflow and other sites but I found some methods to redirect the print statement to print to a file directly, but I want the program to work normally, to display outputs to the console, then to save its contents AFTER all operations of the program done.
I am using PyCharm with Python2.7 if it matters
Ok, so normally to get it done, you have to rewrite python print built-in function. But... There is ipython, which provides some hooks.
First you need to have ipython installed:
#bash
sudo pip install ipython
(I'm using sudo to simple locate then folder I need to reach, read further)
After ipython installation you'll have ipython extensions folder available, so get to it:
#bash
cd ~/.ipython/extensions/
and create there let's say a file called print_to_file.py, here is its content:
#python
class PrintWatcher(object):
def __init__(self, ip):
self.shell = ip
def post_execute(self):
with open('/home/turkus/shell.txt', 'a+') as f:
in_len = len(self.shell.user_ns['In'])
i = in_len - 1
in_ = self.shell.user_ns['In'][i]
out = self.shell.user_ns['Out'].get(i, '')
# you can edit this line if you want different input in shell.txt
f.write('{}\n{}\n'.format(in_, out))
def load_ipython_extension(ip):
pw = PrintWatcher(ip)
ip.events.register('post_run_cell', pw.post_execute)
After saving a file just run:
#bash
ipython profile create
# you will get something like that:
[ProfileCreate] Generating default config file: u'/home/turkus/.ipython/profile_default/ipython_config.py'
Now get back to setting up our hook. We must open ipython_config.py created under path above and put there some magic (there is a lot of stuff there, so go to the end of file):
# some commented lines here
c = get_config()
c.InteractiveShellApp.extensions = [
'print_to_file'
]
After saving it, you can run ipython and write your code. Every your input will be written in a file under path you provided above, in my case it was:
/home/turkus/shell.txt
Notes
You can avoid loading your extension every time ipython fires up, by just delete 'print_to_file' from c.InteractiveShellApp.extensions list in ipython_config.py. But remember that you can load it anytime you need, just by typing in ipython console:
➜ ~ ipython
Python 2.7.12 (default, Jul 1 2016, 15:12:24)
Type "copyright", "credits" or "license" for more information.
IPython 4.0.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: %load_ext print_to_file
Any change in print_to_file.py is being reflected in open ipython shell after using %reload_ext print_to_file command, so you don't have to exit from and fire up it again.
I am unsure how you could receive the contents of a console for any editor however this can be achieved quite simply by replacing your print() statements with .write
class Writer(object):
def __init__(self, out_file, overwrite=False):
self.file_name = out_file
self.overwrite = overwrite
self.history = []
def write(self, statement):
self.history.append(statement)
print statement
def close(self):
if self.overwrite:
self.out_file = open(self.file_name, 'wb')
else:
self.out_file = open(self.file_name, 'ab')
for x in self.history:
self.out_file.write(x+'/n')
self.out_file.close()
self.history = []
p = Writer('my_output_file.txt')
p.write('my string to print and save!')
p.close() #close the writer to save the contents to a file before exiting
After I know understood your question I think you search the tee command
python your_program | tee output.txt
This will show you the output both, in the console and in output.txt
PS: Since you did not answer to my comment which OS you use I assumed that you use either Linux or MACOS. Should work on both. I don't know how to do this on windows...
You could override the print function which will still be accessible through the builtins module
import builtins
f = open("logs.txt", "w")
def print(*args, sep=' ', end='\n', **kwargs):
builtins.print(*args, sep=sep, end=end, **kwargs)
f.write(sep.join(*args) + end)
EDIT: A similar solution for Python 2
from __future__ import print_function
class Print:
def __init__(self, print_function, filename='test', mode='w'):
self.print_function = print_function
self.file = open(filename, 'w')
def __call__(self, *args, **kwargs):
self.print_function(*args, **kwargs)
kwargs['file'] = self.file
self.print_function(*args, **kwargs)
print = Print(print, 'logs.txt')
This creates a print function that you use exactly as the function you import from __future__.
To close the file when everything is done you have to run:
print.file.close()
Maybe you should create a variable that will log the outputs and then put it into a file.
For ex:
print statement
logger += statement+"\n" #a new line char so each statement is on a new line
with open('file.txt', 'a') as f:
f.write(statement)
With all thanks and respect to all who contributed to this question. I have finally found a solution to this problem with minimal modifications to my original code. The solution is provided by the member #Status and here is its link .
Although I searched a lot before posting my question, but the answers of the respected members enlightened my mind to a precise search especially the contributions of #turkus, who performs an exceptional work, and #Glostas who opened my eyes to the "tee" which guided me to find the solution I posted (although it does not contain "tee").
The solution, as of the mentioned post with slight modifications:
1- Put the following Class in the program:
class Logger(object):
"""
Lumberjack class - duplicates sys.stdout to a log file and it's okay
source: https://stackoverflow.com/a/24583265/5820024
"""
def __init__(self, filename="Red.Wood", mode="a", buff=0):
self.stdout = sys.stdout
self.file = open(filename, mode, buff)
sys.stdout = self
def __del__(self):
self.close()
def __enter__(self):
pass
def __exit__(self, *args):
pass
def write(self, message):
self.stdout.write(message)
self.file.write(message)
def flush(self):
self.stdout.flush()
self.file.flush()
os.fsync(self.file.fileno())
def close(self):
if self.stdout != None:
sys.stdout = self.stdout
self.stdout = None
if self.file != None:
self.file.close()
self.file = None
2- At the beginning of the program, before any print statements, put this line:
my_console = Logger('my_console_file.txt') # you can change the file's name
3- At the end of the program, after all of the print statements, put this line:
my_console.close()
I tested this, and It works perfectly, and finally I have a clone of the console's output after the program ends.
With best regards to everybody, and Many thanks to all contributors.
There is a very obvious but not very elegant solution.
instead of:
print statement 1
calculation
print statement 2
you can make something like
sexport =''
calculation
print statement 1
sexport += statement1 + "\n"
calculaztion
print statement 2
sexport += statement 2
finally just save sexport to a file
How do you use Python to determine which Linux device/partition contains a given filesystem?
e.g.
>>> get_filesystem_device('/')
/dev/sda
>>> get_filesystem_partition('/')
/dev/sda1
Your question was about Linux, so this is (more or less) linux specific.
Below is code example for three variants for mapping major/minor to a device name.
Parse /proc/partitions.
Ask hal. Hal also keeps track of "parent" device, meaning you can easily get the disk aswell as the partition.
Check sysfs yourself. This is where hal gets its information from.
I'd say that /proc/partitions is simplest - it is just one file to open and check. hal gives you most information, and abstracts away lots of details. sysfs may be viewed as more correct that /proc/partitions and doesn't require hal to be running.
For a desktop program I would go for hal. On an embedded system I'd go with sysfs.
import os
def main():
dev = os.stat("/home/").st_dev
major, minor = os.major(dev), os.minor(dev)
print "/proc/partitions says:", ask_proc_partitions(major, minor)
print "HAL says:", ask_hal(major, minor)
print "/sys says:", ask_sysfs(major, minor)
def _parse_proc_partitions():
res = {}
for line in file("/proc/partitions"):
fields = line.split()
try:
tmaj = int(fields[0])
tmin = int(fields[1])
name = fields[3]
res[(tmaj, tmin)] = name
except:
# just ignore parse errors in header/separator lines
pass
return res
def ask_proc_partitions(major, minor):
d = _parse_proc_partitions()
return d[(major, minor)]
def ask_hal(major, minor):
import dbus
bus = dbus.SystemBus()
halobj = bus.get_object('org.freedesktop.Hal', '/org/freedesktop/Hal/Manager')
hal = dbus.Interface(halobj, 'org.freedesktop.Hal.Manager')
def getdevprops(p):
bdevi = dbus.Interface(bus.get_object('org.freedesktop.Hal', p),
"org.freedesktop.Hal.Device")
return bdevi.GetAllProperties()
bdevs = hal.FindDeviceByCapability("block")
for bdev in bdevs:
props = getdevprops(bdev)
if (props['block.major'], props['block.minor']) == (major, minor):
parentprops = getdevprops(props['info.parent'])
return (str(props['block.device']),
str(parentprops['block.device']))
def ask_sysfs(major, minor):
from glob import glob
needle = "%d:%d" % (major, minor)
files = glob("/sys/class/block/*/dev")
for f in files:
if file(f).read().strip() == needle:
return os.path.dirname(f)
return None
if __name__ == '__main__':
main()
It looks like this post has some of your answer (still not sure just how to grab the major/minor out of the /dev/sda2 entry to match it up with what os.stat() returns for /:
Device number in stat command output
>>> import os
>>> print hex(os.stat('/')[2])
0x802
\ \minor device number
\major device number
[me#server /]$ ls -l /dev/sda2
brw-rw---- 1 root disk 8, 2 Jun 24 2004 /dev/sda2
[me#server jgaines2]$ \ \minor device number
\major device number
I recently had a need for this solution also. After seeing all the convoluted methods of getting the result I wanted through pure python, I decided to turn to the shell for help.
import subprocess
device = subprocess.check_output("grep '/filesystem' /proc/mounts | awk '{printf $1}'", shell=True)
print device
This gives me exactly what I want, the device string for where my filesystem is mounted.
Short, sweet, and runs in python. :)
There are problems with quite a few of the above solutions. There's actually a problem with the question as well.
The last answer (searching /proc/mounts) just doesn't work: searching for "/" will match every line in /proc/mounts. Even correcting this like this won't work:
import subprocess
device = subprocess.check_output("awk '$2 == \"/filesystem\" { print $1}' /proc/mounts", shell=True)
print device
When "/filesystem" is "/" you'll typically get two entries, one for "rootfs" and one for the actual device. It also won't work when the mounted file system name has spaces in it (the space appears as \040 in /proc/mounts).
The problem is made worse with btrfs subvolumes. Each subvolume is mounted separately but they all share the same device. If you're trying to use a btrfs snapshot for backups (as I was) then you need the subvolume name and an indication of the filesystem type.
This function returns a tuple of (device, mountpoint, filesystem) and seems to work:
import os
def get_filesystem_partition(fs):
res = None
dev = os.lstat(fs).st_dev
for line in file('/proc/mounts'):
# lines are device, mountpoint, filesystem, <rest>
# later entries override earlier ones
line = [s.decode('string_escape') for s in line.split()[:3]]
if dev == os.lstat(line[1]).st_dev:
res = tuple(line)
return res
That seems to work for all the cases I can think of, although I expect that there are still pathological cases where it falls to bits.
It is not the purdiest, but this will get you started:
#!/usr/bin/python
import os, stat, subprocess, shlex, re, sys
dev=os.stat('/')[stat.ST_DEV]
major=os.major(dev)
minor=os.minor(dev)
out = subprocess.Popen(shlex.split("df /"), stdout=subprocess.PIPE).communicate()
m=re.search(r'(/[^\s]+)\s',str(out))
if m:
mp= m.group(1)
else:
print "cannot parse df"
sys.exit(2)
print "'/' mounted at '%s' with dev number %i, %i" % (mp,major,minor)
On OS X:
'/' mounted at '/dev/disk0s2' with dev number 14, 2
On Ubuntu:
'/' mounted at '/dev/sda1' with dev number 8, 1
To get the device name, chop off the minor number from the partition name. On OS X, also chop the 's' + minor number.
How about using the (linux) blkid command (/sbin/blkid)
$ uname --kernel-name --kernel-release
Linux 3.2.0-4-amd64
$ python --version
Python 2.7.3
-
#!/usr/bin/env python
import subprocess
sys_command = "/sbin/blkid"
proc = subprocess.Popen(sys_command,
stdout=subprocess.PIPE,
shell=True)
# proc.communicate() returns a tuple (stdout,stderr)
blkid_output = proc.communicate()[0]
print blkid_output
Here's the output on a dual-boot laptop with an (unmounted) USB drive (sdb1)
$ ./blkid.py
/dev/sda1: LABEL="RECOVERY" UUID="xxxx-xxxx" TYPE="vfat"
/dev/sda2: LABEL="OS" UUID="xxxxxxxxxxxxxxx" TYPE="ntfs"
/dev/sda5: UUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" TYPE="ext4"
/dev/sda6: UUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" TYPE="swap"
/dev/sda7: UUID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" TYPE="ext4"
/dev/sdb1: LABEL="CrunchBang" TYPE="iso9660"
Here is how you can simply get the devices major and minor numbers:
import os
major, minor = divmod(os.stat('/').st_dev, 256)
How can I change my desktop background with python?
I want to do it in both Windows and Linux.
On Windows with python2.5 or higher, use ctypes to load user32.dll and call SystemParametersInfo() with SPI_SETDESKWALLPAPER action.
For example:
import ctypes
SPI_SETDESKWALLPAPER = 20
ctypes.windll.user32.SystemParametersInfoA(SPI_SETDESKWALLPAPER, 0, "image.jpg" , 0)
For Python3.5, SystemParametersInfoA doesn't work. Use SystemParametersInfoW.
import ctypes
ctypes.windll.user32.SystemParametersInfoW(20, 0, "absolute path" , 0)
I use the following method in one of my initial projects:
def set_wallpaper(self,file_loc, first_run):
# Note: There are two common Linux desktop environments where
# I have not been able to set the desktop background from
# command line: KDE, Enlightenment
desktop_env = self.get_desktop_environment()
try:
if desktop_env in ["gnome", "unity", "cinnamon"]:
uri = "'file://%s'" % file_loc
try:
SCHEMA = "org.gnome.desktop.background"
KEY = "picture-uri"
gsettings = Gio.Settings.new(SCHEMA)
gsettings.set_string(KEY, uri)
except:
args = ["gsettings", "set", "org.gnome.desktop.background", "picture-uri", uri]
subprocess.Popen(args)
elif desktop_env=="mate":
try: # MATE >= 1.6
# info from http://wiki.mate-desktop.org/docs:gsettings
args = ["gsettings", "set", "org.mate.background", "picture-filename", "'%s'" % file_loc]
subprocess.Popen(args)
except: # MATE < 1.6
# From https://bugs.launchpad.net/variety/+bug/1033918
args = ["mateconftool-2","-t","string","--set","/desktop/mate/background/picture_filename",'"%s"' %file_loc]
subprocess.Popen(args)
elif desktop_env=="gnome2": # Not tested
# From https://bugs.launchpad.net/variety/+bug/1033918
args = ["gconftool-2","-t","string","--set","/desktop/gnome/background/picture_filename", '"%s"' %file_loc]
subprocess.Popen(args)
## KDE4 is difficult
## see http://blog.zx2c4.com/699 for a solution that might work
elif desktop_env in ["kde3", "trinity"]:
# From http://ubuntuforums.org/archive/index.php/t-803417.html
args = 'dcop kdesktop KBackgroundIface setWallpaper 0 "%s" 6' % file_loc
subprocess.Popen(args,shell=True)
elif desktop_env=="xfce4":
#From http://www.commandlinefu.com/commands/view/2055/change-wallpaper-for-xfce4-4.6.0
if first_run:
args0 = ["xfconf-query", "-c", "xfce4-desktop", "-p", "/backdrop/screen0/monitor0/image-path", "-s", file_loc]
args1 = ["xfconf-query", "-c", "xfce4-desktop", "-p", "/backdrop/screen0/monitor0/image-style", "-s", "3"]
args2 = ["xfconf-query", "-c", "xfce4-desktop", "-p", "/backdrop/screen0/monitor0/image-show", "-s", "true"]
subprocess.Popen(args0)
subprocess.Popen(args1)
subprocess.Popen(args2)
args = ["xfdesktop","--reload"]
subprocess.Popen(args)
elif desktop_env=="razor-qt": #TODO: implement reload of desktop when possible
if first_run:
desktop_conf = configparser.ConfigParser()
# Development version
desktop_conf_file = os.path.join(self.get_config_dir("razor"),"desktop.conf")
if os.path.isfile(desktop_conf_file):
config_option = r"screens\1\desktops\1\wallpaper"
else:
desktop_conf_file = os.path.join(self.get_home_dir(),".razor/desktop.conf")
config_option = r"desktops\1\wallpaper"
desktop_conf.read(os.path.join(desktop_conf_file))
try:
if desktop_conf.has_option("razor",config_option): #only replacing a value
desktop_conf.set("razor",config_option,file_loc)
with codecs.open(desktop_conf_file, "w", encoding="utf-8", errors="replace") as f:
desktop_conf.write(f)
except:
pass
else:
#TODO: reload desktop when possible
pass
elif desktop_env in ["fluxbox","jwm","openbox","afterstep"]:
#http://fluxbox-wiki.org/index.php/Howto_set_the_background
# used fbsetbg on jwm too since I am too lazy to edit the XML configuration
# now where fbsetbg does the job excellent anyway.
# and I have not figured out how else it can be set on Openbox and AfterSTep
# but fbsetbg works excellent here too.
try:
args = ["fbsetbg", file_loc]
subprocess.Popen(args)
except:
sys.stderr.write("ERROR: Failed to set wallpaper with fbsetbg!\n")
sys.stderr.write("Please make sre that You have fbsetbg installed.\n")
elif desktop_env=="icewm":
# command found at http://urukrama.wordpress.com/2007/12/05/desktop-backgrounds-in-window-managers/
args = ["icewmbg", file_loc]
subprocess.Popen(args)
elif desktop_env=="blackbox":
# command found at http://blackboxwm.sourceforge.net/BlackboxDocumentation/BlackboxBackground
args = ["bsetbg", "-full", file_loc]
subprocess.Popen(args)
elif desktop_env=="lxde":
args = "pcmanfm --set-wallpaper %s --wallpaper-mode=scaled" % file_loc
subprocess.Popen(args,shell=True)
elif desktop_env=="windowmaker":
# From http://www.commandlinefu.com/commands/view/3857/set-wallpaper-on-windowmaker-in-one-line
args = "wmsetbg -s -u %s" % file_loc
subprocess.Popen(args,shell=True)
## NOT TESTED BELOW - don't want to mess things up ##
#elif desktop_env=="enlightenment": # I have not been able to make it work on e17. On e16 it would have been something in this direction
# args = "enlightenment_remote -desktop-bg-add 0 0 0 0 %s" % file_loc
# subprocess.Popen(args,shell=True)
#elif desktop_env=="windows": #Not tested since I do not run this on Windows
# #From https://stackoverflow.com/questions/1977694/change-desktop-background
# import ctypes
# SPI_SETDESKWALLPAPER = 20
# ctypes.windll.user32.SystemParametersInfoA(SPI_SETDESKWALLPAPER, 0, file_loc , 0)
#elif desktop_env=="mac": #Not tested since I do not have a mac
# #From https://stackoverflow.com/questions/431205/how-can-i-programatically-change-the-background-in-mac-os-x
# try:
# from appscript import app, mactypes
# app('Finder').desktop_picture.set(mactypes.File(file_loc))
# except ImportError:
# #import subprocess
# SCRIPT = """/usr/bin/osascript<<END
# tell application "Finder" to
# set desktop picture to POSIX file "%s"
# end tell
# END"""
# subprocess.Popen(SCRIPT%file_loc, shell=True)
else:
if first_run: #don't spam the user with the same message over and over again
sys.stderr.write("Warning: Failed to set wallpaper. Your desktop environment is not supported.")
sys.stderr.write("You can try manually to set Your wallpaper to %s" % file_loc)
return False
return True
except:
sys.stderr.write("ERROR: Failed to set wallpaper. There might be a bug.\n")
return False
def get_config_dir(self, app_name=APP_NAME):
if "XDG_CONFIG_HOME" in os.environ:
confighome = os.environ['XDG_CONFIG_HOME']
elif "APPDATA" in os.environ: # On Windows
confighome = os.environ['APPDATA']
else:
try:
from xdg import BaseDirectory
confighome = BaseDirectory.xdg_config_home
except ImportError: # Most likely a Linux/Unix system anyway
confighome = os.path.join(self.get_home_dir(),".config")
configdir = os.path.join(confighome,app_name)
return configdir
def get_home_dir(self):
if sys.platform == "cygwin":
home_dir = os.getenv('HOME')
else:
home_dir = os.getenv('USERPROFILE') or os.getenv('HOME')
if home_dir is not None:
return os.path.normpath(home_dir)
else:
raise KeyError("Neither USERPROFILE or HOME environment variables set.")
The get_desktop_environment method has been posted in another thread.
On a gnome desktop, you usually do this with gconf, either directly calling gconftool or using the gconf python module. The latter is in the link given by unutbu. The first method could be done like this.
import commands
command = "gconftool-2 --set /desktop/gnome/background/picture_filename --type string '/path/to/file.jpg'"
status, output = commands.getstatusoutput(command) # status=0 if success
In gnome, it is probably preferable to use the python binding of gconf directly:
import gconf
conf = gconf.client_get_default()
conf.set_string('/desktop/gnome/background/picture_filename','/path/to/filename.jpg')
On windows, you will need some trickery with pywin32, and the windows API, on 'linux' the answer will depend on which desktop is running - KDE, Gnome, or something more exotic. Under KDE (and maybe Gnome) you can probably send a message using D-Bus, which you could do without including any new libraries by using the command line tool dbus-send.
The other option would be to set the desktop wallpaper to a file which you then edit / replace from python - but this will probably only result in a change when the user logs in.
Firstly, import ctypes: it gives you access to windows components such as the screensaver, wallpapers, etc.
Then call
ctypes.windll.user32.SystemParametersInfoA(20, 0, the_complete_path_of_your_image, 0)
Make sure the path is the complete path of your image, not just the path from the active directory
There is a difference what SystemParametersInfo method to be called based on what if you are running on 64 bit or 32 bit OS. For 64 bit you have to use SystemParametersInfoW (Unicode) and for 32 bit SystemParametersInfoA (ANSI)
import struct
import ctypes
SPI_SETDESKWALLPAPER = 20
WALLPAPER_PATH = 'C:\\your_file_name.jpg'
def is_64_windows():
"""Find out how many bits is OS. """
return struct.calcsize('P') * 8 == 64
def get_sys_parameters_info():
"""Based on if this is 32bit or 64bit returns correct version of SystemParametersInfo function. """
return ctypes.windll.user32.SystemParametersInfoW if is_64_windows() \
else ctypes.windll.user32.SystemParametersInfoA
def change_wallpaper():
sys_parameters_info = get_sys_parameters_info()
r = sys_parameters_info(SPI_SETDESKWALLPAPER, 0, WALLPAPER_PATH, 3)
# When the SPI_SETDESKWALLPAPER flag is used,
# SystemParametersInfo returns TRUE
# unless there is an error (like when the specified file doesn't exist).
if not r:
print(ctypes.WinError())
change_wallpaper()
import ctypes,win32con
def getWallpaper():
ubuf = ctypes.create_unicode_buffer(512)
ctypes.windll.user32.SystemParametersInfoW(win32con.SPI_GETDESKWALLPAPER,len(ubuf),ubuf,0)
return ubuf.value
def setWallpaper(path):
changed = win32con.SPIF_UPDATEINIFILE | win32con.SPIF_SENDCHANGE
ctypes.windll.user32.SystemParametersInfoW(win32con.SPI_SETDESKWALLPAPER,0,path,changed)
Alternatively: (with SystemParametersInfoA)
def getWallpaper():
sbuf = ctypes.create_string_buffer(512) # ctypes.c_buffer(512)
ctypes.windll.user32.SystemParametersInfoA(win32con.SPI_GETDESKWALLPAPER,len(sbuf),sbuf,0)
return sbuf.value
def setWallpaper(path):
changed = win32con.SPIF_UPDATEINIFILE | win32con.SPIF_SENDCHANGE
ctypes.windll.user32.SystemParametersInfoA(win32con.SPI_SETDESKWALLPAPER,0,path.encode(),changed) # "".encode() = b""
Arguments are:
SystemParametersInfo(SetOrGet, GetBufferSize, SetBufferOrGetBuffer, SetChange)
The path has to be absolute, so if you're using something relative to your script, do:
path = os.path.abspath(path)
To see more stuff you can do with SystemParametersInfo, see the docs.
(near the bottom there's an example to change the mouse speed)
P.S. There are many answers already here, but they're leaving out the broadcasting you're supposed to do. Sure it works without it, but it's bad practice not to use it properly.
P.P.S And they only gave hard coded values, rather than the variables they come from.
Also note, i use 512 characters for the buffer size when getting the path, just to be more safe since paths might exceed 256. I doubt anyone will have paths as long as that though.
One more note. I've only tested the above examples in Python 3, but i don't think SystemParametersInfoA needs the .encode() in Python 2. (they updated strings in Python 3 to unicode i believe) The string in SystemParametersInfoW may need converting for Python 2.
I read all the answers and after searching for a while i found a easier solution.
Install the module named py-wallpaper.
pip install py-wallpaper
Import the module.
from wallpaper import set_wallpaper, get_wallpaper
set the wallpaper using set walpaper
set_wallpaper("location/to/image.jpg")
get the current wallpaper's path using get wallpaper
print(get_wallpaper())
thanks.
changing the background image of desktop
import ctypes
import os
SPI_SETDESKWALLPAPER = 20
ctypes.windll.user32.SystemParametersInfoA(SPI_SETDESKWALLPAPER, 0, 'your image path', 3)
#'C:\\Users\\Public\\Pictures\\abc.jpg'
it worked fine for me. windows10, python27
On Windows with python2.5 or higher, use ctypes to load user32.dll and call
import ctypes
ctypes.windll.user32.SystemParametersInfoW(20,0,"Path_wallpaper", 0)
speak("Background changed succesfully")
Just adding a small precision to ShivaGuntuku 's post :
In python 3 you should replace the 'A' by a 'W' in SytemParametersInfoA. Small exemple to change your desktop background in windows10 with python 3 :
import ctypes
import os
SPI_SETDESKWALLPAPER = 20
ctypes.windll.user32.SystemParametersInfoW(
SPI_SETDESKWALLPAPER, 0, 'C:\\Users\\godet\\OneDrive\\Images\\breaker_wall.jpg', 0)
this works for me
import ctypes
ctypes.windll.user32.SystemParametersInfoW(20,0,path:os.PathLike,3)
You can use this library PyWallpaper, worked for me on mac also.
To install type pip install PyWallpaper.
And then to change/set your wallpaper -
from PyWallpaper import change_wallpaper
change_wallpaper("/some_path/sample.jpg")