As an newcomer to python I figured I'd write a little python3 script to help me switch directories on the command line (ubuntu trusty). Unfortunately os.chdir() does not seems to work.
I've tried tinkering with it in various ways such as placing quotes around the path, removing the leading slash (which obviously doesn't work) and even just hardcoding it, but I can't get it to work - can anybody tell me what I'm missing here?
The call to chdir() happens towards the end - you can see the code in github too
#!/usr/bin/env python3
# #python3
# #author sabot <sabot#inuits.eu>
"""Switch directories without wearing out your slash key"""
import sys
import os
import json
import click
__VERSION__ = '0.0.1'
# 3 params are needed for click callback
def show_version(ctx, param, value):
"""Print version information and exit."""
if not value:
return
click.echo('Goto %s' % __VERSION__)
ctx.exit() # quit the program
def add_entry(dictionary, filepath, path, alias):
"""Add a new path alias."""
print("Adding alias {} for path {} ".format(alias,path))
dictionary[alias] = path
try:
jsondata = json.dumps(dictionary, sort_keys=True)
fd = open(filepath, 'w')
fd.write(jsondata)
fd.close()
except Exception as e:
print('Error writing to dictionary file: ', str(e))
pass
def get_entries(filename):
"""Get the alias entries in json."""
returndata = {}
if os.path.exists(filename) and os.path.getsize(filename) > 0:
try:
fd = open(filename, 'r')
entries = fd.read()
fd.close()
returndata = json.loads(entries)
except Exception as e:
print('Error reading dictionary file: ', str(e))
pass
else:
print('Dictionary file not found or empty- spawning new one in', filename)
newfile = open(filename,'w')
newfile.write('')
newfile.close()
return returndata
#click.command()
#click.option('--version', '-v', is_flag=True, is_eager=True,
help='Print version information and exit.', expose_value=False,
callback=show_version)
#click.option('--add', '-a', help="Add a new path alias")
#click.option('--target', '-t', help="Alias target path instead of the current directory")
#click.argument('alias', default='currentdir')
#click.pass_context
def goto(ctx, add, alias, target):
'''Go to any directory in your filesystem'''
# load dictionary
filepath = os.path.join(os.getenv('HOME'), '.g2dict')
dictionary = get_entries(filepath)
# add a path alias to the dictionary
if add:
if target: # don't use current dir as target
if not os.path.exists(target):
print('Target path not found!')
ctx.exit()
else:
add_entry(dictionary, filepath, target, add)
else: # use current dir as target
current_dir = os.getcwd()
add_entry(dictionary, filepath, current_dir, add)
elif alias != 'currentdir':
if alias in dictionary:
entry = dictionary[alias]
print('jumping to',entry)
os.chdir(entry)
elif alias == 'hell':
print("Could not locate C:\Documents and settings")
else:
print("Alias not found in dictionary - did you forget to add it?")
if __name__ == '__main__':
goto()
The problem is not with Python, the problem is that what you're trying to do is impossible.
When you start a Python interpreter (script or interactive REPL), you do so from your "shell" (Bash etc.). The shell has some working directory, and it launches Python in the same one. When Python changes its own working directory, it does not affect the parent shell, nor do changes in the working directory of the shell affect Python after it has started.
If you want to write a program which changes the directory in your shell, you should define a function in your shell itself. That function could invoke Python to determine the directory to change to, e.g. the shell function could be simply cd $(~/myscript.py) if myscript.py prints the directory it wants to switch to.
Here's Python 3 version of #ephemient's C solution:
#!/usr/bin/env python3
"""Change parent working directory."""
#XXX DIRTY HACK, DO NOT DO IT
import os
import sys
from subprocess import Popen, PIPE, DEVNULL, STDOUT
gdb_cmd = 'call chdir("{dir}")\ndetach\nquit\n'.format(dir=sys.argv[1])
with Popen(["gdb", "-p", str(os.getppid()), '-q'],
stdin=PIPE, stdout=DEVNULL, stderr=STDOUT) as p:
p.communicate(os.fsencode(gdb_cmd))
sys.exit(p.wait())
Example:
# python3 cd.py /usr/lib && python3 -c 'import os; print(os.getcwd())'
Related
I'm trying to run my python script against specific folder when I specify the folder name in the terminal e.g
python script.py -'folder'
python script.py 'folder2'
'folder' being the I folder I would like to run the script in. Is there a command line switch that I must use?
The cd command in the shell switches your current directory.
Perhaps see also What exactly is current working directory?
If you would like your Python script to accept a directory argument, you'll have to implement the command-line processing yourself. In its simplest form, it might look something like
import sys
if len(sys.argv) == 1:
mydir = '.'
else:
mydir = sys.argv[1]
do_things_with(mydir)
Usually, you would probably wrap this in if __name__ == '__main__': etc and maybe accept more than one directory and loop over the arguments?
import sys
from os import scandir
def how_many_files(dirs):
"""
Show the number of files in each directory in dirs
"""
for adir in dirs:
try:
files = list(scandir(adir))
except (PermissionError, FileNotFoundError) as exc:
print('%s: %s' % (adir, exc), file=sys.stderr)
continue
print('%s: %i directory entries' % (adir, len(files)))
if __name__ == '__main__':
how_many_files(sys.argv[1:] or ['.'])
I am using the following function to allow the OS to open a third party application associated with the filetype in question. For example: If variable 'fileToOpen' links to a file (it's full path of course) called flower.psd, this function would open up Photoshop in Windows and Gimp in Linux (typically).
def launchFile(fileToOpen):
if platform.system() == 'Darwin': # macOS
subprocess.call(('open', fileToOpen))
elif platform.system() == 'Windows': # Windows
os.startfile(fileToOpen)
else: # linux variants
subprocess.call(('xdg-open', fileToOpen))
While it is running, I want to have the same python script monitor the use of that file and delete it once the third party app is done using it (meaning...the 3rd party app closed the psd file or the third party app itself closed and released the file from use).
I've tried using psutil and pywin32 but neither seem to work in Windows 10 with Python3.9. Does anyone have any success with this? If so, how did you go about getting the process of the third party app while not getting a permission error from Windows?
Ideally, I would like to get a solution that works across Windows, Macs, and Linux but I'll take any help with Windows 10 for now since Mac and Linux can be found easier with commandline assistance with the ps -ax | grep %filename% command
Keep in mind, this would ideally track any file. TIA for your help.
Update by request:
I tried adding this code to mine (from a previous suggestion). Even this alone in a python test.py file spits out permission errors:
import psutil
for proc in psutil.process_iter():
try:
# this returns the list of opened files by the current process
flist = proc.open_files()
if flist:
print(proc.pid,proc.name)
for nt in flist:
print("\t",nt.path)
# This catches a race condition where a process ends
# before we can examine its files
except psutil.NoSuchProcess as err:
print("****",err)
The follow code below does not spit out an error but does not detect a file in use:
import psutil
from pathlib import Path
def has_handle(fpath):
for proc in psutil.process_iter():
try:
for item in proc.open_files():
if fpath == item.path:
return True
except Exception:
pass
return False
thePath = Path("C:\\Users\\someUser\\Downloads\\Book1.xlsx")
fileExists = has_handle(thePath)
if fileExists :
print("This file is in use!")
else :
print("This file is not in use")
Found it!
The original recommendation from another post forgot one function..."Path" The item.path from the process list is returned as a string. This needs to convert to a Path object for comparison of your own path object.
Therefore this line:
if fpath == item.path:
Should be:
if fpath == Path(item.path):
and here is the full code:
import psutil
from pathlib import Path
def has_handle(fpath):
for proc in psutil.process_iter():
try:
for item in proc.open_files():
print (item.path)
if fpath == Path(item.path):
return True
except Exception:
pass
return False
thePath = Path("C:\\Users\\someUser\\Downloads\\Book1.xlsx")
fileExists = has_handle(thePath)
if fileExists :
print("This file is in use!")
else :
print("This file is not in use")
Note: The reason to use Path objects rather than a string is to stay OS independant.
Based on #Frankie 's answer I put together this script. The script above took 16.1 seconds per file as proc.open_files() is quite slow.
The script below checks all files in a directory and returns the pid related to each open file. 17 files only took 2.9s to check. This is due to only calling proc.open_files() if the files default app is open in memory.
As this is used to check if a folder can be moved, the pid can be later used to force close the locking application but BEWARE that that application could have other documents open and all data would be lost.
This does not detect open txt files or may not detect files that dont have a default application
from pathlib import Path
import psutil
import os
import shlex
import winreg
from pprint import pprint as pp
from collections import defaultdict
class CheckFiles():
def check_locked_files(self, path: str):
'''Check all files recursivly in a directory and return a dict with the
locked files associated with each pid (proocess id)
Args:
path (str): root directory
Returns:
dict: dict(pid:[filenames])
'''
fnames = []
apps = set()
for root, _, f_names in os.walk(path):
for f in f_names:
f = Path(os.path.join(root, f))
if self.is_file_in_use(f):
default_app = Path(self.get_default_windows_app(f.suffix)).name
apps.add(default_app)
fnames.append(str(f))
if apps:
return self.find_process(fnames, apps)
def find_process(self, fnames: list[str], apps: set[str]):
'''find processes for each locked files
Args:
fnames (list[str]): list of filepaths
apps (set[str]): set of default apps
Returns:
dict: dict(pid:[filenames])
'''
open_files = defaultdict(list)
for p in psutil.process_iter(['name']):
name = p.info['name']
if name in apps:
try:
[open_files[p.pid].append(x.path) for x in p.open_files() if x.path in fnames]
except:
continue
return dict(open_files)
def is_file_in_use(self, file_path: str):
'''Check if file is in use by trying to rename it to its own name (nothing changes) but if
locked then this will fail
Args:
file_path (str): _description_
Returns:
bool: True is file is locked by a process
'''
path = Path(file_path)
if not path.exists():
raise FileNotFoundError
try:
path.rename(path)
except PermissionError:
return True
else:
return False
def get_default_windows_app(self, suffix: str):
'''Find the default app dedicated to a file extension (suffix)
Args:
suffix (str): ie ".jpg"
Returns:
None|str: default app exe
'''
try:
class_root = winreg.QueryValue(winreg.HKEY_CLASSES_ROOT, suffix)
with winreg.OpenKey(winreg.HKEY_CLASSES_ROOT, r'{}\shell\open\command'.format(class_root)) as key:
command = winreg.QueryValueEx(key, '')[0]
return shlex.split(command)[0]
except:
return None
old_dir = r"C:\path_to_dir"
c = CheckFiles()
r = c.check_locked_files(old_dir)
pp(r)
I am trying to unzip fasta.gz files in order to work with them. I have created a script using cmd base on something I have done before but now I cannot manage to work the newly created function. See below:
import glob
import sys
import os
import argparse
import subprocess
import gzip
#import gunzip
def decompressed_files():
print ('starting decompressed_files')
#files where the data is stored
input_folder=('/home/me/me_files/PB_assemblies_for_me')
#where I want my data to be
output_folder=input_folder + '/fasta_files'
if os.path.exists(output_folder):
print ('folder already exists')
else:
os.makedirs(output_folder)
print ('folder has been created')
for f in input_folder:
fasta=glob.glob(input_folder + '/*.fasta.gz')
#print (fasta[0])
#sys.exit()
cmd =['gunzip', '-k', fasta, output_folder]
my_file=subprocess.Popen(cmd)
my_file.wait
decompressed_files()
print ('The programme has finished doing its job')
But this give the following error:
TypeError: execv() arg 2 must contain only strings
If I write fasta, the programme looks for a file an the error becomes:
fasta.gz: No such file or directory
If I go to the directory where I have the files and I key gunzip, name_file_fasta_gz, it does the job beautifully but I have a few files in the folder and I would like to create the loop. I have used 'cmd' before as you can see in the code below and I didn't have any problem with it. Code from the past where I was able to put string, and non-string.
cmd=['velveth', output, '59', '-fastq.gz', '-shortPaired', fastqs[0], fastqs[1]]
#print cmd
my_file=subprocess.Popen(cmd)#I got this from the documentation.
my_file.wait()
I will be happy to learn other ways to insert linux commands within a python function. The code is for python 2.7, I know it is old but it is the one is install in the server at work.
fasta is a list returned by glob.glob().
Hence cmd = ['gunzip', '-k', fasta, output_folder] generates a nested list:
['gunzip', '-k', ['foo.fasta.gz', 'bar.fasta.gz'], output_folder]
but execv() expects a flat list:
['gunzip', '-k', 'foo.fasta.gz', 'bar.fasta.gz', output_folder]
You can use the list concentration operator + to create a flat list:
cmd = ['gunzip', '-k'] + fasta + [output_folder]
I haven't tested this but it might solve you unzip problem using command.
command gunzip -k is to keep both the compressed and decompressed file then what is the purpose of output directory.
import subprocess
import gzip
def decompressed_files():
print('starting decompressed_files')
# files where the data is stored
input_folder=('input')
# where I want my data to be
output_folder = input_folder + '/output'
if os.path.exists(output_folder):
print('folder already exists')
else:
os.makedirs(output_folder)
print('folder has been created')
for f in os.listdir(input_folder):
if f and f.endswith('.gz'):
cmd = ['gunzip', '-k', f, output_folder]
my_file = subprocess.Popen(cmd)
my_file.wait
print(cmd) will look as shown below
['gunzip', '-k', 'input/sample.gz', 'input/output']
I have a few files in the folder and I would like to create the loop
From above quote your actual problem seems to be unzip multiple *.gz files from path
in that case below code should solve your problem.
import os
import shutil
import fnmatch
def gunzip(file_path,output_path):
with gzip.open(file_path,"rb") as f_in, open(output_path,"wb") as f_out:
shutil.copyfileobj(f_in, f_out)
def make_sure_path_exists(path):
try:
os.makedirs(path)
except OSError:
if not os.path.isdir(path):
raise
def recurse_and_gunzip(input_path):
walker = os.walk(input_path)
output_path = 'files/output'
make_sure_path_exists(output_path)
for root, dirs, files in walker:
for f in files:
if fnmatch.fnmatch(f,"*.gz"):
gunzip(root + '/' + f, output_path + '/' + f.replace(".gz",""))
recurse_and_gunzip('files')
source
EDIT:
Using command line arguments -
subprocess.Popen(base_cmd + args) :
Execute a child program in a new process. On Unix, the class uses os.execvp()-like behavior to execute the child program
fasta.gz: No such file or directory
So any extra element to cmd list is treated as argument and gunzip will look for argument.gz file hence the error fasta.gz file not found.
ref and some useful examples
Now if you want to pass gz files as command line argument you can still do that with below code( you might need to polish little bit as per your need)
import argparse
import subprocess
import os
def write_to_desired_location(stdout_data,output_path):
print("Going to write to path", output_path)
with open(output_path, "wb") as f_out:
f_out.write(stdout_data)
def decompress_files(gz_files):
base_path=('files') # my base path
output_path = base_path + '/output' # output path
if os.path.exists(output_path):
print('folder already exists')
else:
os.makedirs(output_path)
print('folder has been created')
for f in gz_files:
if f and f.endswith('.gz'):
print('starting decompressed_files', f)
proc = subprocess.Popen(['gunzip', '-dc', f], stdout=subprocess.PIPE) # d:decompress and c:stdout
write_to_desired_location(proc.stdout.read(), output_path + '/' + f.replace(".gz", ""))
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument(
"-gzfilelist",
required=True,
nargs="+", # 1 or more arguments
type=str,
help='Provide gz files as arguments separated by space Ex: -gzfilelist test1.txt.tar.gz test2.txt.tar.gz'
)
args = parser.parse_args()
my_list = [str(item)for item in args.gzfilelist] # converting namedtuple into list
decompress_files(gz_files=my_list)
execution:
python unzip_file.py -gzfilelist test.txt.tar.gz
output
folder already exists
('starting decompressed_files', 'test.txt.tar.gz')
('Going to write to path', 'files/output/test.txt.tar')
You can pass multiple gz files as well for example
python unzip_file.py -gzfilelist test1.txt.tar.gz test2.txt.tar.gz test3.txt.tar.gz
Objective
This Python program writes 97 unix commands to 97 separate lists to be processed with the subprocess Popen/run function.
The unix commands written call an application, commands for the application, and in/out files the application needs to run.
The program checks for whether or not a filename is in a global list, and if not, runs the program with subprocess.Popen([each of the 97 unix commmands]), and inserts the file name into the list.
import subprocess
import time
import sys
import os
def readFilenames(sent_directory):
for root, dirs, files in os.walk(sent_directory, topdown=False):
for name in files:
currentFilePath = (os.path.abspath(os.path.join(root, name)))
currentFileName = (os.path.join(root, name))[21:-5]
scriptEx(currentFilePath, currentFileName)
def scriptEx(path, name):
global idxlist
idx = name + 'idx'
args = (['application', '[CMD]', '[CMD]', idx, path])
process(idx, args, idxlist)
def process(idx, args, idxlist):
if idx in idxlist:
return
else:
subprocess.Popen(args)
idxlist.append(idx)
time.sleep(400)
if __name__ == '__main__':
directory_name=sys.argv[0]
try:
directory_name=sys.argv[1]
except:
print('\nPlease pass directory name\n\n')
idxlist = []
readFilenames(directory_name)
Output
Among the output printed to the screen in real time from the application, there is a line I do not encounter when running the commands singly.
Error: index output file could not be opened!
There are no output files from the application as there should be when I run this Python program.
Failed Solutions
I have found one source mentioning this specific error. The only reply was asking OP if he/she could touch input_file.
Thinking it was a permissions problem, I chmod 777 everything to no avail, and touch all_input_files.
Also, I have tried subprocess.call, subprocess.run, and subprocess.check_output.
I'm currently putting together a script in Python which will do the following:-
Create a directory in my Dropbox folder called 'Spartacus'
Create a subdirectory in this location with the naming convention of the date and time of creation
Within this directory, create a file called iprecord.txt and information will then be written to this file.
Here is my code thusfar using Python v2.7 on Windows 7:-
import os
import time
import platform
import urllib
current_dir = os.getcwd()
targetname = "Spartacus"
target_dir = os.path.join(current_dir, targetname)
timenow = time.strftime("\%d-%b-%Y %H-%M-%S")
def directoryVerification():
os.chdir(current_dir)
try:
os.mkdir('Spartacus')
except OSError:
pass
try:
os.system('attrib +h Spartacus')
except OSError:
pass
def gatherEvidence():
os.chdir(target_dir)
try:
evidential_dir = os.mkdir(target_dir + timenow)
os.chdir(evidential_dir)
except OSError:
pass
f = iprecord.txt
with f as open:
ip_addr = urllib.urlopen('http://www.biranchi.com/ip.php').read()
f.write("IP Address:\t %s\t %s" % ip_addr, time.strftime("\%d-%b-%Y %H-%M-%S"))
x = directoryVerification()
y = gatherEvidence()
I keep on getting an error in line 26 whereby it cannot resolve the full path to the dynamically named directory (date and time) one. I've printed out the value of 'evidential_dir' and it shows as being Null.
Any pointers as to where I am going wrong? Thanks
PS: Any other advice on my code to improve it would be appreciated
PPS: Any advice on how to locate the default directory for 'Dropbox'? Is there a way of scanning a file system for a directory called 'Dropbox' and capturing the path?
os.mkdir() does not return a pathname as you might be thinking. It seems like you do inconsistent methods of the same thing at different spots of your code.
Try this:
evidential_dir = os.path.join(target_dir, timenow)
os.mkdir(evidential_dir)
And fix your other line:
f = "iprecord.txt"
os.mkdir doesn't return anything.
evidential_dir = target_dir + timenow
try:
os.mkdir(evidential_dir)
except OSError:
pass
os.chdir(evidential_dir)