I am a beginner and I have a question how to make a "for" loop to check the following condition for me:
if the folder does not exist, create it
if there is a folder, create a file in it
and check if the folder exists again
a) the folder exists, create a file in it
b) does not exist, create a folder, create a file
import os
import datetime
now = datetime.datetime.now().strftime("%Y-%m-%d")
print(now)
directory = "D:/Dir/Dir_" + now
print(directory) # D:/Dir/Dir_2021-11-12
if not os.path.exists(directory):
os.mkdir(directory)
print("Created folder: " + directory)
else:
print("Folder exists")
now_datetime = datetime.datetime.now().strftime("%Y-%m-%d_%H%M")
print("Current date: " + now_datetime)
full_path = directory + "/file_" + now_datetime + ".txt"
print(full_path) # D:/Dir/Dir_2021-11-12/file_2021-11-12_1000.txt
output = print("Something ...")
print(output)
plik = open(full_path,"w+")
plik.write(output)
plik.close()
Ultimately, I would like to make a process that will run all the time, because I want to read messages from the server. So, for example, when I get messages from the server, it will write them to a file and then the loop will check if there is a folder for a given day. If it exists, new files will be created in this folder. Until a new folder appears. Then the files will start falling into a new folder.
Can anyone help me? Somehow guide as the easiest way to do this?
Maybe use a while-loop, either running infinitely or until a certain condition is meet. See this simple example:
import time
x = 0
while x < 10: # while(true)
x += 1
time.sleep(1)
print(x)
Add the time.sleep to make the loop sleep so that it does not run continuously
Related
The following code creates the path: /Users/michellegautier/ACHFiles/26207428-8f40-11e7-8d57-f354c8f15cfa/outbox/PPSACHCOMSpec/mt.ach.06302020.??????_8172363248911097723, or similar.
We are waiting for this file to be created. In the initial time.sleep(1), if I change it to 10, the if not os.path.isfile(filepath) statement returns false as because it finds the file and everything works great.
BUT, if I shorten this time to 1 and the file isn't there yet, I'm trying to find it after it is created. Even if I add large pauses and ensure the file is created, the following if os.path.isfile(filepath): in the for loop NEVER passes and I don't know why.
I've tried adjusting the statements to exists, check for length of file, etc but no matter what, if it doesn't find it in the first check, it never finds the file in subsequent checks even after it is added. Please help!
home = str(Path.home())
file_date = strftime('%m%d%Y')
if company_profile == 'from_storage':
company_profile = context.storage['company_profile_id']
filepath = home + '/ACHFiles/' + context.providerID + '/outbox/PPSACHCOMSpec/mt.ach.' + file_date + '.??????_{}'.format(
company_profile)
time.sleep(int(1))
for name in glob.glob(filepath):
filepath = name
if not os.path.isfile(filepath):
for i in range(3):
time.sleep(3)
if os.path.isfile(filepath):
print('File found: ', filepath)
break
The problem is with your use of glob.glob(). It will only return files that already exist. If none of the matching files exist when you start, glob.glob() won't return anything, and filepath will still be set to the wildcard. os.path.isfile() doesn't do wildcard matching, so it will never succeed.
You need to call glob.glob() in the retry loop, not before the loop.
max_retries = 4
time.sleep(1)
for _ in range(max_retries):
files = glob.glob(filepath):
if len(files) > 0:
filepath = files[-1]
print("File found: " + filepath)
break
time.sleep(3)
else:
print("File not found")
I am aiming to create a function that does the following:
Declare a path with a file, not just a folder. e.g. 'C:/Users/Lampard/Desktop/Folder1/File.py'
Create a folder in same folder as the declared file path - Calling it 'Archive'
Cut the file and paste it into the new folder just created.
If the folder 'Archive' already exists - then simply cut and paste the file into there
I have spent approx. 15-20min going through these:
https://www.programiz.com/python-programming/directory
Join all except last x in list
https://docs.python.org/3/library/pathlib.html#operators
And here is what I got to:
import os
from pathlib import Path, PurePath
from shutil import copy
#This path will change every time - just trying to get function right first
path = 'C:/Users/Lampard/Desktop/Folder1/File.py'
#Used to allow suffix function
p = PurePath(path)
#Check if directory is a file not a folder
if not p.suffix:
print("Not an extension")
#If it is a file
else:
#Create new folder before last file
#Change working directory
split = path.split('/')
new_directory = '/'.join(split[:-1])
apply_new_directory = os.chdir(new_directory)
#If folder does not exist create it
try:
os.mkdir('Archive')#Create new folder
#If not, continue process to copy file and paste it into Archive
except FileExistsError:
copy(path, new_directory + '/Archive/' + split[-1])
Is this code okay? - does anyone know a simpler method?
Locate folder/file in path
print [name for name in os.listdir(".") if os.path.isdir(name)]
Create path
import os
# define the name of the directory to be created
path = "/tmp/year"
try:
os.mkdir(path)
except OSError:
print ("Creation of the directory %s failed" % path)
else:
print ("Successfully created the directory %s " % path)
To move and cut files you can use this library
As you're already using pathlib, there's no need to use shutil:
from pathlib import Path
path = 'C:/Users/Lampard/Desktop/Folder1/File.py' # or whatever
p = Path(path)
target = Path(p.with_name('Archive')) # replace the filename with 'Archive'
target.mkdir() # create target directory
p.rename(target.joinpath(p.name)) # move the file to the target directory
Feel free to add appriopriate try…except statements to handle any errors.
Update: you might find this version more readable:
target = p.parent / 'Archive'
target.mkdir()
p.rename(target / p.name)
This is an example of overloading / operator.
My code needs to look for 3 files with specific names in a directory.For example: 123_xy_report.xlsm, 789_ab_file.xlsm, 222_ftp_review.xlsm.
Only when all these files arrive, continue or else keep looking for the files until they arrive.
I have tried glob and fnmatch, but unable to get it right to look for these specific files.
while True:
os.chdir(dir)
filelist=['08282019_xy_report.xlsm','08282019_ab_file.xlsm','08282019_ftp_review.xlsm']
if all([os.path.isfile(f)for f in filelist]):
print('All files exists')
break
else:
print('waiting for files to arrive')
time.sleep(60)
The files will have the datetime as prefix,so the files comes with new date every day.
So the files changes names after the date but however arrive at random times. Then you need to make a list with names formatted to your files with the current days date. This works. I am sure it can be made shorter but because of the filename requirements this is the best I came up with. Hope it helps!
import os
from datetime import date
import time
dir = ""
names = ["_xy_report.xlsm", "_ab_file.xlsm", "_ftp_review.xlsm"]
formatted_list = []
while True:
# Change to needed directory
os.chdir(dir)
# To days date formatted to list
days_date = str(date.today()).replace('-', '')
for n in names:
formatted_list.append(f"{days_date + n}")
# check if all files has arrived yet
if all([os.path.isfile(f)for f in formatted_list]):
print("All files exist")
break
else:
print(f"Waiting for {formatted_list} to arrive")
time.sleep(60)
Almost there...
If you are looking for all of those files and then, and only then, start doing work, then it doesn't matter in which order you wait for them.
I suggest this simple solution:
import os.path
import time
filelist = ['1.txt', '2.txt', '3.txt']
for f in filelist:
print('waiting for ' + f)
while not os.path.isfile(f):
time.sleep(2)
print(f + ' found')
print('all files found')
This script need to be in the same folder you search OR you can append the absolute path to the file names OR chdir to that directory
Im trying to create a program that deletes files after X days. I got that part working, but now I need to write a code that deletes the file they're located at ONLY if it's empty
This is for a server that is taking log files from builds, and those are getting placed in thousands of other folders. I need to delete those upper level folders if they are empty
import os
import shutil
import sys
import time
#for path you need to use the location of the log files
path = "C:/GroupData"
# Check current working directory.
retval = os.getcwd()
#this will list the name of the current directory
print("Current working directory %s" % retval)
# Now change the directory
#You will put the same location as you did for path
os.chdir('C:/GroupData/Temp')
workdir = os.getcwd()
# Check current working directory.
retval = os.getcwd()
#this will list the new name of the working directory
print("Directory changed successfully %s" % retval)
from subprocess import call
def get_file_directory(file):
return os.path.dirname(os.path.abspath(file))
#this code is getting what today's date is
now = time.time()
#this code is saying that files that are older than todays date by 7 days will be deleted
cutoff = now - (10 * 86400)
files = os.listdir(os.path.join(get_file_directory('C://GroupData//Temp'), "temp"))
file_path = os.path.join(get_file_directory('C://GroupData//Temp'), "temp/")
#this locates what the file name is and looks to see how long ago it was modified
for xfile in files:
files1 = os.listdir(os.path.join(get_file_directory('C://GroupData//Temp//' + xfile), xfile))
file_path1 = os.path.join(get_file_directory('C://GroupData//Temp//' + xfile), xfile)
for xfile1 in files1:
if os.path.isfile(str(file_path1) + "\\" + xfile1):
t = os.stat(str(file_path1) + "\\" + xfile1)
#m is how long ago it was last modified
m = t.st_mtime
#if the cutoff date is older than the modified date the file will be deleted
if m < cutoff:
#if the file IS older than the cutoff os.remove deletes the file from the path
os.remove(str(file_path1) + "\\" + xfile1)
files = os.listdir(os.path.join(get_file_directory('C://GroupData//Temp'), "temp"))
file_path = os.path.join(get_file_directory('C://GroupData//Temp'), "temp/")
os.rmdir(str(file_path1) + "\\")
The code at the bottom works, but only for one file at a time, and I need it to do it as many times as possible so that it can delete all the empty files at once, as this will be run automatically
First of: I know of pyinotify.
What I want is an upload service to my home server using Dropbox.
I will have a Dropbox's shared folder on my home server. Everytime someone else, who is sharing that folder, puts anything into that folder, I want my home server to wait until it is fully uploaded and move all the files to another folder, and removing those files from the Dropbox folder, thus, saving Dropbox space.
The thing here is, I can't just track for changes in the folder and move the files right away, because if someone uploads a large file, Dropbox will already start downloading and therefore showing changes in the folder on my home server.
Is there some workaround? Is that somehow possible with the Dropbox API?
Haven't tried it myself, but the Dropbox CLI version seems to have a 'filestatus' method to check for current file status. Will report back when I have tried it myself.
There is a Python dropbox CLI client, as you mentioned in your question. It returns "Idle..." when it isn't actively processing files. The absolutely simplest mechanism I can imagine for achieving what you want would be a while loop that checked the output of dropbox.py filestatus /home/directory/to/watch and performed an scp of the contents and then a delete on the contents if that suceeded. Then slept for five minutes or so.
Something like:
import time
from subprocess import check_call, check_output
DIR = "/directory/to/watch/"
REMOTE_DIR = "user#my_server.com:/folder"
While True:
if check_output(["dropbox.py", "status", DIR]) == "\nIdle...":
if check_call(["scp", "-r", DIR + "*", REMOTE_DIR]):
check_call(["rm", "-rf", DIR + "*"])
time.sleep(360)
Of course I would be very careful when testing something like this, put the wrong thing in that second check_call and you could lose your filesystem.
You could run incrond and have it wait for IN_CLOSE_WRITE events in your Dropbox folder. Then it would only be triggered when a file transfer completed.
Here is a Ruby version that doesn't wait for Dropbox to be idle, therefore can actually start moving files, while it is still syncing. Also it ignores . and ... It actually checks the filestatus of each file within a given directory.
Then I would run this script either as a cronjob or in a separate screen.
directory = "path/to/dir"
destination = "location/to/move/to"
Dir.foreach(directory) do |item|
next if item == '.' or item == '..'
fileStatus = `~/bin/dropbox.py filestatus #{directory + "/" + item}`
puts "processing " + item
if (fileStatus.include? "up to date")
puts item + " is up to date, starting to move file now."
# cp command here. Something along this line: `cp #{directory + "/" + item + destination}`
# rm command here. Probably you want to confirm that all copied files are correct by comparing md5 or something similar.
else
puts item + " is not up to date, moving on to next file."
end
end
This is the full script, I ended up with:
# runs in Ruby 1.8.x (ftools)
require 'ftools'
directory = "path/to/dir"
destination = "location/to/move/to"
Dir.glob(directory+"/**/*") do |item|
next if item == '.' or item == '..'
fileStatus = `~/bin/dropbox.py filestatus #{item}`
puts "processing " + item
puts "filestatus: " + fileStatus
if (fileStatus.include? "up to date")
puts item.split('/',2)[1] + " is up to date, starting to move file now."
`cp -r #{item + " " + destination + "/" + item.split('/',2)[1]}`
# remove file in Dropbox folder, if current item is not a directory and
# copied file is identical.
if (!File.directory?(item) && File.cmp(item, destination + "/" + item.split('/',2)[1]).to_s)
puts "remove " + item
`rm -rf #{item}`
end
else
puts item + " is not up to date, moving to next file."
end
end