I was trying to iterate over the files in a directory like this:
with open(video_list, 'r') as imf:
index = []
for id, line in enumerate(imf):
video_label = line.strip().split()
video_name = video_label[0] # name of video
label = rectify_label[video_label[1]] # label of video
video_path = os.path.join(video_root, video_name) # video_path is the path of each video
### for sampling triple imgs in the single video_path ####
img_lists = os.listdir(video_path)
img_lists.sort() # sort files by ascending
img_count = len(img_lists) # number of frames in video
num_per_part = int(img_count) // 3
But Python was throwing FileNotFoundError even though the file exists:
---> 36 img_lists = os.listdir(video_path)
37 img_lists.sort() # sort files by ascending
38 img_count = len(img_lists) # number of frames in video
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:/Users/m_rayeni/Emotion-FAN/Data/Train/Fear/013736360'
So what is wrong here?
I think the error here is that you are using forward Slash in the File name instead of back slash.
Your current path is "C:/Users/m_rayeni/Emotion-FAN/Data/Train/Fear/013736360"
It should be "C:\Users\m_rayeni\Emotion-FAN\Data\Train\Fear\013736360"
In all likeliness this won't work either and you would need to escape the backslash like so "C:\\Users\\m_rayeni\\Emotion-FAN\\Data\\Train\\Fear\\013736360"
Or use raw stings like - r"C:\Users\m_rayeni\Emotion-FAN\Data\Train\Fear\013736360"
Related
This is how I am currently doing it but it doesn't seem to work: First I load two binaries as python objects. Then I add them and form a new python object. Then I save this object as a binary. The resulting binary does not form properly. It throws error when I load it and try to further process it. When out of curiosity, I checked the size of the resulting binary, it was much smaller than the component binaries. For example, if the binaries I am adding are 2.27 GB and 40 MB, the resulting binary I get is sometimes a few MBs and at best, a GB. Below is my code:
inputfile1 = open('/path/to/binary/texts1','rb')
inputfile2 = open('/path/to/binary2/texts2','rb')
texts1 = pickle.load(infile1)
texts2 = pickle.load(infile2)
texts = texts1 + texts2
outputfile = open('/Path/to/new/binary/newtext','wb')
pickle.dump(texts,outputfile)
The code I used to create the INPUT binaries is as follows:
my_path = "/path/to/pdfs"
files = os.listdir(my_path)
doc_set = []
for file in files:
newpath = (os.path.join(my_path, file))
newpath1 = textract.process(newpath)
newpath2 = newpath1.decode(encoding='utf-8')
doc_set.append(newpath2)
texts = []
for i in doc_set:
raw = i.lower()
tokens = tokenizer.tokenize(raw)
stopped_tokens = [i for i in tokens if not i in stopwords.words()]
stemmed_tokens = [p_stemmer.stem(i) for i in stopped_tokens]
texts.append(stemmed_tokens)
outfile2 = open('texts1','wb')
pickle.dump(texts, outfile2)
outfile2.close()
This is the error I get:
EOFError Traceback (most recent call last)
<ipython-input-4-9077951a4a43> in <module>
54
55 textfile = open('/path/to/binary','rb')
---> 56 texts = pickle.load(textfile)
57 textfile.close()
58
EOFError: Ran out of input
I'm trying to diff with three files in Python on Linux.
I've got (a,b,c) three files such as
/work/a
/work/b
/work/c
The name of a file consists of absolute paths of some hex files
(for example of a file)
user/work/test0/.../bin0.hex
user/work/test0/.../bin1.hex
user/work/test0/.../bin2.hex
user/work/test0/.../bin3.hex
...
The name of b file consists of absolute path of some hex files
(for example of b file)
user/work/test1/.../bin0.hex
user/work/test1/.../bin1.hex
user/work/test1/.../bin2.hex
user/work/test1/.../bin3.hex
...
The name of c file consists of absolute path of some hex files
(for example of c)
user/work/test2/.../bin0.hex
user/work/test2/.../bin1.hex
user/work/test2/.../bin2.hex
user/work/test2/.../bin3.hex
...
and each hex file contains the string list such as
[ 0] A4B232
[ 1] 14B2F2
[ 2] 1472F1
...
I want to diff the each 3 hex files from a, b and c file.
so I started coding as below in Python. so far, I've got successfully save the data into the global variables.
arr_s_line1 = []
arr_s_line2 = []
arr_s_line3 = []
#def dfile():
# with open('a') as f1:
# f_lines1 = f1.read().splitlines();
# for f_line1 in f_lines1:
# with open(f_line1) as f2:
# s_line1 = f2.read().splitlines()
# for s_line1 in s_lines1:
# arr_s_line1.append(s_line1)
def prtf():
with open ('a') as fprtfa:
linesa = fprtfa.read().splitlines()
with open ('b') as fprtfb:
linesb = fprtfb.read().splitlines()
with open ('c') as fprtfc:
linesc = fprtfc.read().splitlines()
for linea in linesa :
with open(linea) as fa:
s_linesa = fa.read().splitlines()
for s_linea in s_linesa:
arr_s_line1.append(s_linea)
for lineb in linesb :
with open(lineb) as fb:
s_linesb = fb.read().splitlines()
for s_lineb in s_linesb:
arr_s_line2.append(s_lineb)
for linec in linesc :
with open(linec) as fc:
s_linesc = fc.read().splitlines()
for s_linec in s_linesc:
arr_s_line3.append(s_linec)
if __name__== "__main__":
prtf()
and I want to diff among with arr_s_line1[i] & arr_s_line2[i] &arr_s_line3[i]) to know whether there is mismatch or not. and If there is a happening of mismatch then I want to print the name of mismatched file and where line. How to diff from multiple read file in python?
Especially, the problem of this way is when I run a large amount of files, many memories required to run. so I want to avoid it.
I am writing a simple Python script to tell me file size for a set of documents which I am importing from a CSV. I verified that none of the entries are over 100 characters, so this error "ValueError: scandir: path too long for Windows" does not make sense to me.
Here is my code:
# determine size of a given folder in MBytes
import os, subprocess, json, csv, platform
# Function to check if a Drive Letter exists
def hasdrive(letter):
return "Windows" in platform.system() and os.system("vol %s: 2>nul>nul" % (letter)) == 0
# Define Drive to check for
letter = 'S'
# Check if Drive doesnt exist, if not then map drive
if not hasdrive(letter):
subprocess.call(r'net use s: /del /Y', shell=True)
subprocess.call(r'net use s: \\path_to_files', shell=True)
list1 = []
# Import spreadsheet to calculate size
with open('c:\Temp\files_to_delete_subset.csv') as f:
reader = csv.reader(f, delimiter=':', quoting=csv.QUOTE_NONE)
for row in reader:
list1.extend(row)
# Define variables
folder = "S:"
folder_size = 0
# Exporting outcome
for list1 in list1:
folder = folder + str(list1)
for root, dirs, files in os.walk(folder):
for name in files:
folder_size += os.path.getsize(os.path.join(root, name))
print(folder)
# print(os.path.join(root, name) + " " + chr(os.path.getsize(os.path.join(root, name))))
print(folder_size)
From my understanding the max path size in Windows is 260 characters, so 1 driver letter + 100 character path should NOT exceed the Windows max.
Here is an example of a path: '/Document/8669/CORRESP/1722165.doc'
The folder string you're trying to walk is growing forever. Simplifying the code to the problem area:
folder = "S:"
# Exporting outcome
for list1 in list1:
folder = folder + str(list1)
You never set folder otherwise, so it starts out as S:<firstpath>, then on the next loop it's S:<firstpath><secondpath>, then S:<firstpath><secondpath><thirdpath>, etc. Simple fix: Separate drive from folder:
drive = "S:"
# Exporting outcome
for path in list1:
folder = drive + path
Now folder is constructed from scratch on each loop, throwing away the previous path, rather than concatenating them.
I also gave the iteration value a useful name (and removed the str call, because the values should all be str already).
I am attempting to take a file name such as 'OP 40 856101.txt' from a directory, remove the .txt, set each single word to a specific variable, then reorder the filename based on a required order such as '856101 OP 040'. Below is my code:
import os
dir = 'C:/Users/brian/Documents/Moeller'
orig = os.listdir(dir) #original names of the files in the folder
for orig_name in orig: #This loop splits each file name into a list of stings containing each word
f = os.path.splitext(orig_name)[0]
sep = f.split() #Separation is done by a space
for t in sep: #Loops across each list of strings into an if statement that saves each part to a specific variable
#print(t)
if t.isalpha() and len(t) == 3:
wc = t
elif len(t) > 3 and len(t) < 6:
wc = t
elif t == 'OP':
op = t
elif len(t) >= 4:
pnum = t
else:
opnum = t
if len(opnum) == 2:
opnum = '0' + opnum
new_nam = '%s %s %s %s' % (pnum,op,opnum, wc) #This is the variable that contain the text for the new name
print("The orig filename is %r, the new filename is %r" % (orig_name, new_nam))
os.rename(orig_name, new_nam)
However I am getting an error with my last for loop where I attempt to rename each file in the directory.
FileNotFoundError: [WinError 2] The system cannot find the file specified: '150 856101 OP CLEAN.txt' -> '856101 OP 150 CLEAN'
The code runs perfectly until the os.rename() command, if I print out the variable new_nam, it prints out the correct naming order for all of the files in the directory. Seems like it cannot find the original file though to replace the filename to the string in new_nam. I assume it is a directory issue, however I am newer to python and can't seem to figure where to edit my code. Any tips or advice would be greatly appreciated!
Try this (just changed the last line):
os.rename(os.path.join(dir,orig_name), os.path.join(dir,new_nam))
You need to tell Python the actual path of the file to rename - otherwise, it looks only in the directory containing this file.
Incidentally, it's better not to use dir as a variable name, because that's the name of a built-in.
I am trying to make a script which selects every .png file in a folder beginning with the letters "LG". I then want the scipt create a shapefile, replacing the "LG" with "SH", and then i want the script to buffer that shapefile and rename the buffer with the first 2 letters being "SB"!
I keep getting an error 99999 error message at line 37!
( gp.RasterToPolygon_conversion(INPUT_RASTER, Output_polygon_features, "SIMPLIFY", "VALUE") )
Can anyone see why this isnt working? I am very, very new to this and have been staring at this script pulling out my hair for days!!
Here is the script:
# Load required toolboxes...
gp.AddToolbox("C:/Program Files/ArcGIS/ArcToolbox/Toolboxes/Conversion Tools.tbx")
gp.AddToolbox("C:/Program Files/ArcGIS/ArcToolbox/Toolboxes/Analysis Tools.tbx")
# Script arguments...
folder = "D:\\J04-0083\\IMAGEFILES"
for root, dirs, filenames in os.walk(folder): # returms root, dirs, and files
for filename in filenames:
filename_split = os.path.splitext(filename) # filename and extensionname (extension in [1])
filename_zero = filename_split[0]
try:
first_2_letters = filename_zero[0] + filename_zero[1]
except:
first_2_letters = "XX"
if first_2_letters == "LG":
Output_polygon_features = "D:\\J04-0083\\ShapeFiles.gdb\\" + "SH_" + filename + ".shp"
# Process: Raster to Polygon...
INPUT_RASTER = os.path.join(root + "\\" + filename_zero + ".png")
gp.RasterToPolygon_conversion(INPUT_RASTER, Output_polygon_features, "SIMPLIFY", "VALUE")
Distance__value_or_field_ = "5 Meters"
Raster_Buffer_shp = "SB_" + filename + ".shp"
# Process: Buffer...
gp.Buffer_analysis(Output_polygon_features, Raster_Buffer_shp, Distance__value_or_field_, "FULL", "ROUND", "NONE", "")
Is .png the format that this function wants? PNG is a compressed format so I would think that something like this would be expecting an uncompressed format. In fact, since the name of the function is RasterToPolygon_conversion, wouldn't the function be expecting a raster format? The docs say that the input should be an integer raster dataset. In addition, The input raster can have any cell size and may be any valid integer raster dataset. Anyway, I suspect that is the real problem.
The last thing to check, if the file is in the correct format as per above, is if there is a field VALUE in the file.
try using a GRID or TIFF file instead of a PNG.
You can convert the PNG with:
http://webhelp.esri.com/arcgiSDEsktop/9.3/index.cfm?TopicName=raster_to_other_format_(multiple)_(conversion)
and then process it's output into the Raster to Polygon conversion.
You could also check the file path of the INPUT RASTER to make sure it looks correct by:
INPUT_RASTER = os.path.join(root + "\\" + filename_zero + ".png")
print INPUT_RASTER
gp.RasterToPolygon_conversion(INPUT_RASTER, Output_polygon_features, "SIMPLIFY", "VALUE")
There is also a method of building a filepath by:
import os
root + os.sep + filename_zero + '.png'