I'm trying to rename files using an excel sheet and pandas but I keep getting an IO error. In the 'Filename' column I have the original filename along with what I want it to be in the 'rename' column. I know there are other ways to do this but I don't understand why the below doesn't work.
import os
import pandas as pd
from os.path import join
import shutil
dir = os.path.dirname(__file__)
excelFile = join(dir,'test.xlsx')
output_image_dir = os.path.join(dir,'PREAPPROVAL')
df = pd.read_excel(excelFile, sheetname='rename')
for x,y in zip(df['Filename'].astype('str'),df['rename']):
x = join(output_image_dir,x)
y = join(output_image_dir,y)
shutil.move(x,y)
Traceback (most recent call last):
File "D:\Dropbox\1. Projects\2. Python\2. X1\rename_images.py", line 26, in <module>
shutil.move(x,y)
File "c:\python27\lib\shutil.py", line 302, in move
copy2(src, real_dst)
File "c:\python27\lib\shutil.py", line 130, in copy2
copyfile(src, dst)
File "c:\python27\lib\shutil.py", line 82, in copyfile
with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: 'D:\\Dropbox\\1. Projects\\2. Python\\2. X1\\PREAPPROVAL\\VERT.jpg'
[Finished in 0.3s]
Sorry for the trouble, but I figured it out. There was an extra space in one of the filenames that threw everything off. Thanks.
Related
My two line code for moving a file into the cwd is as follows:
import os, shutil
shutil.move(os.path.abspath('hello_file.txt'),os.getcwd())
I do have the file just one up the cwd but I get this traceback:
Traceback (most recent call last):
File "<ipython-input-333-f012757a9dca>", line 1, in <module>
shutil.move(os.path.abspath('hello_file.txt'),os.getcwd())
File "/Users/deepayanbhadra/anaconda3/lib/python3.6/shutil.py", line 558, in move
copy_function(src, real_dst)
File "/Users/deepayanbhadra/anaconda3/lib/python3.6/shutil.py", line 257, in copy2
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "/Users/deepayanbhadra/anaconda3/lib/python3.6/shutil.py", line 120, in copyfile
with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: '/Users/deepayanbhadra/Desktop/DIY-Python-Projects/Automating Tasks/hello_file.txt'
The file path you have provided is the current directory. You first have to change to the directory up. One of the ways you can do this is
import os
os.chdir(“..”)
That’s the simplest way but will take you quite a number of lines of code to achieve the same
I am having issues with getting basic function of the nptdms module working.
First, I am just trying to open a TDMS file and print the contents of specific channels within specific groups.
Using python 2.7 and the nptdms quick start here
Following this, I will be writing these specific pieces of data into a new TDMS file. Then, my ultimate goal is to be able to take a set of source files, open each, and write (append) to a new file. The source data files contain far more information that is needed, so I am breaking out the specifics into their own file.
The problem I have is that I cannot get past a basic error.
When running this code, I get:
Traceback (most recent call last):
File "PullTDMSdataIntoNewFile.py", line 27, in <module>
tdms_file = TdmsFile(r"C:\\Users\daniel.worts\Desktop\this_is_my_tdms_file.tdms","r")
File "C:\Anaconda2\lib\site-packages\nptdms\tdms.py", line 94, in __init__
self._read_segments(f)
File "C:\Anaconda2\lib\site-packages\nptdms\tdms.py", line 119, in _read_segments
object._initialise_data(memmap_dir=self.memmap_dir)
File "C:\Anaconda2\lib\site-packages\nptdms\tdms.py", line 709, in _initialise_data
mode='w+b', prefix="nptdms_", dir=memmap_dir)
File "C:\Anaconda2\lib\tempfile.py", line 475, in NamedTemporaryFile
(fd, name) = _mkstemp_inner(dir, prefix, suffix, flags)
File "C:\Anaconda2\lib\tempfile.py", line 244, in _mkstemp_inner
fd = _os.open(file, flags, 0600)
OSError: [Errno 2] No such file or directory: 'r\\nptdms_yjfyam'
Here is my code:
from nptdms import TdmsFile
import numpy as np
import pandas as pd
#set Tdms file path
tdms_file = TdmsFile(r"C:\\Users\daniel.worts\Desktop\this_is_my_tdms_file.tdms","r")
# set variable for TDMS groups
group_nameone = '101'
group_nametwo = '752'
# set objects for TDMS channels
channel_dataone = tdms_file.object(group_nameone 'Payload_1')
channel_datatwo = tdms_file.object(group_nametwo, 'Payload_2')
# set data from channels
data_dataone = channel_dataone.data
data_datatwo = channel_datatwo.data
print data_dataone
print data_datatwo
Big thanks to anyone who may have encountered this before and can help point to what I am missing.
Best,
- Dan
edit:
Solved the read data issue by removing the 'r' argument from the file path.
Now I am having another error I can't trace when trying to write.
from nptdms import TdmsFile, TdmsWriter, RootObject, GroupObject, ChannelObject
import numpy as np
import pandas as pd
newfilepath = r"C:\\Users\daniel.worts\Desktop\Mined.tdms"
datetimegroup101_channel_object = ChannelObject('101', DateTime, data_datetimegroup101)
with TdmsWriter(newfilepath) as tdms_writer:
tdms_writer.write_segment([datetimegroup101_channel_object])
Returns error:
Traceback (most recent call last):
File "PullTDMSdataIntoNewFile.py", line 82, in <module>
tdms_writer.write_segment([datetimegroup101_channel_object])
File "C:\Anaconda2\lib\site-packages\nptdms\writer.py", line 68, in write_segment
segment = TdmsSegment(objects)
File "C:\Anaconda2\lib\site-packages\nptdms\writer.py", line 88, in __init__
paths = set(obj.path for obj in objects)
File "C:\Anaconda2\lib\site-packages\nptdms\writer.py", line 88, in <genexpr>
paths = set(obj.path for obj in objects)
File "C:\Anaconda2\lib\site-packages\nptdms\writer.py", line 254, in path
self.channel.replace("'", "''"))
AttributeError: 'TdmsObject' object has no attribute 'replace'
Here's all my code to start with:
import shutil
import glob
import os
for filename in glob.glob(r"C:/Users/Aydan/Desktop/RTHPython/Years/*.txt"):
text_file_name = filename.strip()
dst = (os.path.join(r"C:/Users/Aydan/Desktop/SortedImages", text_file_name))
with open (text_file_name) as my_file:
for filename in my_file:
file_name = filename.strip()
src = (os.path.join('C:/Users/Aydan/Desktop', file_name))
src = src.replace('/', '\\')
shutil.move(src, dst)
I've made this to cycle through each text file in \Years\, and with in each text file, cycle through each line. On each line in the text file is a file directory:
1855.txt contents:
/data01/BL/ER/D11/fmp000005578/BL_ER_D11_fmp000005578_0001_1.txt
/data01/BL/ER/D11/fmp000005578/BL_ER_D11_fmp000005578_0002_1.txt
/data01/BL/ER/D11/fmp000005578/BL_ER_D11_fmp000005578_0003_1.txt
1856.txt contents:
/data01/BL/ER/D11/fmp000005578/BL_ER_D11_fmp000005578_0004_1.txt
/data01/BL/ER/D11/fmp000005578/BL_ER_D11_fmp000005578_0005_1.txt
/data01/BL/ER/D11/fmp000005578/BL_ER_D11_fmp000005578_0006_1.txt
This file directory is used as the source of the file using shutil, and the destination is the name of the text file without .txt, so 1855 and 1856. This means the files that are under 1855 go into the 1855 folder, and the same for 1856.
I'm an continuing to get errors with this code though, but I do not understand why:
Traceback (most recent call last):
File "C:\Users\Aydan\AppData\Local\Programs\Python\Python36-32\lib\shutil.py", line 544, in move
os.rename(src, real_dst)
FileNotFoundError: [WinError 2] The system cannot find the file specified: 'C:/Users/Aydan/Desktop/data01/BL/ER/D11/fmp000005578/BL_ER_D11_fmp000005578_0001_1.txt' -> 'C:\\Users\\Aydan\\Desktop\\RTHPython\\Years\\1855.txt'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<pyshell#17>", line 8, in <module>
shutil.move(src, dst)
File "C:\Users\Aydan\AppData\Local\Programs\Python\Python36-32\lib\shutil.py", line 558, in move
copy_function(src, real_dst)
File "C:\Users\Aydan\AppData\Local\Programs\Python\Python36-32\lib\shutil.py", line 257, in copy2
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "C:\Users\Aydan\AppData\Local\Programs\Python\Python36-32\lib\shutil.py", line 120, in copyfile
with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: 'C:/Users/Aydan/Desktop/data01/BL/ER/D11/fmp000005578/BL_ER_D11_fmp000005578_0001_1.txt'
>>>
Even with it saying No such file or directory:, I know it's there as when I paste in the exact directory it has an error on, the text file opens...
I would greatly appreciate help to fix this and come to an almost finish on this project :D
Thanks!
Aydan.
Update code:
import shutil
import glob
import os
for filename in glob.glob(r"C:/Users/Aydan/Desktop/RTHPython/Years/*.txt"):
text_file_name = filename.strip()
dst = (os.path.join(r"C:/Users/Aydan/Desktop/SortedImages", text_file_name))
with open (text_file_name) as my_file:
for filename in my_file:
file_name = filename.strip()
src = (os.path.join('C:/Users/Aydan/Desktop', file_name))
src = src.replace('/', '\\')
shutil.move(src, dst)
It's new errors:
Traceback (most recent call last):
File "C:\Users\Aydan\AppData\Local\Programs\Python\Python36-32\lib\shutil.py", line 544, in move
os.rename(src, real_dst)
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\data01\\BL\\ER\\D11\\fmp000005578\\BL_ER_D11_fmp000005578_0001_1.txt' -> 'C:/Users/Aydan/Desktop/RTHPython/Years\\1855.txt'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<pyshell#11>", line 9, in <module>
shutil.move(src, dst)
File "C:\Users\Aydan\AppData\Local\Programs\Python\Python36-32\lib\shutil.py", line 558, in move
copy_function(src, real_dst)
File "C:\Users\Aydan\AppData\Local\Programs\Python\Python36-32\lib\shutil.py", line 257, in copy2
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "C:\Users\Aydan\AppData\Local\Programs\Python\Python36-32\lib\shutil.py", line 120, in copyfile
with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\data01\\BL\\ER\\D11\\fmp000005578\\BL_ER_D11_fmp000005578_0001_1.txt'
import shutil
import glob
import os
import ntpath
for filename in glob.glob(r"C:/Users/Aydan/Desktop/RTHPython/Years/*.txt"):
text_file_name = filename.strip()
text_name_folder = ntpath.basename(filename)
dst = os.path.join(r"C:/Users/Aydan/Desktop/SortedImages", text_name_folder.strip(".txt"))
dst = dst.replace('/', '\\')
with open (text_file_name) as my_file:
for filename in my_file:
file_name = filename.strip()
src = (os.path.normpath('C:/Users/Aydan/Desktop' + file_name))
src = src.replace('/', '\\')
shutil.move(src, dst)
It works!
I am using Pandas on Mac, to read and write a CSV file, and the weird thing is when using full path, it has error and when using just a file name, it works. I post my code which works and which not works in my comments below, and also detailed error messages. Anyone have any good ideas?
sourceDf = pd.read_csv(path_to_csv)
sourceDf['nameFull'] = sourceDf['nameFirst'] + ' ' + sourceDf['nameLast']
sourceDf.to_csv('newMaster.csv') # working
sourceDf.to_csv('~/Downloads/newMaster.csv') # not working
Traceback (most recent call last):
File "/Users/foo/PycharmProjects/DataWranglingTest/CSVTest1.py", line 36, in <module>
add_full_name(path_to_csv, path_to_new_csv)
File "/Users/foo/PycharmProjects/DataWranglingTest/CSVTest1.py", line 28, in add_full_name
sourceDf.to_csv('~/Downloads/newMaster.csv')
File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/frame.py", line 1189, in to_csv
formatter.save()
File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/format.py", line 1442, in save
encoding=self.encoding)
File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/common.py", line 2831, in _get_handle
f = open(path, mode)
IOError: [Errno 2] No such file or directory: '~/Downloads/newMaster.csv'
Tried to use prefix r, but not working,
path_to_csv = r'~/Downloads/Master.csv'
path_to_new_csv = r'~/Downloads/Master_new.csv'
File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/frame.py", line 1189, in to_csv
formatter.save()
File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/format.py", line 1442, in save
encoding=self.encoding)
File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/common.py", line 2831, in _get_handle
f = open(path, mode)
IOError: [Errno 2] No such file or directory: '~/Downloads/Master_new.csv'
thanks in advance,
Lin
Try using os.path.join().
import os
(...)
output_filename = 'newMaster.csv'
output_path = os.path.join('Downloads', output_filename)
(...)
sourceDf.to_csv(output_path)
Use the same methodology to point pandas.read_csv() in the right direction.
You didn't specify python version.
On 3.4 you can use pathlib, otherwise use os.path.join() or quoting:
sourceDf.to_csv(r'~/Downloads/newMaster.csv')
Notice the r.
The problem is that /n is newline, which is not allowed in a path.
I'm beginner in python and I have a problem with this script:
import errno import shutil import os
def copystuff(src, dest):
if os.path.isfile(src):
dest_dirname = os.path.dirname(src)
if not os.path.isdir(dest_dirname):
os.makedirs(dest_dirname)
shutil.copy2(src, dest)
else:
shutil.copytree(src, dest)
copystuff('C:\\Downloads\\index.html', 'J:\\include\\')
Where J it's FlashDriveUSB, and I'm using Python 2.7.
When I'm launching this I got something like that:
C:\Python27>python copy_file.py
Traceback (most recent call last):
File "copy_file.py", line 24, in <module>
copystuff('C:\\Downloads\index.html', 'D:\\include\\')
File "copy_file.py", line 20, in copystuff
shutil.copy2(src, dest)
File "C:\Python27\lib\shutil.py", line 127, in copy2
copyfile(src, dst)
File "C:\Python27\lib\shutil.py", line 82, in copyfile
with open(dst, 'wb') as fdst:
IOError: [Errno 22] invalid mode ('wb') or filename: 'D:\\include\\'
Please help me.
I resolved my problem. I wanted copy file to folder witch doesn't exist so i added to code a few lines checking if inputted path exists and if not path is created.