I have been using the copy2 function to copy files and it has been working fine on the Linux system with both the source and destination directories in Linux. Recently I tried to use the Windows File share as the backend and was trying to copy files to my linux system. The code and the error displayed is as follows:
for afile in self.task_files:
copy2(os.path.join(self.uploads_directory, afile), self.files_directory)
count += 1`
Output:
Traceback (most recent call last):
File "/opt/backend/worker.py", line 43, in <module>
task.initialize()
File "/opt/backend/services/task.py", line 58, in initialize
copy2(os.path.join(self.uploads_directory, afile), self.files_directory)
File "/usr/lib/python3.5/shutil.py", line 252, in copy2
copystat(src, dst, follow_symlinks=follow_symlinks)
File "/usr/lib/python3.5/shutil.py", line 219, in copystat
_copyxattr(src, dst, follow_symlinks=follow)
File "/usr/lib/python3.5/shutil.py", line 151, in _copyxattr
names = os.listxattr(src, follow_symlinks=follow_symlinks)
OSError: [Errno 38] Function not implemented: '/data/uas/uploads/DJI_0049_vnbh0.JPG'
Is the copy2 function unable to transfer the files because of the OS or is it something else? Also is there any other function in python which can be used to copy files along with its metadata?
Related
My two line code for moving a file into the cwd is as follows:
import os, shutil
shutil.move(os.path.abspath('hello_file.txt'),os.getcwd())
I do have the file just one up the cwd but I get this traceback:
Traceback (most recent call last):
File "<ipython-input-333-f012757a9dca>", line 1, in <module>
shutil.move(os.path.abspath('hello_file.txt'),os.getcwd())
File "/Users/deepayanbhadra/anaconda3/lib/python3.6/shutil.py", line 558, in move
copy_function(src, real_dst)
File "/Users/deepayanbhadra/anaconda3/lib/python3.6/shutil.py", line 257, in copy2
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "/Users/deepayanbhadra/anaconda3/lib/python3.6/shutil.py", line 120, in copyfile
with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: '/Users/deepayanbhadra/Desktop/DIY-Python-Projects/Automating Tasks/hello_file.txt'
The file path you have provided is the current directory. You first have to change to the directory up. One of the ways you can do this is
import os
os.chdir(“..”)
That’s the simplest way but will take you quite a number of lines of code to achieve the same
I am trying to serialize/deserialize spaCy documents (setup is Windows 7, Anaconda) and am getting errors. I haven't been able to find any explanations. Here is a snippet of code and the error it generates:
import spacy
nlp = spacy.load('en')
text = 'This is a test.'
doc = nlp(text)
fout = 'test.spacy' # <-- according to the API for Doc.to_disk(), this needs to be a directory (but for me, spaCy writes a file)
doc.to_disk(fout)
doc.from_disk(fout)
Traceback (most recent call last):
File "<ipython-input-7-aa22bf1b9689>", line 1, in <module>
doc.from_disk(fout)
File "doc.pyx", line 763, in spacy.tokens.doc.Doc.from_disk
File "doc.pyx", line 806, in spacy.tokens.doc.Doc.from_bytes
ValueError: [E033] Cannot load into non-empty Doc of length 5.
I have also tried creating a new Doc object and loading from that, as shown in the example ("Example: Saving and loading a document") in the spaCy docs, which results in a different error:
from spacy.tokens import Doc
from spacy.vocab import Vocab
new_doc = Doc(Vocab()).from_disk(fout)
Traceback (most recent call last):
File "<ipython-input-16-4d99a1199f43>", line 1, in <module>
Doc(Vocab()).from_disk(fout)
File "doc.pyx", line 763, in spacy.tokens.doc.Doc.from_disk
File "doc.pyx", line 838, in spacy.tokens.doc.Doc.from_bytes
File "stringsource", line 646, in View.MemoryView.memoryview_cwrapper
File "stringsource", line 347, in View.MemoryView.memoryview.__cinit__
ValueError: buffer source array is read-only
EDIT:
As pointed out in the replies, the path provided should be a directory. However, the first code snippet creates a file. Changing this to a non-existing directory path doesn't help as spaCy still creates a file. Attempting to write to an existing directory causes an error too:
fout = 'data'
doc.to_disk(fout) Traceback (most recent call last):
File "<ipython-input-8-6c30638f4750>", line 1, in <module>
doc.to_disk(fout)
File "doc.pyx", line 749, in spacy.tokens.doc.Doc.to_disk
File "C:\Users\Username\AppData\Local\Continuum\anaconda3\lib\pathlib.py", line 1161, in open
opener=self._opener)
File "C:\Users\Username\AppData\Local\Continuum\anaconda3\lib\pathlib.py", line 1015, in _opener
return self._accessor.open(self, flags, mode)
File "C:\Users\Username\AppData\Local\Continuum\anaconda3\lib\pathlib.py", line 387, in wrapped
return strfunc(str(pathobj), *args)
PermissionError: [Errno 13] Permission denied: 'data'
Python has no problem writing at this location via standard file operations (open/read/write).
Trying with a Path object yields the same results:
from pathlib import Path
import os
fout = Path(os.path.join(os.getcwd(), 'data'))
doc.to_disk(fout)
Traceback (most recent call last):
File "<ipython-input-17-6c30638f4750>", line 1, in <module>
doc.to_disk(fout)
File "doc.pyx", line 749, in spacy.tokens.doc.Doc.to_disk
File "C:\Users\Username\AppData\Local\Continuum\anaconda3\lib\pathlib.py", line 1161, in open
opener=self._opener)
File "C:\Users\Username\AppData\Local\Continuum\anaconda3\lib\pathlib.py", line 1015, in _opener
return self._accessor.open(self, flags, mode)
File "C:\Users\Username\AppData\Local\Continuum\anaconda3\lib\pathlib.py", line 387, in wrapped
return strfunc(str(pathobj), *args)
PermissionError: [Errno 13] Permission denied: 'C:\\Users\\Username\\workspace\\data'
Any ideas why this might be happening?
doc.to_disk(fout)
must be
a path to a directory, which will be created if it doesn't exist.
Paths may be either strings or Path-like objects.
as the documentation for spaCy states in https://spacy.io/api/doc
Try changing fout to a directory, it might do the trick.
EDIT:
Examples from the spacy documentation:
for doc.to_disk:
doc.to_disk('/path/to/doc')
and for doc.from_disk:
from spacy.tokens import Doc
from spacy.vocab import Vocab
doc = Doc(Vocab()).from_disk('/path/to/doc')
I'm attempting to download all the files in an SFTP directory to a local folder using the pysftp library. My code looks like this:
import pysftp
sftp = pysftp.Connection('server', username = 'name', password = 'password')
sftp.get_d('Daily_Reports', '/home/jchrysostom/Documents/SupplyChain/Daily_Reports/')
Daily_Reports is a folder that exists on the SFTP server - I have verified this. I have also verified that /home/jchrysostom/Documents/SupplyChain/Daily_Reports/ exists. I can cd to it in terminal with no problems.
However, when I run this python script, I get the following error: IOError: [Errno 2] File not found.
Any ideas what may be causing this?
UPDATE: A little investigation shows that the files actually have downloaded. In fact, all of them have downloaded just fine. However, I'm unable to run the rest of the script, because it's erroring out for some reason. Is this just a bug in the library?
UPDATE 2 - Full Traceback, as requested:
Traceback (most recent call last):
File "supplychain.py", line 20, in <module>
sftp.get_d('Daily_Reports','/home/jchrysostom/Documents/SupplyChain/Daily_Reports/')
File "/usr/local/lib/python2.7/dist-packages/pysftp.py", line 255, in get_d
preserve_mtime=preserve_mtime)
File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
self.gen.next()
File "/usr/local/lib/python2.7/dist-packages/pysftp.py", line 497, in cd
self.cwd(original_path)
File "/usr/local/lib/python2.7/dist-packages/pysftp.py", line 510, in chdir
self._sftp.chdir(remotepath)
File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 580, in chdir
if not stat.S_ISDIR(self.stat(path).st_mode):
File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 413, in stat
t, msg = self._request(CMD_STAT, path)
File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 729, in _request
return self._read_response(num)
File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 776, in _read_response
self._convert_status(msg)
File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 802, in _convert_status
raise IOError(errno.ENOENT, text)
IOError: [Errno 2] File not found
As best as I can tell, this is a bug in pysftp. The files are being copied successfully, but (at least according to the traceback here) the library is blowing up when it tries to change back to the original remote working directory on the FTP server.
Workaround is to iterate over the files in the directory and get() each individually...
for filename in sftp.listdir('Daily_Reports'):
sftp.get('Daily_Reports/' + filename, localpath = '/home/jchrysostom/Documents/SupplyChain/Daily_Reports/' + filename)
I'm beginner in python and I have a problem with this script:
import errno import shutil import os
def copystuff(src, dest):
if os.path.isfile(src):
dest_dirname = os.path.dirname(src)
if not os.path.isdir(dest_dirname):
os.makedirs(dest_dirname)
shutil.copy2(src, dest)
else:
shutil.copytree(src, dest)
copystuff('C:\\Downloads\\index.html', 'J:\\include\\')
Where J it's FlashDriveUSB, and I'm using Python 2.7.
When I'm launching this I got something like that:
C:\Python27>python copy_file.py
Traceback (most recent call last):
File "copy_file.py", line 24, in <module>
copystuff('C:\\Downloads\index.html', 'D:\\include\\')
File "copy_file.py", line 20, in copystuff
shutil.copy2(src, dest)
File "C:\Python27\lib\shutil.py", line 127, in copy2
copyfile(src, dst)
File "C:\Python27\lib\shutil.py", line 82, in copyfile
with open(dst, 'wb') as fdst:
IOError: [Errno 22] invalid mode ('wb') or filename: 'D:\\include\\'
Please help me.
I resolved my problem. I wanted copy file to folder witch doesn't exist so i added to code a few lines checking if inputted path exists and if not path is created.
I'm new to Python and I writing a script that uses the pysftp module. There is one function in the pysftp module that I'm having trouble with -- it's the rename function.
Here is the call
srv = pysftp.Connection(host = 'xxx.com', username = 'xxx', password = 'xxx')
y = srv.rename('ftptest.txt', 'renamedfile.txt')
The error I'm getting is
Traceback (most recent call last):
File "./ftp.py", line 49, in <module>
y = srv.rename('ftptest.txt', 'renamedfile.txt')
File "/usr/local/lib/python2.7/dist-packages/pysftp.py", line 740, in rename
self._sftp.rename(remote_src, remote_dest)
File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 365, in rename
self._request(CMD_RENAME, oldpath, newpath)
File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 729, in _request
return self._read_response(num)
File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 776, in _read_response
self._convert_status(msg)
File "/usr/local/lib/python2.7/dist-packages/paramiko/sftp_client.py", line 804, in _convert_status
raise IOError(errno.EACCES, text)
IOError: [Errno 13] Permission denied
I have verified that the file exists and it has the 777 for permissions.
With SFTP protocol (which lacks concept of a current working directory), using an absolute paths is recommended. With a relative path, the result is server implementation specific. So you cannot be sure what files you are actually trying to rename.
On *nix systems, to rename a file, you need write permissions to a containing folder. The permissions to the file you are renaming do not matter.