I am sure I am missing this, as it has to be easy, but I have looked in Google, The Docs, and SO and I can not find a simple way to copy a bunch of directories into another directory that already exists from Python 3?
All of the answers I find recommend using shutil.copytree but as I said, I need to copy a bunch of folders, and all their contents, into an existing folder (and keep any folders or that already existed in the destination)
Is there a way to do this on windows?
I would look into using the os module built into python. Specifically os.walk which returns all the files and subdirectories within the directory you point it to. I will not just tell you exactly how to do it, however here is some sample code that i use to backup my files. I use the os and zipfile modules.
import zipfile
import time
import os
def main():
date = time.strftime("%m.%d.%Y")
zf = zipfile.ZipFile('/media/morpheous/PORTEUS/' + date, 'w')
for root, j, files in os.walk('/home/morpheous/Documents'):
for i in files:
zf.write(os.path.join(root, i))
zf.close()
Hope this helps. Also it should be pointed out that i do this in linux, however with a few changes to the file paths it should work the same
Related
I have tried to use shutil module in python. The shutil.copytree fails to copy folders and brings FileExistsError. I have tried it using different files but it still fails.Please check
import shutil
from pathlib import Path
p = Path(r"C:\Users\rapid\OneDrive\Desktop")
shutil.copytree(p/"folder_1", p/"folder_2")
Please check for the error image
I tried the code myself and works fine the first time.
After the destination folder (in your case folder_2) already created shutil can't create it again and it fails. So you can either delete the destination folder manually or using shutil.rmtree.
Good Luck!
All credit to Timus. He provided the solution on the comment section. Just used
shutil.copytree(p/"folder_1", p/"folder_2", dirs_exist_ok=True)
It works fine. Thanks Timux
I have a python file, converted from a Jupiter Notebook, and there is a subfolder called 'datasets' inside this file folder. When I'm trying to open a file that is inside that 'datasets' folder, with this code:
import pandas as pd
# Load the CSV data into DataFrames
super_bowls = pd.read_csv('/datasets/super_bowls.csv')
It says that there is no such file or folder. Then I add this line
os.getcwd()
And the output is the top-level folder of the project, and not the subfolder when is this python file. And I think maybe that's the reason why it's not working.
So, how can I open that csv file with relative paths? I don't want to use absolute path because this code is going to be used in another computers.
Why os.getcwd() is not getting the actual folder path?
My observation, the dot (.) notation to move to the parent directory sometimes does not work depending on the operating system. What I generally do to make it os agnostic is this:
import pandas as pd
import os
__location__ = os.path.realpath(os.path.join(os.getcwd(), os.path.dirname(__file__)))
super_bowls = pd.read_csv(__location__ + '/datasets/super_bowls.csv')
This works on my windows and ubantu machine equally well.
I am not sure if there are other and better ways to achieve this. Would like to hear back if there are.
(edited)
Per your comment below, the current working directory is
/Users/ivanparra/AprendizajePython/
while the file is in
/Users/ivanparra/AprendizajePython/Jupyter/datasets/super_bowls.csv
For that reason, going to the datasets subfolder of the current working directory (CWD) takes you to /Users/ivanparra/AprendizajePython/datasets which either doesn't exist or doesn't contain the file you're looking for.
You can do one of two things:
(1) Use an absolute path to the file, as in
super_bowls = pd.read_csv("/Users/ivanparra/AprendizajePython/Jupyter/datasets/super_bowls.csv")
(2) use the right relative path, as in
super_bowls = pd.read_csv("./Jupyter/datasets/super_bowls.csv")
There's also (3) - use os.path.join to contact the CWD to the relative path - it's basically the same as (2).
(you can also use
The answer really lies in the response by user2357112:
os.getcwd() is working fine. The problem is in your expectations. The current working directory is the directory where Python is running, not the directory of any particular source file. – user2357112 supports Monica May 22 at 6:03
The solution is:
data_dir = os.path.dirname(__file__)
Try this code
super_bowls = pd.read_csv( os.getcwd() + '/datasets/super_bowls.csv')
I noticed this problem a few years ago. I think it's a matter of design style. The problem is that: your workspace folder is just a folder, not a project folder. Most of the time, your relative reference is based on the current file.
VSCode actually supports the dynamic setting of cwd, but that's not the default. If your work folder is not a rigorous and professional project, I recommend you adding the following settings to launch.json. This is the simplest answer you need.
"cwd": "${fileDirname}"
Thanks to everyone that tried to help me. Thanks to the Roy2012 response, I got a code that works for me.
import pandas as pd
import os
currentPath = os.path.dirname(__file__)
# Load the CSV data into DataFrames
super_bowls = pd.read_csv(currentPath + '/datasets/super_bowls.csv')
The os.path.dirname gives me the path of the current file, and let me work with relative paths.
'/Users/ivanparra/AprendizajePython/Jupyter'
and with that it works like a charm!!
P.S.: As a side note, the behavior of os.getcwd() is quite different in a Jupyter Notebook than a python file. Inside the notebook, that function gives the current file path, but in a python file, gives the top folder path.
I have a Windows folder structure and files, something like below
c:\sourcefolder\file1.txt
c:\sourcefolder\subfolder1\file2.txt
c:\sourcefolder\subfolder2\file3.txt
I want to copy all files to a destination folder like below
c:\destinationfolder\file1.txt
c:\destinationfolder\file2.txt
c:\destinationfolder\file3.txt
I tried using glob.glob() to get list of files in the source tree structure but am not able copy them to destination. I tried shutils.copytree() but it copies the tree structure also and I want the destinationfolder to be flat. I tried os.walk() but it also does not work for me. Or I don't know how to set the parameters for the above functions to make them work.
Any help would be highly appreciated.
Thanks
try looping through the list you got from glob.glob and use shutil.copy()
syntax for shutil is shutil.copy(src,dest)
you have to make a destination folder and destination path like (foldername/filename.txt)
import os
import shutil
for f in list(glob.glob()):
destpath = os.path.join('c:\\dest',f.split(\\)[-1])
shutil.copy(f,destpath)
There are many ways, to search a dir for containing a string, that's not really my question. But is there something built in for Python Kivy, that allows automatically searching for files (*.mp3) in a directory with subdirs, or do I have to create one on my own?
If I have to do so, how do I get all the files and subdirs in a directory?
Thank you :)
Finally I decided coding the needed function on my own:
import os
def listfiles(path):
files = []
for base, directory, filename in os.walk(path):
for i in range(len(filename)):
files.append(base+"/"+filename[i])
return files
print(listfiles("/path/path/"))
Checking files for extensions should be easy enaugh :)
Unfortunately the process might take very long for bigger directories, so I'm still looking for a different solution.
Check out Kivy kivy.uix.filechooser FileChooserController and its method files
The list of files in the directory specified by path after applying the filters.
files is a read-only ListProperty.
#edit
Here's what I also found in Kivy docs and this one seems even nicer:
from kivy.uix.filechooser import FileSystemLocal
file_system = FileSystemLocal()
file_system.listdir('/path/to/dir') # this returns a list of files in dir
Instead of FileSystemLocal you can also use FileSystemAbstract if you are not going to browse only local files.
files = [] # list with files in directory
suff = ('.mp3', '.wav')
for i in files:
if i.endswith(suff):
print files
Suffixes needs to be tuples for this to work.
I'm not sure even where to start.
I have a list of output files from a program, lets call them foo. They are numbered outputs like foo_1.out
I'd like to make a directory for each file, move the file to its directory, run a bash script within that directory, take the output from each script, copy it to the root directory as a concatenated single file.
I understand that this is not a forum for "hey, do my work for me", I'm honestly trying to learn. Any suggestions on where to look are sincerely appreciated!
Thanks!
You should probably look up the documentation for the python modules os - specifically os.path and a couple of others - and subprocess which can be found here and here respectively.
Without wanting to do it all for you as you stated - you'll be wanting to do something like:
for f in filelist:
[pth, ext] = os.path.splitext(f)
os.mkdir(pth)
out = subprocess.Popen(SCRIPTNAME, stdout=...)
# and so on...
To get a list of all files in a directory or make folders, check out the os module. Specifically, try os.listdir and os.mkdir
To copy files, you could either manually open each file, copy the contents to a string, and rewrite it to a different file. Alternatively, look at the shutil module
To run bash scripts, use the subprocess library.
All three of those should be a part of python's standard library.