I'm writing a function that does some operations with a .log file: The program checks if /logs/ansible.log exists before proceeding. If /logs/ansible.log doesn't exist, it should go ahead and create the file / directory structure (both don't exist prior).
try:
if not os.path.exists("/logs/ansible.log"):
# create the /logs/ansible.log file
finally:
# do something
I know I can create the ansible.log file with open('ansible.log', 'w') and create the directory with os.makedirs('/logs/'), however how can I simply create '/logs/ansible.log' at once?
*** Assume program is being executed as root
def createAndOpen(filename, mode):
os.makedirs(os.path.dirname(path), exist_ok=True)
return open(filename, mode)
Now, you can open the file and create the folder at once:
with createAndOpen('/logs/ansible.log', 'a') as f:
f.write('Hello world')
Otherwise, this isn’t possible. The operating system does not give you a single function that does this, so whatever else existed that does this would have to have a similar logic as above. Just less visible to you. But since it simply doesn’t exist, you can just create it yourself.
Related
I'm creating a program to take YouTube videos and convert them into mp3's and store them in a specific folder.
Every time my program is ran, it will make sure the file directory exists as well as the folder and text file inside of it, if not, it will create these. However when running my program, it does in fact create a file in the specified directory, but it will not create the json file (which is where I will store the file path directory so the user doesn't have to input the directory every time).
The program doesn't close, it doesn't throw an error, it just simply does nothing after creating the file. It shows a blinking cursor and just idles. I'm a newbie here so I'm probably missing something very simple but. I can't figure it out, the code I'm supplying below isn't the whole program, just the function that keeps messing up:
def _check_dir_path() -> bool:
if os.path.exists(file_dir):
while True:
song_path = os.path.join(file_dir, "sp_songs")
dir_file = os.path.join(file_dir, "Song File Directory.json")
if os.path.isdir(song_path):
if os.path.isfile(dir_file):
return True
else:
sp_songs = open("sp_songs", "w")
json.dump(path_dir, sp_songs)
sp_songs.close()
continue
else:
os.mkdir(song_path)
continue
I realized I wasn't specifying a directory, I was creating a file in my python project folder but checking for that file in the specified directory which meant it'll never return true:
sp_songs = open("sp_songs", "w")
By replacing 'sp_songs' with 'dir_file' I was specifying the path:
sp_songs = open(dir_file, "w")
I'm writing a function that does some operations with a .log file: The program checks if /logs/ansible.log exists before proceeding. If /logs/ansible.log doesn't exist, it should go ahead and create the file / directory structure (both don't exist prior).
try:
if not os.path.exists("/logs/ansible.log"):
# create the /logs/ansible.log file
finally:
# do something
I know I can create the ansible.log file with open('ansible.log', 'w') and create the directory with os.makedirs('/logs/'), however how can I simply create '/logs/ansible.log' at once?
*** Assume program is being executed as root
def createAndOpen(filename, mode):
os.makedirs(os.path.dirname(path), exist_ok=True)
return open(filename, mode)
Now, you can open the file and create the folder at once:
with createAndOpen('/logs/ansible.log', 'a') as f:
f.write('Hello world')
Otherwise, this isn’t possible. The operating system does not give you a single function that does this, so whatever else existed that does this would have to have a similar logic as above. Just less visible to you. But since it simply doesn’t exist, you can just create it yourself.
How do I get the data from multiple txt files that placed in a specific folder. I started with this could not fix. It gives an error like 'No such file or directory: '.idea' (??)
(Let's say I have an A folder and in that, there are x.txt, y.txt, z.txt and so on. I am trying to get and print the information from all the files x,y,z)
def find_get(folder):
for file in os.listdir(folder):
f = open(file, 'r')
for data in open(file, 'r'):
print data
find_get('filex')
Thanks.
If you just want to print each line:
import glob
import os
def find_get(path):
for f in glob.glob(os.path.join(path,"*.txt")):
with open(os.path.join(path, f)) as data:
for line in data:
print(line)
glob will find only your .txt files in the specified path.
Your error comes from not joining the path to the filename, unless the file was in the same directory you were running the code from python would not be able to find the file without the full path. Another issue is you seem to have a directory .idea which would also give you an error when trying to open it as a file. This also presumes you actually have permissions to read the files in the directory.
If your files were larger I would avoid reading all into memory and/or storing the full content.
First of all make sure you add the folder name to the file name, so you can find the file relative to where the script is executed.
To do so you want to use os.path.join, which as it's name suggests - joins paths. So, using a generator:
def find_get(folder):
for filename in os.listdir(folder):
relative_file_path = os.path.join(folder, filename)
with open(relative_file_path) as f:
# read() gives the entire data from the file
yield f.read()
# this consumes the generator to a list
files_data = list(find_get('filex'))
See what we got in the list that consumed the generator:
print files_data
It may be more convenient to produce tuples which can be used to construct a dict:
def find_get(folder):
for filename in os.listdir(folder):
relative_file_path = os.path.join(folder, filename)
with open(relative_file_path) as f:
# read() gives the entire data from the file
yield (relative_file_path, f.read(), )
# this consumes the generator to a list
files_data = dict(find_get('filex'))
You will now have a mapping from the file's name to it's content.
Also, take a look at the answer by #Padraic Cunningham . He brought up the glob module which is suitable in this case.
The error you're facing is simple: listdir returns filenames, not full pathnames. To turn them into pathnames you can access from your current working directory, you have to join them to the directory path:
for filename in os.listdir(directory):
pathname = os.path.join(directory, filename)
with open(pathname) as f:
# do stuff
So, in your case, there's a file named .idea in the folder directory, but you're trying to open a file named .idea in the current working directory, and there is no such file.
There are at least four other potential problems with your code that you also need to think about and possibly fix after this one:
You don't handle errors. There are many very common reasons you may not be able to open and read a file--it may be a directory, you may not have read access, it may be exclusively locked, it may have been moved since your listdir, etc. And those aren't logic errors in your code or user errors in specifying the wrong directory, they're part of the normal flow of events, so your code should handle them, not just die. Which means you need a try statement.
You don't do anything with the files but print out every line. Basically, this is like running cat folder/* from the shell. Is that what you want? If not, you have to figure out what you want and write the corresponding code.
You open the same file twice in a row, without closing in between. At best this is wasteful, at worst it will mean your code doesn't run on any system where opens are exclusive by default. (Are there such systems? Unless you know the answer to that is "no", you should assume there are.)
You don't close your files. Sure, the garbage collector will get to them eventually--and if you're using CPython and know how it works, you can even prove the maximum number of open file handles that your code can accumulate is fixed and pretty small. But why rely on that? Just use a with statement, or call close.
However, none of those problems are related to your current error. So, while you have to fix them too, don't expect fixing one of them to make the first problem go away.
Full variant:
import os
def find_get(path):
files = {}
for file in os.listdir(path):
if os.path.isfile(os.path.join(path,file)):
with open(os.path.join(path,file), "r") as data:
files[file] = data.read()
return files
print(find_get("filex"))
Output:
{'1.txt': 'dsad', '2.txt': 'fsdfs'}
After the you could generate one file from that content, etc.
Key-thing:
os.listdir return a list of files without full path, so you need to concatenate initial path with fount item to operate.
there could be ideally used dicts :)
os.listdir return files and folders, so you need to check if list item is really file
You should check if the file is actually file and not a folder, since you can't open folders for reading. Also, you can't just open a relative path file, since it is under a folder, so you should get the correct path with os.path.join. Check below:
import os
def find_get(folder):
for file in os.listdir(folder):
if not os.path.isfile(file):
continue # skip other directories
f = open(os.path.join(folder, file), 'r')
for line in f:
print line
I have the following method in Python.
def get_rum_data(file_path, query):
if file_path is not None and query is not None:
command = FETCH_RUM_COMMAND_DATA % (constants.RUM_JAR_PATH,
constants.RUM_SERVER, file_path,
query)
print command
execute_command(command).communicate()
Now inside get_rum_data I need to create the file if it does not exists, if it exists I need to append the data. How to do that in python.
I tried, open(file_path, 'w') , which gave me an exception.
Traceback (most recent call last):
File "utils.py", line 180, in <module>
get_rum_data('/User/rokumar/Desktop/sample.csv', '\'show tables\'')
File "utils.py", line 173, in get_rum_data
open(file_path, 'w')
IOError: [Errno 2] No such file or directory: '/User/rokumar/Desktop/sample.csv'
I though open would create file in write mode.
It shall be as simple as:
fname = "/User/rokumar/Desktop/sample.csv"
with open(fname, "a") as f:
# do here what you want
# it will get closed at this point by context manager
But I would suspect, that you are trying to use not existing directory. Usually, "a" mode creates the file if it can be created.
Make sure, the directory exists.
You can check if all directories in file_path exist prior to trying to write a file.
import os
file_path = '/Users/Foo/Desktop/bar.txt'
print os.path.dirname(file_path)
# /Users/Foo/Desktop
if not os.path.exists(os.path.dirname(file_path)):
os.mkdirs(os.path.dirname(file_path))
# recursively create directories if necessary
with open(file_path, "a") as my_file:
# mode a will either create the file if it does not exist
# or append the content to its end if it exists.
my_file.write(your_text_to_append)
-- Edit: Small and probably unnecessary extension --
expanduser:
In your case, as it turned out that the initial problem was a missing s in the path to the user directory, there is a useful feature for resolving the current users base directory (works for unix, linux and windows): see expanduser from the os.path module. With that you could write your path as path = '~/Desktop/bar.txt' and the tilde (~) will be expanded just like on your shell. (with the additional benefit that if you started your script from another user it would then expand to her home directory.
App config directory:
Since in most cases it is not desirable to write a file to the desktop (*nix systems for instance might not have a desktop installed), there is a great utility function in the click package. If you look at the get_app_dir() function on Github, you can see how they provide expanding to an appropriate app dir and supporting multiple operating systems (the function has no dependencies besides the WIN variable that is defined in the _compat.py module as WIN = sys.platform.startswith('win') and the _posixify() function defined on line 17. Often that's a good starting point for defining an app directory to store certain data in.
With the code open, I can create a new file in the computer. How do i decide which folder it goes? I need to put them in a certain folder when i am creating them. Should I put sth in the brackets?
e.g. open("apple juice. txt", "a")
If you don't specify a path, then the file will be created in the current directory. Where exactly that is depends on how you started the interpreter. For example, when you start Python 3.4 from the Windows Start Menu, then the file will be saved in C:\Python34\.
If you want to specify a certain path, then do so:
f = open(r"C:\Users\David\Python Files\apple juice.txt", "a")
Give the full path:
with open("path_where/to_save/apple_juice.txt", "a") as f:
# do work
with will automatically close your file.
If you are looking for a place for temp files, use the module tempfile.
You can use the function tempfile.gettempdir() to get a path to a folder directory.
You can use tempfile.TemporaryFile() to generate a full path to the place where the temp files are usually stored on your OS.
The method used in either case to generate the temp path is explained here.