I want to create a tool called unifile for saving and opening files
like this unifile.open.yaml("file.yaml").
This is my structure:
unifile
|
├-open
| └--__init__.py
|
└-save
└--__init__.py
Code that call my module:
import unifile
a = unifile.open.yaml("file.yaml")
open/init.py
import yaml
class open():
def yml(self, file_path):
try:
with open(file_path, "r", encoding="utf-8") as yaml_conf:
yaml_file = yaml.safe_load(yaml_conf)
return yaml_file
except OSError:
print("Can't load yaml")
1 error if I import unifile always say:
module unifile has no atribute open
2 error in __init__.py I can't open file
[pylint] Context manager 'open' doesn't implement enter and exit. [not-context-manager]
here adding solution to ur problem, make your project structure like this.
add unifile/__init__.py file in the unifile itself not in other modules.
then unifile/open/_open.py file content
import yaml
class Open():
def __init__(self):
pass
def yml(self, file_path):
try:
with open(file_path, "r", encoding="utf-8") as yaml_conf:
yaml_file = yaml.safe_load(yaml_conf)
return yaml_file
except OSError:
print("Can't load yaml")
content of the unifile/__init__.py file
from .open._open import Open
in terminal run the program like this
Also, It is better to create a object element first then proceed ahead.
Two issues, two answers.
First, you should add an init file in unifile. With this, Python will understand that unifile is a package with a sub package.
Second, open is a built-in function and you overwrite it by calling your class open. Change your class name and it should work.
You are getting this error because unifile is not a package. There isn't any init.py file at the top level same as open and save. You also cannot call open.yml directly, because open is a class in package open, so either you will have to import open from open, create its instance and then call iml on that instance.
from open import open
a = open().yml('file.yml')
You are getting this error, because you are trying to override an existing keyword in Python open which you should strictly prohibit doing. So you should name your class anything except a reserved keyword.
Related
I am attempting to create and write to a temporary file on Windows OS using Python. I have used the Python module tempfile to create a temporary file.
But when I go to write that temporary file I get an error Permission Denied. Am I not allowed to write to temporary files?! Am I doing something wrong? If I want to create and write to a temporary file how should should I do it in Python? I want to create a temporary file in the temp directory for security purposes and not locally (in the dir the .exe is executing).
IOError: [Errno 13] Permission denied: 'c:\\users\\blah~1\\appdata\\local\\temp\\tmpiwz8qw'
temp = tempfile.NamedTemporaryFile().name
f = open(temp, 'w') # error occurs on this line
NamedTemporaryFile actually creates and opens the file for you, there's no need for you to open it again for writing.
In fact, the Python docs state:
Whether the name can be used to open the file a second time, while the named temporary file is still open, varies across platforms (it can be so used on Unix; it cannot on Windows NT or later).
That's why you're getting your permission error. What you're probably after is something like:
f = tempfile.NamedTemporaryFile(mode='w') # open file
temp = f.name # get name (if needed)
Use the delete parameter as below:
tmpf = NamedTemporaryFile(delete=False)
But then you need to manually delete the temporary file once you are done with it.
tmpf.close()
os.unlink(tmpf.name)
Reference for bug: https://github.com/bravoserver/bravo/issues/111
regards,
Vidyesh
Consider using os.path.join(tempfile.gettempdir(), os.urandom(24).hex()) instead. It's reliable, cross-platform, and the only caveat is that it doesn't work on FAT partitions.
NamedTemporaryFile has a number of issues, not the least of which is that it can fail to create files because of a permission error, fail to detect the permission error, and then loop millions of times, hanging your program and your filesystem.
The following custom implementation of named temporary file is expanded on the original answer by Erik Aronesty:
import os
import tempfile
class CustomNamedTemporaryFile:
"""
This custom implementation is needed because of the following limitation of tempfile.NamedTemporaryFile:
> Whether the name can be used to open the file a second time, while the named temporary file is still open,
> varies across platforms (it can be so used on Unix; it cannot on Windows NT or later).
"""
def __init__(self, mode='wb', delete=True):
self._mode = mode
self._delete = delete
def __enter__(self):
# Generate a random temporary file name
file_name = os.path.join(tempfile.gettempdir(), os.urandom(24).hex())
# Ensure the file is created
open(file_name, "x").close()
# Open the file in the given mode
self._tempFile = open(file_name, self._mode)
return self._tempFile
def __exit__(self, exc_type, exc_val, exc_tb):
self._tempFile.close()
if self._delete:
os.remove(self._tempFile.name)
This issue might be more complex than many of you think. Anyway this was my solution:
Make use of atexit module
def delete_files(files):
for file in files:
file.close()
os.unlink(file.name)
Make NamedTemporaryFile delete=False
temp_files = []
result_file = NamedTemporaryFile(dir=tmp_path(), suffix=".xlsx", delete=False)
self.temp_files.append(result_file)
Register delete_files as a clean up function
atexit.register(delete_files, temp_files)
tempfile.NamedTemporaryFile() :
It creates and opens a temporary file for you.
f = open(temp, 'w') :
You are again going to open the file which is already open and that's why you are getting Permission Denied error.
If you really wants to open the file again then you first need to close it which will look something like this-
temp= tempfile.NamedTemporaryFile()
temp.close()
f = open(temp.name, 'w')
Permission was denied because the file is Open during line 2 of your code.
close it with f.close() first then you can start writing on your tempfile
I am working through an online python course and have an issue that I am having trouble working through. I have a executable directory with the following layout
reader/
|--__main__.py
|--reader
|--__init__.py
|--reader.py
|--compressed
|--gzipped.py
|--bzipped.py
|--__init__.py
When I do not have these modules in the top directory, I can import just fine and use all of the functionality. When I put them in the top level directory and run the executable directory from the command line with
python reader test.gz
I am getting the following error
AttributeError: module 'reader' has not attribute 'Reader'
The code for main.py is
import sys
import reader
r = reader.Reader(sys.argv[1])
try:
print(r.read())
finally:
r.close()
The code for reader.py is
import os
from reader.compressed import gzipped, bzipped
extension_map = {
'.bz2': bzipped.opener,
'.gz': gzipped.opener,
}
class Reader:
def __init__(self, filename):
extension = os.path.splitext(filename)[1]
opener = extension_map.get(extension, open)
self.f = opener(filename, 'rt')
def close(self):
self.f.close()
def read(self):
return self.f.read()
I can provide the rest of the files if needed. I am using the current distribution of Anaconda. Any help or explanations would be appreciated.
It seems to me that because of your __init__.py file the folder reader is seen as a python module. Thereore your main.py code tries to find the class Reader at the folder level, but you need to look for Reader within reader.py. Can you try changing
r = reader.Reader(sys.argv[1])
to
r = reader.reader.Reader(sys.argv[1])
Also, you have a lot of modules/files/modules called reader, e.g. the class Reader is in reader/reader/reader.py. This I would try to avoid as it can lead to confusion.
When you have multiple modules/modules that are named identically you can often get the namespace into a recursive loop. Also, the Reader class is capitalized.
Like mentioned above, try python reader.reader test.gz
While you are troubleshooting try the tab-complete feature to see if your modules are being loaded properly.
i've created a file with the name enviroment.py which contain a Profile class however when i in another file try to import the file like import testProject.enviroment import Profile i get below error
yaml_helper.py", line 11, in read_yaml
with open(file_path, 'r') as f:
IOError: [Errno 2] No such file or directory:
however i'm not sure why it gives this error since the Profile hasnt even been initialized. all i've done is imported it?
enviroment.py
class Profile(object):
def __init__(self, profile):
self.profile = yaml_helper.read_yaml(project.default_profiles_dir, profile)
self.run_enviroment()
def get(self, key):
return self.profile.get(key)
def run_enviroment(self):
return common.validate(connectors.exasol_credentials, self.profile)
That's not an import error. That's an error when the program is trying to open a file. The problem is where you call yaml_helper.read_yaml.
Are you passing in the correct parameters? for example, in that call, you are passing in project.default_profies_dir What is project? I don't see it defined. That could be your problem.
I have the following structure:
run.py
app/hdfs_lib_test.py
app/src/HDFSFileReader.py
This is a Flask app.
The HSFDFileReader.py contains:
class HDFSFileReader:
"""This class represents a reader for accessing files in a HDFS."""
def __init__(self):
pass
#classmethod
def read(cls, file_path):
lines = []
try:
client = Config().get_client('dev')
with client.read('Serien_de', encoding='utf-8', delimiter='\n') as reader:
for line in reader:
lines.append(line)
except:
print("ERROR: Could not read from HDFS.")
raise
return lines
When I run ./run.py I get the ImportError: No module named 'hdfs'. However the library is installed and I can call python hdfs_lib_test.py, which contains the following:
from hdfs import Config
try:
client = Config().get_client('dev')
with client.read('Serien_de',encoding='utf-8',delimiter='\n') as reader:
for line in reader:
print(line)
except:
raise
So to me it seems, that because HDFSFileReader is part of a submodule and hdfs_lib_test.py isn't. The former doesn't work, but the later does. But how do I fix the import problem?
I can't be a circular import issue, I searched the whole project for imports of hdfs and the hdfs_lib_test.py is not used by the actual project.
I want to use tempfile.NamedTemporaryFile() to write some contents into it and then open that file. I have written following code:
tf = tempfile.NamedTemporaryFile()
tfName = tf.name
tf.seek(0)
tf.write(contents)
tf.flush()
but I am unable to open this file and see its contents in Notepad or similar application. Is there any way to achieve this? Why can't I do something like:
os.system('start notepad.exe ' + tfName)
at the end.
I don't want to save the file permanently on my system. I just want the contents to be opened as a text in Notepad or similar application and delete the file when I close that application.
This could be one of two reasons:
Firstly, by default the temporary file is deleted as soon as it is closed. To fix this use:
tf = tempfile.NamedTemporaryFile(delete=False)
and then delete the file manually once you've finished viewing it in the other application.
Alternatively, it could be that because the file is still open in Python Windows won't let you open it using another application.
Edit: to answer some questions from the comments:
As of the docs from 2 when using delete=False the file can be removed by using:
tf.close()
os.unlink(tf.name)
You can also use it with a context manager so that the file will be closed/deleted when it goes out of scope. It will also be cleaned up if the code in the context manager raises.
import tempfile
with tempfile.NamedTemporaryFile() as temp:
temp.write('Some data')
temp.flush()
# do something interesting with temp before it is destroyed
Here is a useful context manager for this.
(In my opinion, this functionality should be part of the Python standard library.)
# python2 or python3
import contextlib
import os
#contextlib.contextmanager
def temporary_filename(suffix=None):
"""Context that introduces a temporary file.
Creates a temporary file, yields its name, and upon context exit, deletes it.
(In contrast, tempfile.NamedTemporaryFile() provides a 'file' object and
deletes the file as soon as that file object is closed, so the temporary file
cannot be safely re-opened by another library or process.)
Args:
suffix: desired filename extension (e.g. '.mp4').
Yields:
The name of the temporary file.
"""
import tempfile
try:
f = tempfile.NamedTemporaryFile(suffix=suffix, delete=False)
tmp_name = f.name
f.close()
yield tmp_name
finally:
os.unlink(tmp_name)
# Example:
with temporary_filename() as filename:
os.system('echo Hello >' + filename)
assert 6 <= os.path.getsize(filename) <= 8 # depending on text EOL
assert not os.path.exists(filename)