I am working through an online python course and have an issue that I am having trouble working through. I have a executable directory with the following layout
reader/
|--__main__.py
|--reader
|--__init__.py
|--reader.py
|--compressed
|--gzipped.py
|--bzipped.py
|--__init__.py
When I do not have these modules in the top directory, I can import just fine and use all of the functionality. When I put them in the top level directory and run the executable directory from the command line with
python reader test.gz
I am getting the following error
AttributeError: module 'reader' has not attribute 'Reader'
The code for main.py is
import sys
import reader
r = reader.Reader(sys.argv[1])
try:
print(r.read())
finally:
r.close()
The code for reader.py is
import os
from reader.compressed import gzipped, bzipped
extension_map = {
'.bz2': bzipped.opener,
'.gz': gzipped.opener,
}
class Reader:
def __init__(self, filename):
extension = os.path.splitext(filename)[1]
opener = extension_map.get(extension, open)
self.f = opener(filename, 'rt')
def close(self):
self.f.close()
def read(self):
return self.f.read()
I can provide the rest of the files if needed. I am using the current distribution of Anaconda. Any help or explanations would be appreciated.
It seems to me that because of your __init__.py file the folder reader is seen as a python module. Thereore your main.py code tries to find the class Reader at the folder level, but you need to look for Reader within reader.py. Can you try changing
r = reader.Reader(sys.argv[1])
to
r = reader.reader.Reader(sys.argv[1])
Also, you have a lot of modules/files/modules called reader, e.g. the class Reader is in reader/reader/reader.py. This I would try to avoid as it can lead to confusion.
When you have multiple modules/modules that are named identically you can often get the namespace into a recursive loop. Also, the Reader class is capitalized.
Like mentioned above, try python reader.reader test.gz
While you are troubleshooting try the tab-complete feature to see if your modules are being loaded properly.
Related
In my Django project I have the following directory structure:
project/build/contracts/MyFile.json
And I am writing code in the following directory
project/homeapp/views.py
In my views.py I have the following code:
with open("../build/contracts/MyFile.json", "r") as f:
data = json.loads(f.read())
abi = data["abi"]
When I try to python manage.py runserver I get the following error:
The strange part is that I couldn't figure out what was wrong so I made a viewstest.py and placed the exact same code in it. When I run it with python .\viewstest.py and print the JSON to console, it works perfectly fine.
I even tried importing the abi variable from viewstest.py to views.py but got the same error. So I assume that it is an error relating to Django, but I can't seem to figure it out.
Thanks!
It should be json.load() and not json.loads()
Change the following code to:
with open("../build/contracts/MyFile.json", "r") as file:
data = json.load(file)
abi = data["abi"]
Edit:
Another alternative to get the path correct can be to use Pathlib.
from pathlib import Path
def your_view_func(request):
current_dir = Path.cwd()
file_loc = 'build/contracts/MyFile.json'
with open(current_dir.joinpath(file_loc), 'r') as file:
data = json.load(file)
abi = data["abi"]
I want to create a tool called unifile for saving and opening files
like this unifile.open.yaml("file.yaml").
This is my structure:
unifile
|
├-open
| └--__init__.py
|
└-save
└--__init__.py
Code that call my module:
import unifile
a = unifile.open.yaml("file.yaml")
open/init.py
import yaml
class open():
def yml(self, file_path):
try:
with open(file_path, "r", encoding="utf-8") as yaml_conf:
yaml_file = yaml.safe_load(yaml_conf)
return yaml_file
except OSError:
print("Can't load yaml")
1 error if I import unifile always say:
module unifile has no atribute open
2 error in __init__.py I can't open file
[pylint] Context manager 'open' doesn't implement enter and exit. [not-context-manager]
here adding solution to ur problem, make your project structure like this.
add unifile/__init__.py file in the unifile itself not in other modules.
then unifile/open/_open.py file content
import yaml
class Open():
def __init__(self):
pass
def yml(self, file_path):
try:
with open(file_path, "r", encoding="utf-8") as yaml_conf:
yaml_file = yaml.safe_load(yaml_conf)
return yaml_file
except OSError:
print("Can't load yaml")
content of the unifile/__init__.py file
from .open._open import Open
in terminal run the program like this
Also, It is better to create a object element first then proceed ahead.
Two issues, two answers.
First, you should add an init file in unifile. With this, Python will understand that unifile is a package with a sub package.
Second, open is a built-in function and you overwrite it by calling your class open. Change your class name and it should work.
You are getting this error because unifile is not a package. There isn't any init.py file at the top level same as open and save. You also cannot call open.yml directly, because open is a class in package open, so either you will have to import open from open, create its instance and then call iml on that instance.
from open import open
a = open().yml('file.yml')
You are getting this error, because you are trying to override an existing keyword in Python open which you should strictly prohibit doing. So you should name your class anything except a reserved keyword.
I am having trouble creating and writing to a text file in Python. I am running Python 3.5.1 and have the following code to try and create and write to a file:
from os import *
custom_path = "MyDirectory/"
if not path.exists(custom_path)
mkdir(custom_path)
text_path = custom_path + "MyTextFile.txt"
text_file = open(text_path, "w")
text_file.write("my text")
But I get a TypeError saying an integer is required (got type str) at the line text_file = open(text_path, "w").
I don't know what I'm doing wrong as my code is just about identical to that of several tutorial sites showing how to create and write to files.
Also, does the above code create the text file if it doesn't exist, and if not how do I create it?
Please don't import everything from os module:
from os import path, mkdir
custom_path = "MyDirectory/"
if not path.exists(custom_path):
mkdir(custom_path)
text_path = custom_path + "MyTextFile.txt"
text_file = open(text_path, 'w')
text_file.write("my text")
Because there also a "open" method in os module which will overwrite the native file "open" method.
I have the following structure:
run.py
app/hdfs_lib_test.py
app/src/HDFSFileReader.py
This is a Flask app.
The HSFDFileReader.py contains:
class HDFSFileReader:
"""This class represents a reader for accessing files in a HDFS."""
def __init__(self):
pass
#classmethod
def read(cls, file_path):
lines = []
try:
client = Config().get_client('dev')
with client.read('Serien_de', encoding='utf-8', delimiter='\n') as reader:
for line in reader:
lines.append(line)
except:
print("ERROR: Could not read from HDFS.")
raise
return lines
When I run ./run.py I get the ImportError: No module named 'hdfs'. However the library is installed and I can call python hdfs_lib_test.py, which contains the following:
from hdfs import Config
try:
client = Config().get_client('dev')
with client.read('Serien_de',encoding='utf-8',delimiter='\n') as reader:
for line in reader:
print(line)
except:
raise
So to me it seems, that because HDFSFileReader is part of a submodule and hdfs_lib_test.py isn't. The former doesn't work, but the later does. But how do I fix the import problem?
I can't be a circular import issue, I searched the whole project for imports of hdfs and the hdfs_lib_test.py is not used by the actual project.
I'm trying to build tests for some models that have a FileField. The model looks like this:
class SolutionFile(models.Model):
'''
A file from a solution.
'''
solution = models.ForeignKey(Solution)
file = models.FileField(upload_to=make_solution_file_path)
I have encountered two problems:
When saving data to a fixture using ./manage.py dumpdata, the file contents are not saved, only the file name is saved into the fixture. While I find this to be the expected behavior as the file contents are not saved into the database, I'd like to somehow include this information in the fixture for tests.
I have a test case for uploading a file that looks like this:
def test_post_solution_file(self):
import tempfile
import os
filename = tempfile.mkstemp()[1]
f = open(filename, 'w')
f.write('These are the file contents')
f.close()
f = open(filename, 'r')
post_data = {'file': f}
response = self.client.post(self.solution.get_absolute_url()+'add_solution_file/', post_data,
follow=True)
f.close()
os.remove(filename)
self.assertTemplateUsed(response, 'tests/solution_detail.html')
self.assertContains(response, os.path.basename(filename))
While this test works just fine, it leaves the uploaded file in the media directory after finishing. Of course, the deletion could be taken care of in tearDown(), but I was wondering if Django had another way of dealing with this.
One solution I was thinking of was using a different media folder for tests which must be kept synced with the test fixtures. Is there any way to specify another media directory in settings.py when tests are being run? And can I include some sort of hook to dumpdata so that it syncs the files in the media folders?
So, is there a more Pythonic or Django-specific way of dealing with unit tests involving files?
Django provides a great way to write tests on FileFields without mucking about in the real filesystem - use a SimpleUploadedFile.
from django.core.files.uploadedfile import SimpleUploadedFile
my_model.file_field = SimpleUploadedFile('best_file_eva.txt', b'these are the contents of the txt file')
It's one of django's magical features-that-don't-show-up-in-the-docs :). However it is referred to here.
You can override the MEDIA_ROOT setting for your tests using the #override_settings() decorator as documented:
from django.test import override_settings
#override_settings(MEDIA_ROOT='/tmp/django_test')
def test_post_solution_file(self):
# your code here
I've written unit tests for an entire gallery app before, and what worked well for me was using the python tempfile and shutil modules to create copies of the test files in temporary directories and then delete them all afterwards.
The following example is not working/complete, but should get you on the right path:
import os, shutil, tempfile
PATH_TEMP = tempfile.mkdtemp(dir=os.path.join(MY_PATH, 'temp'))
def make_objects():
filenames = os.listdir(TEST_FILES_DIR)
if not os.access(PATH_TEMP, os.F_OK):
os.makedirs(PATH_TEMP)
for filename in filenames:
name, extension = os.path.splitext(filename)
new = os.path.join(PATH_TEMP, filename)
shutil.copyfile(os.path.join(TEST_FILES_DIR, filename), new)
#Do something with the files/FileField here
def remove_objects():
shutil.rmtree(PATH_TEMP)
I run those methods in the setUp() and tearDown() methods of my unit tests and it works great! You've got a clean copy of your files to test your filefield that are reusable and predictable.
with pytest and pytest-django, I use this in conftest.py file:
import tempfile
import shutil
from pytest_django.lazy_django import skip_if_no_django
from pytest_django.fixtures import SettingsWrapper
#pytest.fixture(scope='session')
##pytest.yield_fixture()
def settings():
"""A Django settings object which restores changes after the testrun"""
skip_if_no_django()
wrapper = SettingsWrapper()
yield wrapper
wrapper.finalize()
#pytest.fixture(autouse=True, scope='session')
def media_root(settings):
tmp_dir = tempfile.mkdtemp()
settings.MEDIA_ROOT = tmp_dir
yield settings.MEDIA_ROOT
shutil.rmtree(tmp_dir)
#pytest.fixture(scope='session')
def django_db_setup(media_root, django_db_setup):
print('inject_after')
might be helpful:
https://dev.funkwhale.audio/funkwhale/funkwhale/blob/de777764da0c0e9fe66d0bb76317679be964588b/api/tests/conftest.py
https://framagit.org/ideascube/ideascube/blob/master/conftest.py
https://stackoverflow.com/a/56177770/5305401
This is what I did for my test. After uploading the file it should end up in the photo property of my organization model object:
import tempfile
filename = tempfile.mkstemp()[1]
f = open(filename, 'w')
f.write('These are the file contents')
f.close()
f = open(filename, 'r')
post_data = {'file': f}
response = self.client.post("/org/%d/photo" % new_org_data["id"], post_data)
f.close()
self.assertEqual(response.status_code, 200)
## Check the file
## org is where the file should end up
org = models.Organization.objects.get(pk=new_org_data["id"])
self.assertEqual("These are the file contents", org.photo.file.read())
## Remove the file
import os
os.remove(org.photo.path)