What is the best way the update the total field with value total the rows the file?
Implement in model or views or other? How to make The file registration will always be through django-admin
models.py
class Registry(models.Model):
file_upload = models.FileField(blank=True, null=False) #csv or xlsx
total = models.CharField(max_length=100, null=True, blank=True, default=None)
def save(self):
with open(self.file_upload) as f:
self.total = sum(1 for line in f)
return self.total
Error:
TypeError: expected str, bytes or os.PathLike object, not FieldFile
You can simply read the file content of the uploaded file as using .read() method.
And then do whatever you want to do with that content.
def save(self):
self.total = sum(1 for line in self.file_upload.read())
super(Registry, self).save(*args, **kwargs)
No need to again open at OS level.
The output of self.file_upload is a FieldFile object. You should change it to self.file_upload.path where will give you the string path of file.
And to makesure your self.file_upload is not None/Null, you should validate it also.
def save(self):
if self.file_upload:
with open(self.file_upload.path) as f:
....
You can read this docs for more https://docs.djangoproject.com/en/dev/topics/files/#using-files-in-models
I generally choose model part if I need to use the method for most of the instances that will be created. But in this case, I probably choose Django Forms to handle this business logic. By the way, you can choose all the possibilities. At least you can achieve what you need in both cases. If the business logics changes very often, I can suggest to move these logics to views or forms.
The error you have encountered is about open statement, which requires one of the types that declared in error message. To achieve that, you can change self.file_upload to self.file_upload.path which is the path that file uploaded. I strongly recommend you to use csv module or an excel file library to handle file read operations.
Related
I am using Django 2.1.1.
I have a model Analysis that, among other fields, contains a ForeignKey to a MyFile model (a model I wrote to handle files):
from polymorphic.models import PolymorphicModel
from django.db.models import Model, DateTimeField, FileField, SET_NULL
from django.db.models.signals import pre_delete
class MyFile(Model):
file = FileField(upload_to='./', null=False, blank=False)
description = CharField(max_length=255, null=True, blank=True)
date_added = DateTimeField(auto_now_add=True)
#receiver(pre_delete, sender=MyFile)
def mymodel_delete(sender, instance, **kwargs):
"""
To delete the file connected to the `sender` class: receive the pre_delete signal
and delete the file associated with the model instance.
"""
instance.file.delete(False)
class Analysis(PolymorphicModel):
# ... other fields ...
file_results = ForeignKey(MyFile, on_delete=SET_NULL,
related_name='file_results',
null=True, blank=True)
Analysis is a PolymorphicModel for reasons related to the bigger project.
In Analysis.file_results I set on_delete=SET_NULL because I want to allow an Analysis instance to exist even without a file_result, which can be populated later.
Let's suppose I have added a few files (the MyFile table has a few rows) and a few Analysis instances. Now, if I want to delete the file related to one of the instances of Analysis I do:
a = Analysis.objects.get(pk=0)
a.file_results.delete()
a.save()
but I get the following error:
File "/Users/mtazzari/djangos/views.py" in update_job_refs
377. a.save()
File "/Users/mtazzari/anaconda/envs/djangos/lib/python3.6/site-packages/polymorphic/models.py" in save
83. return super(PolymorphicModel, self).save(*args, **kwargs)
File "/Users/mtazzari/anaconda/envs/djangos/lib/python3.6/site-packages/django/db/models/base.py" in save
670. "unsaved related object '%s'." % field.name
ValueError: save() prohibited to prevent data loss due to unsaved
related object 'file_results'.
The mymodel_delete function that is called on pre_delete signal works correctly as the file gets actually deleted from the file system.
However, I really don't understand how to solve the ValueError.
Interestingly, I notice that the following lines work fine, i.e. do not raise any ValueError, get the file deleted from the file system, and the FK in a.file_results set to Null:
a = Analysis.objects.get(pk=0)
tmp = a.file_results
a.file_results = None
tmp.file_results.delete()
a.save()
But, is this a proper way of doing this? What is the best practice for deleting a related object?
Thanks!
First, note that you don't need to save() just because of the delete(). The delete() will update the database as required.
That said, it's reasonable to want to continue using the instance to do other operations, leading to a save(). The reason you're getting the error is that the a.file_results Python object still exists, and references a database row that is now missing. The documentation for delete() mentions this:
This only deletes the object in the database; the Python instance will still exist and will still have data in its fields.
So if you want to continue to work with the instance object, just set the attribute to None yourself. Similar to your code above, except you don't need the temp object.
a = Analysis.objects.get(pk=0)
a.file_results.delete()
a.file_results = None
# ... more operations on a
a.save() # no error
I´m trying to upload file into database, but when i want put the function uoload to and i want to that function store the file in a root with the data the user submit in the for, for example year, course, section, I get that information and the file they uploaded I want to store in that subfolders, the code that I´m using only store the data in the Media_root, but not in the carpets how can I do in order to store the files in the subfolder I want. I'n my model document I store the information of the document, like the autor, the title,etc.
class File(models.Model):
titulo_file=models.ForeignKey(Document,on_delete=models.CASCADE,null=True,verbose_ name='Título de la Tesis')
file = models.FileField(null=True,blank=True, upload_to=generate_path)
Here the function I use to upload the file, Int only works to store the file into a folder with it´s same name.
def generate_path(instance, filename):
path =os.path.join(instance.file.name,filename)
return path
In my view, after the user submit the information, I use a os.path.join, that do the path with the information submited like this:
year/course/section
I want to send that path to my generate_path and store in that location the file, I tried with session but it doesn't work, what can I do?
models.py
class Year(models.Model):
year_number=models.CharField(max_length=10,primary_key=True)
class Course(models.Model):
name=models.CharField(max_length=50,primary_key=True)
class Section(models.Model):
numberSection=models.IntegerField(null=True,blank=True)
year = models.ForeignKey(Year, on_delete=models.CASCADE, null=True)
course = models.ForeignKey(Course, on_delete=models.CASCADE)
thopic_documents=(
('CS','CS'),
('SE','SE'),
('IS','IS'),
('IT','IT'),
)
class Document(models.Model):
title=models.CharField(max_length=200,primary_key=True)
thopic=models.CharField(max_length=50,choices=thopic_documents, default=None,null=True)
class Historial(models.Model):
id_section=models.ForeignKey(Section,on_delete=models.CASCADE)
title_document=models.ForeignKey(Document,on_delete=models.CASCADE,)
To get the year/course/section you need to be able to navigate from a File to a Section. However, the many-many relationship that exists between Document and Section through Historical makes it unclear how you would do that. Which Historical would you choose?
It may be better if File was linked directly to Historical:
class File(models.Model):
titulo_file=models.ForeignKey(Historical, on_delete=models.CASCADE,null=True,verbose_name='Título de la Tesis')
file = models.FileField(null=True,blank=True, upload_to=generate_path)
If that was the case, you could then implement your generate_path() such as:
def generate_path(instance, filename):
section = instance.titulo_file.id_section
year = section.year.year_number
course = section.course.name
return os.path.join(str(year), course, str(section.numberSection), filename)
To do the same thing with the model as it currently stands, you would have to do something like this:
def generate_path(instance, filename):
section = instance.titulo_file.historical_set.first().id_section
year = section.year.year_number
course = section.course.name
return os.path.join(str(year), course, str(section.numberSection), filename)
That example uses historical_set.first() to get the first Historical linked the Document. Maybe that would be ok, but otherwise you'd need to know which Historical to use.
Where a file with the same name is uploaded and you don't want to overwrite it, you could implement your own storage:
from django.core.files.storage import FileSystemStorage
class UseExistingStorage(FileSystemStorage):
def save(self, name, content, max_length=None):
if not self.exists(name):
return super().save(name, content, max_length)
return name # Don't save when the file exists, just return the name
Then reference the UseExistingStorage from your File model.
file = models.FileField(null=True,blank=True, upload_to=generate_path, storage=UseExistingStorage())
I am trying to create a folder for each users to put their project in. So their file will have the path ..\project\id\filename, id is the user id and filename is the name of the file. Now using the arguments allowed for upload_to (instance and filename) in the Filefield, I realize that instance.id will be None and the path to the file will be ..\project\None\filename instead of ..\project\id\filename.
Now reading the Django documentation upload_to I saw this:
In most cases, this object will not have been saved to the database
yet, so if it uses the default AutoField, it might not yet have a
value for its primary key field.
My interpretation is that creating a new record and user_directory_path are not instantiated at the same time, that is, when I call create on Project model, instance.id will be None. My question is now, is there a way to get around this? While I see upload_to convenient, it is not necessarily convenient for dynamic path such as the one I am doing. I was thinking of creating the record, then adding the file path in an update, but I am in search of a way that can save everything in one step.
models.py
def user_directory_path(instance, filename):
# file will be uploaded to MEDIA_ROOT/user_<id>/<filename>
return 'project/{0}/{1}'.format(instance.user.id, filename)
class Project(models.Model):
email = models.ForeignKey(User,
to_field="email",
max_length=50
)
title = models.CharField(max_length=100)
date_created = models.DateTimeField(auto_now_add=True)
updated = models.DateTimeField(auto_now=True)
file = models.FileField(upload_to=user_directory_path, validators=[validate_file_type], null=True)
This is views.py when the form passes validation. Notice user_directory_path is called just before the create.
email = request.user.email
title = request.POST.get('title', '')
file = request.FILES['file']
filename = file.name
instance = Usermie.objects.get(email=request.user.email)
# Save to model
user_directory_path(instance=instance, filename=filename)
Project.objects.create(
title=title, file=file,
)
If, as you say, the id that you want to use in the file path is the id of the User, not the id of the Project.. then there's no problem because the User already exists when you are saving the Project. Since email is a foreign key to User, you would just do:
def user_directory_path(instance, filename):
# file will be uploaded to MEDIA_ROOT/user_<id>/<filename>
return 'project/{0}/{1}'.format(instance.email.id, filename)
But I will point out that, in the Django way of doing things, making a field called email that is a foreign key to User is actually pretty confusing. The field in the database will be called email_id.. and the value of the model field will return an instance of User.. not the actual email address, even though the email address is what's stored in the column. To get the email address you'd need to do one of:
myproject.email.email
myproject.email_id
Neither one is very clear. So unless you have a really good reason for doing it like that, you should call the field user and eliminate the to_field='email'. Allow Django to join the tables via id, which is the default behavior.
Then if you need the user email address you can get it any time via
myproject.user.email
And the bonus is that if the user changes their email address it will change everywhere, you don't have to rely on cascaded updates to fix all the foreign keys.
Trust me, when using Django you want to do ForeignKey by id (the default) unless there's a reason...
One simple solution can be saving object without file and then saving file like this
email = request.user.email
title = request.POST.get('title', '')
file = request.FILES['file']
filename = file.name
instance = Usermie.objects.get(email=request.user.email)
# Save to model
user_directory_path(instance=instance, filename=filename)
project = Project.objects.create(title=title)
project.file = file
project.save()
I have a model for which I am making an api in tastypie. I have a field which stores the path to a file which I maintain manually (I am not using FileField since users are not uploading the files). Here is a gist of a model:
class FooModel(models.Model):
path = models.CharField(max_length=255, null=True)
...
def getAbsPath(self):
"""
returns the absolute path to a file stored at location self.path
"""
...
Here is my tastypie config:
class FooModelResource(ModelResource):
file = fields.FileField()
class Meta:
queryset = FooModel.objects.all()
def dehydrate_file(self, bundle):
from django.core.files import File
path = bundle.obj.getAbsPath()
return File(open(path, 'rb'))
In the api in the file field this returns full path to a file. I want tastypie to be able to serve the actual file or at least an url to a file. How do I do that? Any code snippets are appreciated.
Thank you
Decide on a URL scheme how your files will be exposed through the APIs first. You don't really need file or dehydrate_file (unless you want to change the representation of the file for the model itself in Tastypie). Instead just add an additional action on the ModelResource. Example:
class FooModelResource(ModelResource):
file = fields.FileField()
class Meta:
queryset = FooModel.objects.all()
def override_urls(self):
return [
url(r"^(?P<resource_name>%s)/(?P<pk>\w[\w/-]*)/download%s$" % (self._meta.resource_name, trailing_slash()), self.wrap_view('download_detail'), name="api_download_detail"),
]
def download_detail(self, request, **kwargs):
"""
Send a file through TastyPie without loading the whole file into
memory at once. The FileWrapper will turn the file object into an
iterator for chunks of 8KB.
No need to build a bundle here only to return a file, lets look into the DB directly
"""
filename = self._meta.queryset.get(pk=kwargs[pk]).file
wrapper = FileWrapper(file(filename))
response = HttpResponse(wrapper, content_type='text/plain') #or whatever type you want there
response['Content-Length'] = os.path.getsize(filename)
return response
GET .../api/foomodel/3/
Returns:
{
...
'file' : 'localpath/filename.ext',
...
}
GET .../api/foomodel/3/download/
Returns:
...contents of actual file...
Alternatively you could create a non-ORM Sub Resource file in FooModel. You would have to define resource_uri (how to uniquely identify each instance of the resource), and override dispatch_detail to do exactly what download_detail above does.
The only conversion tastypie does on a FileField is to look for an 'url' attribute on what you return, and return that if it exists, else it will return the string-ized object, which as you have noticed is just the filename.
If you want to return the file content as a field, you will need to handle the encoding of the file. You have a few options:
Simplest: Use CharField and use the base64 module to convert the bytes read from the file into a string
More general but functionally equivalent: write a custom tastypie Serializer that knows how to turn File objects into string representations of their contents
Override the get_detail function of your resource to serve just the file using whatever content-type is appropriate, to avoid the JSON/XML serialization overhead.
I'm trying to rename a file after it's uploaded in the model's save method. I'm renaming the file to a combination the files primary key and a slug of the file title.
I have it working when a file is first uploaded, when a new file is uploaded, and when there are no changes to the file or file title.
However, when the title of the file is changed, and the system tries to rename the old file to the new path I get the following error:
WindowsError at /admin/main/file/1/
(32, 'The process cannot access the file because it is being used by another process')
I don't really know how to get around this. I've tried just coping the file to the new path. This works, but I don't know I can delete the old version.
Shortened Model:
class File(models.Model):
nzb = models.FileField(upload_to='files/')
name = models.CharField(max_length=256)
name_slug = models.CharField(max_length=256, blank=True, null=True, editable=False)
def save(self):
# Create the name slug.
self.name_slug = re.sub('[^a-zA-Z0-9]', '-', self.name).strip('-').lower()
self.name_slug = re.sub('[-]+', '-', self.name_slug)
# Need the primary key for naming the file.
super(File, self).save()
# Create the system paths we need.
orignal_nzb = u'%(1)s%(2)s' % {'1': settings.MEDIA_ROOT, '2': self.nzb}
renamed_nzb = u'%(1)sfiles/%(2)s_%(3)s.nzb' % {'1': settings.MEDIA_ROOT, '2': self.pk, '3': self.name_slug}
# Rename the file.
if orignal_nzb not in renamed_nzb:
if os.path.isfile(renamed_nzb):
os.remove(renamed_nzb)
# Fails when name is updated.
os.rename(orignal_nzb, renamed_nzb)
self.nzb = 'files/%(1)s_%(2)s.nzb' % {'1': self.pk, '2': self.name_slug}
super(File, self).save()
I suppose the question is, does anyone know how I can rename an uploaded file when the uploaded file isn't be re-uploaded? That's the only time it appears to be locked/in-use.
Update:
Tyler's approach is working, except when a new file is uploaded the primary key is not available and his technique below is throwing an error.
if not instance.pk:
instance.save()
Error:
maximum recursion depth exceeded while calling a Python object
Is there any way to grab the primary key?
I think you should look more closely at the upload_to field. This would probably be simpler than messing around with renaming during save.
http://docs.djangoproject.com/en/dev/ref/models/fields/#filefield
This may also be a callable, such as a
function, which will be called to
obtain the upload path, including the
filename. This callable must be able
to accept two arguments, and return a
Unix-style path (with forward slashes)
to be passed along to the storage
system. The two arguments that will be
passed are:
My other answer is deprecated, use this instead:
class File(models.Model):
nzb = models.FileField(upload_to=get_filename)
...
def get_filename(instance, filename):
if not instance.pk:
instance.save()
# Create the name slug.
name_slug = re.sub('[^a-zA-Z0-9]', '-', instance.name).strip('-').lower()
name_slug = re.sub('[-]+', '-', name_slug)
filename = u'filess/%(2)s_%(3)s.nzb' % {'2': instance.pk, '3': name_slug}
return filename
As of 1.0, upload_to can be callable, in which case it is expected to return the filename, including path (relative to MEDIA_ROOT).
Once uploaded, all you have is an image object in memory, right?
You could save this object yourself in the folder of your choice, and then edit the database entry by hand.
You'd be bypassing the whole Django ORM, and is not something I'd do unlessI couldn't find a more Django way.