I have a view where the user can upload several files.
#login_required(login_url='/login/')
def FileUploadView(request):
if request.method == 'POST':
form = CallgateUploadForm(request.POST,request.FILES)
fecha = datetime.date.today()
if form.is_valid():
for each in form.cleaned_data['archivo']:
Callgate_Syni.objects.create(fecha=fecha,archivo=each)
os.system('/home/pyc/DjangoProjects/tap_app/media/docs/callgate/envia.sh')
return HttpResponseRedirect('/lista/')
else:
form = CallgateUploadForm()
return render_to_response('callgateupload.html',
{'form':form},
context_instance=RequestContext(request))
Form:
class CallgateUploadForm(forms.ModelForm):
archivo = MultiFileField(min_num=1, max_num=20, max_file_size=1024*1024*5)
def __init__(self,*args,**kwargs):
super(CallgateUploadForm,self).__init__(*args,**kwargs)
self.helper = FormHelper(self)
class Meta:
model = Callgate_Syni
In the same day the user can upload the same file several times, this create a duplicated records for the same file ("archivo"). How can avoid to insert the information of the already uploaded files.
Or if the file already exist update the records in the model and if the file is not exist insert the record.
Thanks in advance.
What you need is the Django QuerySet update_or_create() method.
The usage is simple as:
updated_values = {'archivo': archivo}
obj, created = Callgate_Syni.objects.update_or_create(fecha=fecha, defaults=updated_values)
if created:
print('The object was created')
else:
print('The object was updated')
The keyword args are the filters to specify if the record exists or not, and the defaults parameter is a filedname: fieldvalue dictionary with the fields to update.
The return tuple have the object that was created or updated, and a flag to indicate if the object was created.
From the description and for each in form.cleaned_data['archivo']: I understand what your want is:
If all of the files are new, just save and create;
If some of the files already exist, update them and insert the new files.
So you might want to give a date flag to the filenames so that you can check if the file already exists for a specific date and filename. For example, sample_file_20160125.csv will indicate a file named 'sample_file.csv' has already been uploaded by the user on 01/25/2016. Then you can match such a file in database, if found, update the file object; if not found, create a new record.
If you just want to save the file and not touch the database when the file is already there, use exists() to check if the record in the database.
...
if not Callgate_Syni.objects.filter(fecha=fecha,archivo=each).exists():
Callgate_Syni.objects.create(fecha=fecha,archivo=each)
os.system('/home/pyc/DjangoProjects/tap_app/media/docs/callgate/envia.sh')
...
You can't rely on filenames, but on its content, so one idea is using hash function. Therefore one solution is:
Add hash field to your file model.
When a file is uploaded calculate hash of its content, without the name, so that it won't affect the resulting hash.
Save file to disk, and its hash to DB.
When user uploads another file calculate its hash and see if there's a record with the same hash in DB.
a. If no go to step 3.
b. If yes do nothing.
The drawback calculating hash requires resources, but it's more reliable then just filename.
Related
What is the best way the update the total field with value total the rows the file?
Implement in model or views or other? How to make The file registration will always be through django-admin
models.py
class Registry(models.Model):
file_upload = models.FileField(blank=True, null=False) #csv or xlsx
total = models.CharField(max_length=100, null=True, blank=True, default=None)
def save(self):
with open(self.file_upload) as f:
self.total = sum(1 for line in f)
return self.total
Error:
TypeError: expected str, bytes or os.PathLike object, not FieldFile
You can simply read the file content of the uploaded file as using .read() method.
And then do whatever you want to do with that content.
def save(self):
self.total = sum(1 for line in self.file_upload.read())
super(Registry, self).save(*args, **kwargs)
No need to again open at OS level.
The output of self.file_upload is a FieldFile object. You should change it to self.file_upload.path where will give you the string path of file.
And to makesure your self.file_upload is not None/Null, you should validate it also.
def save(self):
if self.file_upload:
with open(self.file_upload.path) as f:
....
You can read this docs for more https://docs.djangoproject.com/en/dev/topics/files/#using-files-in-models
I generally choose model part if I need to use the method for most of the instances that will be created. But in this case, I probably choose Django Forms to handle this business logic. By the way, you can choose all the possibilities. At least you can achieve what you need in both cases. If the business logics changes very often, I can suggest to move these logics to views or forms.
The error you have encountered is about open statement, which requires one of the types that declared in error message. To achieve that, you can change self.file_upload to self.file_upload.path which is the path that file uploaded. I strongly recommend you to use csv module or an excel file library to handle file read operations.
I am using Django 2.1.1.
I have a model Analysis that, among other fields, contains a ForeignKey to a MyFile model (a model I wrote to handle files):
from polymorphic.models import PolymorphicModel
from django.db.models import Model, DateTimeField, FileField, SET_NULL
from django.db.models.signals import pre_delete
class MyFile(Model):
file = FileField(upload_to='./', null=False, blank=False)
description = CharField(max_length=255, null=True, blank=True)
date_added = DateTimeField(auto_now_add=True)
#receiver(pre_delete, sender=MyFile)
def mymodel_delete(sender, instance, **kwargs):
"""
To delete the file connected to the `sender` class: receive the pre_delete signal
and delete the file associated with the model instance.
"""
instance.file.delete(False)
class Analysis(PolymorphicModel):
# ... other fields ...
file_results = ForeignKey(MyFile, on_delete=SET_NULL,
related_name='file_results',
null=True, blank=True)
Analysis is a PolymorphicModel for reasons related to the bigger project.
In Analysis.file_results I set on_delete=SET_NULL because I want to allow an Analysis instance to exist even without a file_result, which can be populated later.
Let's suppose I have added a few files (the MyFile table has a few rows) and a few Analysis instances. Now, if I want to delete the file related to one of the instances of Analysis I do:
a = Analysis.objects.get(pk=0)
a.file_results.delete()
a.save()
but I get the following error:
File "/Users/mtazzari/djangos/views.py" in update_job_refs
377. a.save()
File "/Users/mtazzari/anaconda/envs/djangos/lib/python3.6/site-packages/polymorphic/models.py" in save
83. return super(PolymorphicModel, self).save(*args, **kwargs)
File "/Users/mtazzari/anaconda/envs/djangos/lib/python3.6/site-packages/django/db/models/base.py" in save
670. "unsaved related object '%s'." % field.name
ValueError: save() prohibited to prevent data loss due to unsaved
related object 'file_results'.
The mymodel_delete function that is called on pre_delete signal works correctly as the file gets actually deleted from the file system.
However, I really don't understand how to solve the ValueError.
Interestingly, I notice that the following lines work fine, i.e. do not raise any ValueError, get the file deleted from the file system, and the FK in a.file_results set to Null:
a = Analysis.objects.get(pk=0)
tmp = a.file_results
a.file_results = None
tmp.file_results.delete()
a.save()
But, is this a proper way of doing this? What is the best practice for deleting a related object?
Thanks!
First, note that you don't need to save() just because of the delete(). The delete() will update the database as required.
That said, it's reasonable to want to continue using the instance to do other operations, leading to a save(). The reason you're getting the error is that the a.file_results Python object still exists, and references a database row that is now missing. The documentation for delete() mentions this:
This only deletes the object in the database; the Python instance will still exist and will still have data in its fields.
So if you want to continue to work with the instance object, just set the attribute to None yourself. Similar to your code above, except you don't need the temp object.
a = Analysis.objects.get(pk=0)
a.file_results.delete()
a.file_results = None
# ... more operations on a
a.save() # no error
I am trying to create a folder for each users to put their project in. So their file will have the path ..\project\id\filename, id is the user id and filename is the name of the file. Now using the arguments allowed for upload_to (instance and filename) in the Filefield, I realize that instance.id will be None and the path to the file will be ..\project\None\filename instead of ..\project\id\filename.
Now reading the Django documentation upload_to I saw this:
In most cases, this object will not have been saved to the database
yet, so if it uses the default AutoField, it might not yet have a
value for its primary key field.
My interpretation is that creating a new record and user_directory_path are not instantiated at the same time, that is, when I call create on Project model, instance.id will be None. My question is now, is there a way to get around this? While I see upload_to convenient, it is not necessarily convenient for dynamic path such as the one I am doing. I was thinking of creating the record, then adding the file path in an update, but I am in search of a way that can save everything in one step.
models.py
def user_directory_path(instance, filename):
# file will be uploaded to MEDIA_ROOT/user_<id>/<filename>
return 'project/{0}/{1}'.format(instance.user.id, filename)
class Project(models.Model):
email = models.ForeignKey(User,
to_field="email",
max_length=50
)
title = models.CharField(max_length=100)
date_created = models.DateTimeField(auto_now_add=True)
updated = models.DateTimeField(auto_now=True)
file = models.FileField(upload_to=user_directory_path, validators=[validate_file_type], null=True)
This is views.py when the form passes validation. Notice user_directory_path is called just before the create.
email = request.user.email
title = request.POST.get('title', '')
file = request.FILES['file']
filename = file.name
instance = Usermie.objects.get(email=request.user.email)
# Save to model
user_directory_path(instance=instance, filename=filename)
Project.objects.create(
title=title, file=file,
)
If, as you say, the id that you want to use in the file path is the id of the User, not the id of the Project.. then there's no problem because the User already exists when you are saving the Project. Since email is a foreign key to User, you would just do:
def user_directory_path(instance, filename):
# file will be uploaded to MEDIA_ROOT/user_<id>/<filename>
return 'project/{0}/{1}'.format(instance.email.id, filename)
But I will point out that, in the Django way of doing things, making a field called email that is a foreign key to User is actually pretty confusing. The field in the database will be called email_id.. and the value of the model field will return an instance of User.. not the actual email address, even though the email address is what's stored in the column. To get the email address you'd need to do one of:
myproject.email.email
myproject.email_id
Neither one is very clear. So unless you have a really good reason for doing it like that, you should call the field user and eliminate the to_field='email'. Allow Django to join the tables via id, which is the default behavior.
Then if you need the user email address you can get it any time via
myproject.user.email
And the bonus is that if the user changes their email address it will change everywhere, you don't have to rely on cascaded updates to fix all the foreign keys.
Trust me, when using Django you want to do ForeignKey by id (the default) unless there's a reason...
One simple solution can be saving object without file and then saving file like this
email = request.user.email
title = request.POST.get('title', '')
file = request.FILES['file']
filename = file.name
instance = Usermie.objects.get(email=request.user.email)
# Save to model
user_directory_path(instance=instance, filename=filename)
project = Project.objects.create(title=title)
project.file = file
project.save()
I am new to django and sqlite. From the code below i will upload my file to 'documents', since there is an overwrite storage function, if i upload the same file name with another query i will overwrite the old one which i dont wan it to be happend. So what I am thinking is to get the query id or pk as the directory.Its there anyway to store my upload file via their pk or id?
eg. document/name/test.zip
eg. document/name/test.zip
First define a function
def upload(instance, filename):
return "document/" + instance.user.name +"/" + filename
And in the filefield of model
upload_to=upload
I have a number of models, each with a user field. Each time a new user is registered I want to copy the instances of these models which have user.id = 1, and save them with a new user id in the user field. I've read that a copy of a model instance can be made by setting the pk to None so this is what I've tried to do first. I'm trying to do this on a QuerySet since there are many instances for some tables (over 1000 for some models, though mostly less than 100).
def copy_tables(user):
# This approach fails with "Voice has no field named 'pk'"
voice = Voice.objects.filter(user=1).update(
pk=None, user=user.id, right_answers=0, wrong_answers=0)
I've also tried using copy.deepcopy(Voice.objects.filter(user=1)) but calling update() on this is also still changing the original table in the database.
def copy_tables(user):
copy.deepcopy(Voice.objects.filter(user=1)).update(
user=user.id, right_answers=0, wrong_answers=0)
# This test throws an error because the original models have been changed
assert Voice.objects.filter(user=1).first()
Am I missing some other way of doing this? Also, I thought that all model instances had a primary key, pk, which is automatically set by Django. Is this not the case?
When calling the save method on a model instance, Django determines if it must be inserted or updated by checking if it has a primary key (ref.).
So if you set the PK to None on a model instance, Django will trying adding it as a new row in the database.
Demo:
# Retrieve the object belonging to the user whose ID is 1.
# Note that you cannot use "user=1", you need to tell Django you
# are querying by id, using "user__id=1" .
obj = MyModel.objects.get(user__id=1)
# Set PK to None so that Django thinks it is not save in the database.
obj.pk = None
# Save to insert it.
obj.save()
In your case where you have multiple objects, you can use bulk_create to gain in performances:
voices = Voice.objects.filter(user__id=1)
new_voices = []
for voice in voices:
voice.pk = None
voice.user = user
right_answsers = 0
wrong_anwsers = 0
new_voices.append(voice)
Voice.objects.bulk_create(new_voices)