We Keep Coding

What is the "instance" being passed to the to_representation function of my ListSerializer?

The goal of this project is to create an API that refreshes hourly with the most up to date betting odds for a list of games that I'll be scraping hourly from the internet. The goal structure for the JSON returned will be each game as the parent object and the nested children will be the top 1 record for each of linesmakers being scraped by updated date. My understanding is that the best way to accomplish this is to modify the to_representation function within the ListSerializer to return the appropriate queryset.
Because I need the game_id of the parent element to grab the children of the appropriate game, I've attempted to pull the game_id out of the data that gets passed. The issue is that this line looks to be populated correctly when I see what it contains through an exception, but when I let the full code run, I get a list index is out of range exception.
For ex.
class OddsMakerListSerializer(serializers.ListSerializer):
def to_representation(self, data):
game = data.all()[0].game_id
#if I put this here it evaluates to 1 which should run the raw sql below correctly
raise Exception(game)
data = OddsMaker.objects.filter(odds_id__in = RawSQL(''' SELECT o.odds_id
FROM gamesbackend_oddsmaker o
INNER JOIN (
SELECT game_id
, oddsmaker
, max(updated_datetime) as last_updated
FROM gamesbackend_oddsmaker
WHERE game_id = %s
GROUP BY game_id
, oddsmaker
) l on o.game_id = l.game_id
and o.oddsmaker = l.oddsmaker
and o.updated_datetime = l.last_updated
''', [game]))
#if I put this here the data appears to be populated correctly and contain the right data
raise Exception(data)
data = [game for game in data]
return data
Now, if I remove these raise Exceptions, I get the list index is out of range. My initial thought was that there's something else that depends on "data" being returned as a list, so I created the list comprehension snippet, but that doesn't resolve the issue.
So, my question is 1) Is there an easier way to accomplish what I'm going for? I'm not using a postgres backend so distinct on isn't available to me. and 2) If not, its not clear to me what instance is that's being passed in or what is expected to be returned. I've consulted the documentation and it looks as though it expects a dictionary and that might be part of the issue, but again the error message references a list. https://www.django-rest-framework.org/api-guide/serializers/#overriding-serialization-and-deserialization-behavior
I appreciate any help in understanding what is going on here in advance.
Edit:
The rest of the serializers:
class OddsMakerSerializer(serializers.ModelSerializer):
class Meta:
list_serializer_class = OddsMakerListSerializer
model = OddsMaker
fields = ('odds_id','game_id','oddsmaker','home_ml',
'away_ml','home_spread','home_spread_odds',
'away_spread_odds','total','total_over_odds',
'total_under_odds','updated_datetime')
class GameSerializer(serializers.ModelSerializer):
oddsmaker_set = OddsMakerSerializer(many=True, read_only=True)
class Meta:
model = Game
fields = ('game_id','date','sport', 'home_team',
'away_team','home_score', 'away_score',
'home_win','away_win', 'game_completed',
'oddsmaker_set')
models.py:
class Game(models.Model):
game_id = models.AutoField(primary_key=True)
date = models.DateTimeField(null=True)
sport=models.CharField(max_length=256, null=True)
home_team = models.CharField(max_length=256, null=True)
away_team = models.CharField(max_length=256, null=True)
home_score = models.IntegerField(default=0, null=True)
away_score = models.IntegerField(default=0, null=True)
home_win = models.BooleanField(default=0, null=True)
away_win = models.BooleanField(default=0, null=True)
game_completed = models.BooleanField(default=0, null=True)
class OddsMaker(models.Model):
odds_id = models.AutoField(primary_key=True)
game = models.ForeignKey('Game', on_delete = models.CASCADE)
oddsmaker = models.CharField(max_length=256)
home_ml = models.IntegerField(default=999999)
away_ml = models.IntegerField(default=999999)
home_spread = models.FloatField(default=999)
home_spread_odds = models.IntegerField(default=9999)
away_spread_odds = models.IntegerField(default=9999)
total = models.FloatField(default=999)
total_over_odds = models.IntegerField(default=999)
total_under_odds = models.IntegerField(default=999)
updated_datetime = models.DateTimeField(auto_now=True)
views.py:
class GameView(viewsets.ModelViewSet):
queryset = Game.objects.all()
serializer_class = GameSerializer
Thanks

To answer the question in the title:
The instance being passed to the Serializer.to_representation() is the instance you pass when initializing the serializer
queryset = MyModel.objects.all()
Serializer(queryset, many=True)
instance = MyModel.objects.all().first()
Serializer(data)
Usually you don't have to inherit from ListSerializer per se. You can inherit from BaseSerializer and whenever you pass many=True during initialization, it will automatically 'becomeaListSerializer`. You can see this in action here
To answer your problem
from django.db.models import Max
class OddsMakerListSerializer(serializers.ListSerializer):
def to_representation(self, data): # data passed is a queryset of oddsmaker
# Do your filtering here
latest_date = data.aggregate(
latest_date=Max('updated_datetime')
).get('latest_date').date()
latest_records = data.filter(
updated_date_time__year=latest_date.year,
updated_date_time__month=latest_date.month,
updated_date_time__day=latest_date.day
)
return super().to_representation(latest_records)

Modify value in data table UserProfile using function

I am trying to run .save() to change the value of a user model field.
Here is my code:
Views.py:
def traffic_task(request):
tasks_traffic = Task.objects.filter(category="traffic")
random_task = random.choice(tasks_traffic)
task_id = random_task.pk
user = request.user
user.userprofile.daily_task = task_id
user.save()
return task_detail(request=request, pk=task_id)
Models.py
class UserProfile(models.Model):
user = models.OneToOneField(User, on_delete=models.CASCADE)
daily_task = models.IntegerField(default=0)
daily_task_done = models.BooleanField(default=False)
daily_task_done_time = models.DateTimeField(default=datetime.now() - timedelta(days=2))
They are in two different apps so maybe it's an import missing?

You should save the UserProfile object, not the User object, so something like:
def traffic_task(request):
tasks_traffic = Task.objects.filter(category="traffic")
random_task = random.choice(tasks_traffic)
task_id = random_task.pk
userprofile = request.user.userprofile
userprofile.daily_task = task_id
# perhaps you want to set daily_task_done to False here
userprofile.save()
return task_detail(request=request, pk=task_id)
Furthermore based on the code you provide, it looks like you want to add a ForeignKey to Task, it is better not to save the value of the primary key, since the FOREIGN KEY constraints, etc. are not enforced:
class UserProfile(models.Model):
user = models.OneToOneField(User, on_delete=models.CASCADE)
daily_task = models.ForeignKey('someapp.Task', null=True, default=None)
daily_task_done = models.BooleanField(default=False)
daily_task_done_time = models.DateTimeField(default=datetime.now() - timedelta(days=2))
Then you can use a Task object, like:
def traffic_task(request):
tasks_traffic = Task.objects.filter(category="traffic")
random_task = random.choice(tasks_traffic)
userprofile = request.user.userprofile
userprofile.daily_task = random_task
userprofile.save()
return task_detail(request=request, pk=task_id)
This thus creates extra validation, but it is also more convenient to work with the Task object, and in case you want to obtain the Tasks "in bulk", one can use .select_related(..), or .prefetch_related(..) (although one can do this with an IntegerField as well, it will require extra logic, and thus is less elegant).

django duplicates in pagination with order_by() on manytomany field

I am sorting a query on a timefield from a manytomany object but have trouble during pagination.
models.py:
class Validated(models.Model):
job_id_list = models.ManyToManyField(JobId , related_name='JobId', blank=True)
class JobId(models.Model):
user = models.ForeignKey(User, blank=True, null=True, default=None)
job_time = models.DateTimeField(auto_now_add=True)
views.py:
results_list = Validated.objects.filter(job_id_list__user=request.user).\
distinct().order_by('-job_id_list__job_time')
My problem is that pagination is messed up and I get duplicates in my pagination output, even when the query output is ok. (other have this problem too)
I tried several solutions but my issue does not get solved as sorting is bases on a field from a many-to-many object order_by('job_id_list__job_time')
I was thinking of a work-around through creating an annotation to my model by the use of a Model.Manager. Inside this annotation, i try to add the job_time which is a result of a function inside the model. In that way, the job_time would be easily accessible:
for i in result_list:
job_time_from_model = result_list[i].job_time
I would do this as followed but I don't know how to incorporate the %%function logic%% inside the annotation.
Is this even possible in this way or is another approach required?
models.py:
class ValidatedManager(models.Manager):
**%%function-logic%%**
def date(self, user, id):
xlinkdatabase_validated_object = self.get(id=id)
job_id_list = xlinkdatabase_validated_object.\
job_id_list.all().order_by('-job_time')
date_added = None
for item in job_id_list:
if item.user == user:
date_added = item.job_time
break
return date_added
def get_queryset2(self, user):
qs = super(XlinkdatabaseValidatedManager, self).\
get_queryset().annotate(job_date= **%%function-logic%%**, output_field=DateTimeField())
return qs
views.py:
results_list = xlinkdatabase_validated.objects.get_queryset2(request.user).\
filter(job_id_list__user=request.user).distinct().order_by('-job_date')

Django - Query parameter takes model type? RelatedObjectDoesNotExist: Posting has no textbook

I have installed this Django app: http://django-auditlog.readthedocs.io/en/latest/_modules/auditlog/models.html#LogEntry
The log entries are setup with two different models which are below:
class Posting(models.Model):
textbook = models.ForeignKey(Textbook)
condition = models.CharField(max_length=200)
price = models.DecimalField(max_digits=5, decimal_places=2)
user = models.ForeignKey(User)
image = models.ImageField(upload_to='postingpics/%Y/%m/%d', default="/textchange/nophoto.png")
post_date = models.DateTimeField('date_posted')
comments = models.CharField(max_length=50, default="")
def __str__(self):
return str(self.textbook)
def was_posted_recently(self):
return self.post_date >= timezone.now() - datetime.timedelta(days=1)
was_posted_recently.admin_order_field = 'post_date'
was_posted_recently.boolean = True
was_posted_recently.short_description = 'Posted recently'
class Wishlist(models.Model):
textbook = models.ForeignKey(Textbook)
user = models.ForeignKey(User)
wish_date = models.DateTimeField('date_wish')
def __str__(self):
return str(self.textbook)
def was_wished_recently(self):
return self.wish_date >= timezone.now() - datetime.timedelta(days=1)
was_wished_recently.admin_order_field = 'date_wish'
was_wished_recently.boolean = True
was_wished_recently.short_description = 'Wished recently'
auditlog.register(Posting)
auditlog.register(Wishlist)
So in the database the model EntryLog is getting a row each time something happens to Posting or Wishlist whether that be delete, create, or update.
I am trying to complete the query below because I want to count the number of occurrences that a Posting is deleted. Separately I will do the same query for Wishlist.
LogEntry.objects.filter(action=2 , content_type=Posting).count()
content_type is of type Foreign key for the record.
When I run this query I get the error message:
RelatedObjectDoesNotExist: Posting has no textbook.
This makes me think that Posting is the wrong value to put in for content_type. Am I thinking about this correctly? Should the value for content_type be something different? I understand usually when you refer to a Foreign Key you are querying on the Foreign keys fields like Posting__condition or something like that.
Thanks in advance.

The content_type argument should be a ContentType object, not a model class.
Instead of doing doing this manually you should use the LogEntryManagers get_for_model method:
LogEntry.objects.get_for_model(Posting).filter(action=2).count()

Django Rest Framework: Updating / creating multiple objects dynamically, without giving the pk

I just stumbled upon the hardest problem I ever had with Django Rest Framework. Let me give you my models first, and then explain:
class Stampcardformat(models.Model):
workunit = models.ForeignKey(
Workunit,
on_delete=models.CASCADE
)
uuid = models.UUIDField(
default=uuid.uuid4,
editable=False,
unique=True
)
limit = models.PositiveSmallIntegerField(
default=10
)
category = models.CharField(
max_length=255
)
class Stampcard(models.Model):
stampcardformat = models.ForeignKey(
Stampcardformat,
on_delete=models.CASCADE
)
user = models.ForeignKey(
User,
on_delete=models.CASCADE
)
uuid = models.UUIDField(
default=uuid.uuid4,
editable=False,
unique=True
)
class Stamp(models.Model):
stampcardformat = models.ForeignKey(
Stampcardformat,
on_delete=models.CASCADE
)
stampcard = models.ForeignKey(
Stampcard,
on_delete=models.CASCADE,
blank=True,
null=True
)
uuid = models.UUIDField(
default=uuid.uuid4,
editable=False,
unique=True
)
These models describe a simple stampcard model. A stampcard is considered full, when it has as many stamps via foreignkey associated to it as it's stampcardformat's limit number dictates.
I need to write view that does the following:
The view takes in a list of stamps (see below) consisting of their
uuid's.
It then needs to find the right stampcardformat for each given
stamp.
Next it needs to check, whether the requests user has a stampcard
with the corresponding stampcardformat.
a) If it has, it needs to check, if the stampcard is full or not.
i) If it is full, it needs to create a new stampcard of the given format
and update the stamps stampcard-foreignkey to the created stampcard.
ii) If it isn't full, it needs update the stamps stampcard-foreignkey
to the found stampcard
b) If the user hasn't got a stampcard of the given
stampcardformat, it needs to create a new stampcard and update the
stamps stampcard-foreignkey to the created stampcard.
Here is the request body list of stamps:
[
{
"stamp_uuid": "62c4070f-926a-41dd-a5b1-1ddc2afc01b2"
},
{
"stamp_uuid": "4ad6513f-5171-4684-8377-1b00de4d6c87"
},
...
]
The class based views don't seem to support this behaviour. I tried modifying the class based views, to no avail. I fail besides many points, because the view throws the error:
AssertionError: Expected view StampUpdate to be called with a URL keyword argument named "pk". Fix your URL conf, or set the `.lookup_field` attribute on the view correctly.
Edit
For additional context: I need the url to be without pk, slug or anything.
So the url should just be something like:
/api/stampcards/stamps/
and do a put (or any request that has a body and works) to it.
The route I wrote is:
url(r'^stamps/$', StampUpdate.as_view(), name='stamp-api-update'),
Edit:
HUGE update. So I managed to cheese together a view that works.
First I updated the stampcard model like this (I did add anew field 'done' to track if it is full):
class Stampcard(models.Model):
stampcardformat = models.ForeignKey(
Stampcardformat,
on_delete=models.CASCADE
)
user = models.ForeignKey(
User,
on_delete=models.CASCADE
)
uuid = models.UUIDField(
default=uuid.uuid4,
editable=False,
unique=True
)
done = models.BooleanField(default=False)
Then I wrote the view like this:
class StampUpdate(APIView):
permission_classes = (IsAuthenticated,)
def get_object(self, uuid):
try:
return Stamp.objects.get(uuid=uuid)
except Stamp.DoesNotExist():
raise Http404
def put(self, request, format=None):
for stamp_data in request.data:
stamp = self.get_object(stamp_data['stamp_uuid'])
if stamp.stampcard==None:
user_stampcard = self.request.user.stampcard_set.exclude(done=True).filter(stampcardformat=stamp.stampcardformat)
if user_stampcard.exists():
earliest_stampcard = user_stampcard.earliest('timestamp')
stamp.stampcard = earliest_stampcard
stamp.save()
if earliest_stampcard.stamp_set.count() == earliest_stampcard.stampcardformat.limit:
earliest_stampcard.done=True
earliest_stampcard.save()
else:
new_stampcard = Stampcard(stampcardformat=stamp.stampcardformat, user=self.request.user)
new_stampcard.save()
stamp.stampcard = new_stampcard
stamp.save()
new_stampcards = Stampcard.objects.exclude(done=True).filter(user=self.request.user)
last_full_stampcard = Stampcard.objects.filter(user=self.request.user).filter(done=True)
if last_full_stampcard.exists():
last_full_stampcard_uuid=last_full_stampcard.latest('updated').uuid
last_full_stampcard = Stampcard.objects.filter(uuid=last_full_stampcard_uuid)
stampcards = new_stampcards | last_full_stampcard
else:
stampcards = new_stampcards
print(stampcards)
stampcard_serializer = StampcardSerializer(stampcards, many=True)
return Response(stampcard_serializer.data)
But I have two issues with this code:
My intuition tells me that the parts where is just call save() on the model instance (e.g. stamp.save()) are very unsafe for an api. I couldn't get it to work to serialize the data first. My question is: Is this view okay like this? Or can I improve anything? It doesn't use generic class based used for example, but I don't know how to use them here...
I would also love to return the stampcard, if it got filled up by this method. But I also want to exclude all non-relevant stampcards, which is why I called .exclude(done=True). A stampcard that got filled up unfortunately has done=True though! How can I add stampcards that got filled up in the process to the return value?

I don't think it's unsafe to have stamp.save() in PUT method because by definition it supposes to alter object's value.
For returning only relevant stampcards, you could just add stampcard to a set like this
class StampUpdateView(APIView):
def get_object(self, uuid):
try:
return Stamp.objects.get(uuid=uuid)
except Stamp.DoesNotExist():
raise Http404
def put(self, request, *args, **kwargs):
stampcard_set = set()
for stamp_data in request.data:
stamp = self.get_object(stamp_data['stamp_uuid'])
user_stampcard = request.user.stampcard_set.exclude(done=True).filter(stampcardformat=stamp.stampcardformat)
if user_stampcard.exists():
stampcard = user_stampcard.earliest('timestamp')
else:
stampcard = Stampcard(stampcardformat=stamp.stampcardformat, user=request.user)
stampcard.save()
stamp.stampcard = stampcard
stamp.save()
if stampcard.stamp_set.count() == stampcard.stampcardformat.limit:
stampcard.done = True
stampcard.save()
stampcard_set.add(stampcard)
stampcard_serializer = StampcardSerializer(stampcard_set, many=True)
return Response(stampcard_serializer.data)
This way it doesn't matter if returning stampcards are already done or not.
Also note that I move down limit checking lines in your code to be after when stamp was saved because if limit was set to 1, stampcard will have to be set as done immediately after a stamp was added.