The following is my model:
I have two tables match and team:
class Match(DictModel):
date_time = db.DateTimeProperty()
team1 = db.StringProperty()
team2 = db.StringProperty()
venue = db.StringProperty()
result = db.IntegerProperty()
class Team(DictModel):
tslug = db.StringProperty()
name = db.StringProperty()
matches_played = db.IntegerProperty()
matches_won = db.IntegerProperty()
rating = db.FloatProperty()
At runtime, when a post request is made to one of the handler functions, i want to dynamically associate a team rating with the queryset of Match and send the value, this is how i try to do:
matches = Match.all()
matches.filter('date_time <=', end)
matches.filter('date_time >=', start)
match_dict = functs.create_dict(matches)
self.response.out.write(match_dict)
and i have a custom function to get fetch the rating from the current team, it is as follows:
def to_dict(self):
return dict([(p, unicode(getattr(self, p))) for p in self.properties()])
def create_dict(matches):
lst = []
for m in matches:
t1 = m.team1
t2 = m.team2
te1 = Team.all().filter("name =", t1).get()
te2 = Team.all().filter("name =", t2).get()
m.setattr('rating1', te1.rating)
m.setattr('rating2', te2.rating)
lst.append(m)
data_dict = json.dumps([l.to_dict() for l in lst])
return data_dict
Trouble: i get error in setattr in place of setattr i also tried m.rating1 = te1 and m.rating2 = te2 but even that does not seem to work. Everything else is working flawlessly.
Please help thanks!
The syntax is setattr(m, 'rating1', te1.rating1); but this is no different from m.rating1 = te1.rating1.
May we see the trace back?
Related
I'm on peewee 3.14.0 ... is it a valid use case to do something like this? This is ofc a toy example of my actual use-case:
class A(BaseModel):
id = BigAutoField(primary_key=True)
a1 = CharField()
class B(BaseModel):
a = ForeignKeyField(A, backref='bs')
b1 = CharField()
and then do something like:
a = A.select()...
for b in a.bs:
b.b1 = "doesn't work, this will insert a new record rather than update the existing"
b.save()
to cover this update use-case I need to do the following instead:
a = A.select()...
for b in a.bs:
q = (B.update({b1: "this however, works"}).where(B.a==a, ..))
q.execute()
Is there a way to fix the first approach or is it not covered by peewee?
It does not insert new records in this example. I think your example is either missing some important information or something else is missing.
class User(Base):
username = TextField()
class Tweet(Base):
user = ForeignKeyField(User, backref='tweets')
content = TextField()
db.create_tables([User, Tweet])
u = User.create(username='u1')
for j in range(2):
Tweet.create(user=u, content='t%s' % j)
u = User.get(User.username == 'u1')
for tweet in u.tweets:
tweet.content = tweet.content + '-x'
tweet.save()
for tweet in u.tweets:
print(tweet.content)
This correctly prints:
t0-x
t1-x
I have a function which search for json files in a directory, parse the file and write data in the database. My problem is writing in database, because it take around 30 minutes. Any idea how can I speed up writting in a database? I have few quite big files to parse, but parsing the file is not a problem it take around 3 minutes. Currently I am using sqlite but in the future I will change it to PostgreSQL.
Here is my function:
def create_database():
with transaction.atomic():
directory = os.fsencode('data/web_files/unzip')
for file in os.listdir(directory):
filename = os.fsdecode(file)
with open('data/web_files/unzip/{}'.format(filename.strip()), encoding="utf8") as f:
data = json.load(f)
cve_items = data['CVE_Items']
for i in range(len(cve_items)):
database_object = DataNist()
try:
impact = cve_items[i]['impact']['baseMetricV2']
database_object.severity = impact['severity']
database_object.exp_score = impact['exploitabilityScore']
database_object.impact_score = impact['impactScore']
database_object.cvss_score = impact['cvssV2']['baseScore']
except KeyError:
database_object.severity = ''
database_object.exp_score = ''
database_object.impact_score = ''
database_object.cvss_score = ''
for vendor_data in cve_items[i]['cve']['affects']['vendor']['vendor_data']:
database_object.vendor_name = vendor_data['vendor_name']
for description_data in cve_items[i]['cve']['description']['description_data']:
database_object.description = description_data['value']
for product_data in vendor_data['product']['product_data']:
database_object.product_name = product_data['product_name']
database_object.save()
for version_data in product_data['version']['version_data']:
if version_data['version_value'] != '-':
database_object.versions_set.create(version=version_data['version_value'])
My models.py:
class DataNist(models.Model):
vendor_name = models.CharField(max_length=100)
product_name = models.CharField(max_length=100)
description = models.TextField()
date = models.DateTimeField(default=timezone.now)
severity = models.CharField(max_length=10)
exp_score = models.IntegerField()
impact_score = models.IntegerField()
cvss_score = models.IntegerField()
def __str__(self):
return self.vendor_name + "-" + self.product_name
class Versions(models.Model):
data = models.ForeignKey(DataNist, on_delete=models.CASCADE)
version = models.CharField(max_length=50)
def __str__(self):
return self.version
I will appreciate if you can give me any advice how can I improve my code.
Okay, given the structure of the data, something like this might work for you.
This is standalone code aside from that .objects.bulk_create() call; as commented in the code, the two classes defined would actually be models within your Django app.
(By the way, you probably want to save the CVE ID as an unique field too.)
Your original code had the misassumption that every "leaf entry" in the affected version data would have the same vendor, which may not be true. That's why the model structure here has a separate product-version model that has vendor, product and version fields. (If you wanted to optimize things a little, you might deduplicate the AffectedProductVersions even across DataNists (which, as an aside, is not a perfect name for a model)).
And of course, as you had already done in your original code, the importing should be run within a transaction (transaction.atomic()).
Hope this helps.
import json
import os
import types
class DataNist(types.SimpleNamespace): # this would actually be a model
severity = ""
exp_score = ""
impact_score = ""
cvss_score = ""
def save(self):
pass
class AffectedProductVersion(types.SimpleNamespace): # this too
# (foreign key to DataNist here)
vendor_name = ""
product_name = ""
version_value = ""
def import_item(item):
database_object = DataNist()
try:
impact = item["impact"]["baseMetricV2"]
except KeyError: # no impact object available
pass
else:
database_object.severity = impact.get("severity", "")
database_object.exp_score = impact.get("exploitabilityScore", "")
database_object.impact_score = impact.get("impactScore", "")
if "cvssV2" in impact:
database_object.cvss_score = impact["cvssV2"]["baseScore"]
for description_data in item["cve"]["description"]["description_data"]:
database_object.description = description_data["value"]
break # only grab the first description
database_object.save() # save the base object
affected_versions = []
for vendor_data in item["cve"]["affects"]["vendor"]["vendor_data"]:
for product_data in vendor_data["product"]["product_data"]:
for version_data in product_data["version"]["version_data"]:
affected_versions.append(
AffectedProductVersion(
data_nist=database_object,
vendor_name=vendor_data["vendor_name"],
product_name=product_data["product_name"],
version_name=version_data["version_value"],
)
)
AffectedProductVersion.objects.bulk_create(
affected_versions
) # save all the version information
return database_object # in case the caller needs it
with open("nvdcve-1.0-2019.json") as infp:
data = json.load(infp)
for item in data["CVE_Items"]:
import_item(item)
This code only creates one record. What is wrong?
class PartnerTagCreate(models.TransientModel):
""" Choose tags to be added to partner."""
_name = 'partner.tags.create'
_description = __doc__
market_id = fields.Many2one('partner.tags', string='Market Tag')
application_id = fields.Many2one('partner.tags', string='Application Tag')
partner_id = fields.Integer()
#api.multi
def create_contact_tag(self):
for record in self.env['sale.order.line'].browse(self._context.get('active_ids', [])):
vals = {}
vals['partner_id'] = record.order_partner_id
self.write(vals)
return True
I need this function to create one record for each order_partner_id I selected before opening the wizard...
How to achieve that?
Here my new code (function) ...
def create_contact_tag(self):
sale_order_line_ids = self.env['sale.order.line'].browse(self._context.get('active_ids', []))
for partner in sale_order_line_ids:
values = {}
values['partner_id'] = partner.order_partner_id
self.create(values)
return {}
This creates one record for marketing_id and/or application_id and dedicated records for each partner_id in the record.
You use the 'create' method to create new records; this is the same for TransientModel as for the persistent Model.
So, replace
self.write(vals)
by
self.create(vals)
and you should be fine.
I have a legacy Google App engine code, which is having the following entity classes in Python
class AffiliateParent(db.Model):
name = db.StringProperty(required = True)
class Affiliate(db.Model):
email = db.StringProperty(required = True)
point_gain = db.IntegerProperty()
point_used = db.IntegerProperty()
#feature_upgrade = db.ListProperty(str)
modified_time = db.DateTimeProperty(auto_now = True)
I was wondering, is the following the correct way to generate unique id? Are they guarantee unique within Affiliate table?
affiliate_parent_key = AffiliateParent.all(keys_only=True).filter('name =', 'yancheng').get();
affiliate_parent = db.get(affiliate_parent_key);
# check whether affiliate exist, if not create one
affiliate_parent = db.get(affiliate_parent_key);
q = Affiliate.all()
q.ancestor(affiliate_parent)
q.filter('email =', email)
affiliate = q.get()
if not affiliate:
affiliate = Affiliate(
email = email,
point_gain = 0,
point_used = 0,
parent = affiliate_parent
)
affiliate.put()
# 4503602445942784
# is this unique?
unique_id = affiliate.key().id()
yes, if you don't supply id or key when instantiating the model, then datastore will generate a unique ID and assign it to your entity when you .put() it... thus affiliate.key.id() will be unique
you can also generate unique IDs using allocate_ids(count)
https://cloud.google.com/appengine/docs/python/datastore/functions#allocate_ids
I have 4 models:
class TransitLine(models.Model):
name = models.CharField(max_length=32)
class Stop(models.Model):
line = models.ForeignKey(TransitLine, related_name='stops')
class ExitType(models.Model):
name = models.CharField(max_length=32)
people_limit = models.PositiveSmallIntegerField()
class Exits(models.Model):
TOKEN_BOOTH = 0
GATE = 1
REVOLVING_DOOR = 2
EXIT_TYPES = (
(TOKEN_BOOTH, 'Token booth'),
(GATE, 'Gate'),
(REVOLVING_DOOR, 'Revolving door'),
)
name = models.CharField(max_length=32)
stop = models.ForeignKey(Stop, related_name='exits')
type = models.ForeignKey(ExitType, related_name='exits')
I have one TransitLine object. I want to retrieve all the unique ExitType objects that are related to the Stop objects of the TransitLine (that was a mouth full).
Some semi-pseudo code of what I want to do:
tl = TransitLine.objects.get(id=1)
exit_types = []
for s in tl.stops:
exit_types.append([e.type for e in s.exits])
unique_exit_types = list(set(exit_types))
Obviously, prefer to do this in one QuerySet call. Any suggestions?
I would try something like this:
ExitType.objects.filter(exits__stop__line=line).distinct()