Given the following models,
class Server(BaseModel):
name = peewee.CharField(unique=True)
class Member(BaseModel):
name = peewee.CharField(unique=True)
server = peewee.ForeignKeyField(Server, related_name='members')
and a dictionary with keys being Server names and values being tuples of Member names,
data = {
'Server01': ('Laurence', 'Rose'),
'Server02': ('Rose', 'Chris'),
'Server03': ('Isaac',)
}
what is the fastest way of bulk inserting Members using peewee? It seems like one should use Model.insert_many() here, but since Member.server expects a Server or Server.id, that would require iterating over data.items() and selecting a Server for each name.
for server_name, member_names in data.items():
server = Server.select().where(Server.name == server_name)
member_data = []
for name in member_names:
member_data.append({'name': name, 'server': server})
with db.atomic():
Member.insert_many(member_data)
Needless to say, this is terribly inefficient. Is there a better way of doing it?
Well, if you don't know ahead of time which servers are present in the DB it seems like your problem is the data-structure you're using. Keeping server_name -> member_names in a dict like that and trying to insert it all in one go is not how relational databases work.
Try this:
server_to_id = {}
for server_name in data:
if server_name not in server_to_id:
server = Server.create(name=server_name)
server_to_id[server_name] = server.id
for server_name, member_names in data.items():
server_id = server_to_id[server_name]
member_data = [{'name': name, 'server': server_id} for name in member_names]
Member.insert_many(member_data).execute()
Note: don't forget to call .execute() when using insert() or insert_many().
Related
There are environment variables set in the operating system (macos):
MYSQL_HOST="127.0.0.1"
MYSQL_DATABASE="invoice"
MYSQL_UID="dude"
MYSQL_PWD="pass1234"
I would like to build a list called db_config such that the end result will look like:
db_config = {'host':"127.0.0.1", 'database':"invoice", 'user':"dude",
'password':"pass1234"}
(Note that the environment variable names differ from the keys in db_config. db_config will be used to pass database connection credentials, and the keys must be those listed in the above db_config.)
I can "manually" set db_config using:
db_config={'host':os.environ['MYSQL_HOST'], 'database':os.environ['MYSQL_DATABASE'],
'user':os.environ['MYSQL_UID'], 'password':os.environ['MYSQL_PWD']}
...but it seems like there should be a cleaner more pythonic way of doing this, but I can't figure it out.
config = {
'host': 'MYSQL_HOST',
'database': 'MYSQL_DATABASE',
'user': 'MYSQL_UID',
'password': 'MYSQL_PWD'
}
db_config = {k: os.environ.get(v) for k, v in config.items()}
Depending on how you want to treat items that aren't in os.environ, you can use a conditional dict comprehension to ignore them.
db_config = {k: os.environ.get(v) for k, v in config.items()
if v in os.environ}
all ENV variable related to MYSQL in one dict
import os
keys = dict(os.environ).keys()
dic = {}
for key in keys:
if 'MYSQL_' in key:
dic.update({key.split('MYSQL_')[1]:os.environ.get(key)})
print(dic)
You may wish to rename old_name and new_name in the code below; and if you are sure that the environment variables will be present or require them to be available for your code to work correctly, you can remove the if section of the dictionary comprehension shown below:
import os
transformations = (
('MYSQL_HOST', 'host'),
('MYSQL_DATABASE', 'database'),
('MYSQL_UID', 'user'),
('MYSQL_PWD', 'password')
)
db_config = {
new_name: os.environ[old_name]
for old_name, new_name in transformations
if old_name in os.environ
}
I am trying to update 'host record name', and for achieving the same I am using infoblox-client but I am hitting a roadblock coz of which m not able to modify/update the host record. I have tried the same via REST API and m able to modify the name as required but via infoblox-client m kind of stuck.
Following are the steps that m trying to update the record:
opts = {'host': '192.168.1.10', 'username': 'admin', 'password': 'admin'}
conn = connector.Connector(opts)
my_ip = objects.IP.create(ip='192.168.1.25')
#to create new host record with parameters
record = objects.HostRecord.create(conn, view='my_dns_view', name='my_host_record.my_zone.com', ip=my_ip)
#to modify m trying following but no luck:
#1_both of below snippet adds a new name to record instead of upodating the same
-> result = objects.HostRecord.create(conn, view='my_dns_view', name='new_name.my_zone.com', ip=my_ip, check_if_exists=True, update_if_exists=True)
-> result = objects.ARecordBase.create(conn, ip='192.168.102.14', name='some-new_name.ansible.com', view='default', update_if_exists=True)
#2_I have gone through infoblox-client test case(test_update_fields_on_create) which I have following steps to modify record:
a_record = [{'_ref': 'record:previously_created_record', 'ip': '192.168.1.52', 'name': 'other_name'}]
#this gives me error for host,username missing.
conn = connector.Connector(a_record)
#this results to None which ultimately blocks me
res = conn.get_object('record:host',a_record)
I was able to resolve the issue, got mixed up with creating and updating.
So, for any one who face similar kind of issue here is the coded solution:
opts = {'host': '192.168.1.10', 'username': 'admin', 'password': 'admin'}
conn = connector.Connector(opts)
my_ip = objects.IP.create(ip='192.168.1.25')
#to create new host record with parameters
record = objects.HostRecord.create(conn, view='my_dns_view', name='host_record.my_zone.com', ip=my_ip)
#to update the name, directly use update_object:
testConn = conn.update_object(record._ref, {'name': 'new_name_host_record.my_zone.com'})
Simple.. I was beating around the bush earlier..
I have two models Storage and Drawers
class Storage(BaseModel):
id = PrimaryKeyField()
name = CharField()
description = CharField(null=True)
class Drawer(BaseModel):
id = PrimaryKeyField()
name = CharField()
storage = ForeignKeyField(Storage, related_name="drawers")
at the moment I'm producing json from a select query
storages = Storage.select()
As a result I have got a json array, which looks like this:
[{
description: null,
id: 1,
name: "Storage"
},
{
description: null,
id: 2,
name: "Storage 2"
}]
I know, that peewee allows to query for all drawers with storage.drawer(). But I'm struggling to include a json array to every storage which contains all drawers of that storage. I tried to use a join
storages = Storage.select(Storage, Drawer)
.join(Drawer)
.where(Drawer.storage == Storage.id)
.group_by(Storage.id)
But I just retrieve the second storage which does have drawers, but the array of drawers is not included. Is this even possible with joins? Or do I need to iterate over every storage retrieve the drawers and append them to the storage?
This is the classic O(n) query problem for ORMs. The documentation goes into some detail on various ways to approach the problem.
For this case, you will probably want prefetch(). Instead of O(n) queries, it will execute O(k) queries, one for each table involved (so 2 in your case).
storages = Storage.select().order_by(Storage.name)
drawers = Drawer.select().order_by(Drawer.name)
query = prefetch(storages, drawers)
To serialize this, we'll iterate through the Storage objects returned by prefetch. The associated drawers will have been pre-populated using the Drawer.storage foreign key's related_name + '_prefetch' (drawers_prefetch):
accum = []
for storage in query:
data = {'name': storage.name, 'description': storage.description}
data['drawers'] = [{'name': drawer.name}
for drawer in storage.drawers_prefetch]
accum.append(data)
To make this even easier you can use the playhouse.shortcuts.model_to_dict helper:
accum = []
for storage in query:
accum.append(model_to_dict(storage, backrefs=True, recurse=True))
I have a django application that is utilizing a third party API and needs to receive several arguments such as client_id, user_id etc. I currently have these values labeled at the top of my file as variables, but I'd like to store them in an object instead.
My current set up looks something like this:
user_id = 'ID HERE'
client_id = 'ID HERE'
api_key = 'ID HERE'
class Social(LayoutView, TemplateView):
def grab_data(self):
authenticate_user = AuthenticateService(client_id, user_id)
I want the default values set up as an object
SERVICE_CONFIG = {
'user_id': 'ID HERE',
'client_id': 'ID HERE'
}
So that I can access them in my classes like so:
authenticate_user = AuthenticateService(SERVICE_CONFIG.client_id, SERVICE_CONFIG.user_id)
I've tried SERVICE_CONFIG.client_id, and SERVICE_CONFIG['client_id'], as well as setting up the values as a mixin but I can't figure out how to access them any other way.
Python is not Javascript. That's a dictionary, not an object. You access dictionaries using the item syntax, not the attribute syntax:
AuthenticateService(SERVICE_CONFIG['client_id'], SERVICE_CONFIG['user_id'])
You can use a class, an instance, or a function object to store data as properties:
class ServiceConfig:
user_id = 1
client_id = 2
ServiceConfig.user_id # => 1
service_config = ServiceConfig()
service_config.user_id # => 1
service_config = lambda:0
service_config.user_id = 1
service_config.client_id = 2
service_config.user_id # => 1
Normally using a dict is the simplest way to store data, but in some cases higher readability of property access can be preferred, then you can use the examples above. Using a lambda is the easiest way but more confusing for someone reading your code, therefore the first two approaches are preferable.
I am using RestKit for iOS to perform a POST to my Python (Flask) server. The POST arguments is a set of nested dictionaries. When I create my hierarchical argument object on the client side and perform the post, there are no errors. But on the server side, the form data is flattened into a single set of keys which are themselves indexed strings:
#interface TestArgs : NSObject
#property (strong, nonatomic) NSDictionary *a;
#property (strong, nonatomic) NSDictionary *b;
#end
RKObjectMapping *requestMapping = [RKObjectMapping requestMapping]; // objectClass == NSMutableDictionary
[requestMapping addAttributeMappingsFromArray:#[
#"a.name",
#"a.address",
#"a.gender",
#"b.name",
#"b.address",
#"b.gender",
]];
RKRequestDescriptor *requestDescriptor = [RKRequestDescriptor requestDescriptorWithMapping:requestMapping objectClass:[TestArgs class] rootKeyPath:nil];
RKObjectManager *manager = [RKObjectManager managerWithBaseURL:[NSURL URLWithString:#"http://localhost:5000"]];
[manager addRequestDescriptor:requestDescriptor];
NSDictionary *a = [NSDictionary dictionaryWithObjectsAndKeys:
#"Alexis", #"name",
#"Boston", #"address",
#"female", #"gender",
nil];
NSDictionary *b = [NSDictionary dictionaryWithObjectsAndKeys:
#"Chris", #"name",
#"Boston", #"address",
#"male", #"gender",
nil];
TestArgs *tArgs = [[TestArgs alloc] init];
tArgs.a = a;
tArgs.b = b;
[manager postObject:tArgs path:#"/login" parameters:nil success:nil failure:nil];
On the server side, the POST body is this:
{'b[gender]': u'male', 'a[gender]': u'female', 'b[name]': u'Chris', 'a[name]': u'Alexis', 'b[address]': u'Boston', 'a[address]': u'Boston'}
When what I really want is this:
{'b': {'gender' : u'male', 'name': u'Chris', 'address': u'Boston'}, 'a': {'gender': u'female', 'name': u'Alexis', 'address': u'Boston'}}
Why is the POST body not maintaining its hierarchy on the server side? Is this an error with my client side encoding logic? On the server side with Flask decoding the JSON? Any ideas?
Thanks
The error is with the client side mapping. Your mapping needs to represent the structure of the data you want and the relationships it contains. Currently the mapping uses keypaths which effectively hide the structural relationships.
You need 2 mappings:
The dictionary of parameters
The container of the dictionaries
The mappings are defined as:
paramMapping = [RKObjectMapping requestMapping];
containerMapping = [RKObjectMapping requestMapping];
[paramMapping addAttributeMappingsFromArray:#[
#"name",
#"address",
#"gender",
]];
RKRelationshipMapping *aRelationship = [RKRelationshipMapping
relationshipMappingFromKeyPath:#"a"
toKeyPath:#"a"
withMapping:paramMapping];
RKRelationshipMapping *bRelationship = [RKRelationshipMapping
relationshipMappingFromKeyPath:#"b"
toKeyPath:#"b"
withMapping:paramMapping]
[containerMapping addPropertyMapping:aRelationship];
[containerMapping addPropertyMapping:bRelationship];
Then your request descriptor is defined using the container mapping.