I've got a Python program with about a dozen classes, with several classes possessing instances of other classes, e.g. ObjectA has a list of ObjectB's, and a dictionary of (ObjectC, ObjectD) pairs.
My goal is to put the program's functionality on a website.
I've written and tested JSON encode and decode methods for each class. The problem as I see it now is that I need to choose between starting over and writing the models and logic afresh from a database perspective, or simply storing the python objects (encoded as JSON) in the database, and pulling out the saved states for changes.
Can someone confirm that these are both valid approaches, and that I'm not missing any other simple options?
Man, what I think you can do is convert the classes you already have made into django model classes. Of course, only the ones that need to be saved to a database. The other classes, as the rest of the code, I recommend you to encapsulate them for use as helper functions. So you don't have to change too much your code and it's going to work fine. ;D
Or, another choice, that can be easier to implement is: put everything in a helper, the classes, the functions and everything else.
SO you'll just need to call the functions in your views and define models to save your data into the database.
Your idea of saving the objects as JSON on the database works, but it's ugly. ;)
Anyway, if you are in a hurry to deliver the website, anything is valid. Just remember that things made in this way always give us lots of problems in the future.
It hopes that it could be useful! :D
Related
I'm writing a Discord bot that, among many other things, must engage in question-answer dialogues with users. There may be multiple of the same kind of dialogue ("script") running at once asynchronously.
On program startup, a YAML file is read which contains scripts (not in the programming sense), which in turn contain lists of questions, each of which contain the question text. Both scripts and questions contain accessory attributes such as regex formulas, rejection responses, and display names.
Currently, ChatBotManager contains a list of Script objects which in turn contain Question objects, all of which can error-check the relevant portion of the YAML they are passed on initialization. Since the YAML is user-editable, this initialization is non-trivial, and should throw critical-exceptions when passed bad data.
After all this initialization, I have a very nice hierarchical structure with "has a" rather than "is a" relationships, and each level of data can contain methods relevant to it.
The question is this: which of the below ways of recording user responses is most pythonic, expandable, and readable?
Record user responses in a separate, partially-mirrored datastructure and refer to the Script and contained Question objects as read-only
Record user responses in Question objects themselves, then use some way to ensure that data does not contaminate parallel, identical chatbots. Both kinds of objects can have methods relevant to only their function, and other functions need little knowledge of the contents.
Note: When finally processing the responses, I need access to accessory attributes of each question.
I'm not asking this because I am stumped and can't progress: the first option is clearly within my knowledge. I want to know best practice. The second option embraces decoupling and clearly delineated responsibilities, but I don't know how to implement it properly. Perhaps my Googling for a solution lacked the right terminology?
I don't want to re-read the YAML file on every object's creation, nor run the validation code each time. Reading and validation need only happen on bot startup. I'd prefer to keep validation methods in the object that will store the validated data.
One idea I had is to deep-copy a Script object on each chatbot creation, then discard it on completion. I've read this is an expensive operation, and while my program isn't low on resources, it seems bad practice.
Should each Question be able to generate a Response object that contains the response and the data about the question that is needed later? These could then be compiled in a ScriptResponse object generated by the Script which also contains information about the script.
I have a small program with a command line interface that uses a number of python classes with thorough implementations. I want to scrap the command line interface and wrap the app within a Django app, but I'm just learning Django and I'm unfamiliar with the conventions.
I have a number of classes, in-memory storage structures, getters/setters etc and I'd like to convert them into Django models so that I can persist them to the database and interact with them around the django app. Is there a general approach for doing something like this?
Should I just inherit the django.db.models.Model class in my existing classes and set them up for direct interaction? Or is there a better, more general/conventional way to do this?
I would like to be able to use all of this code in other apps, not necesarilly Django ones, so I don't really want to modify my existing classes in a way that would make them only work with Django. I thought of creating the models separately and then a sort of middle-man class to manage interaction of the actual in-memory class with the django model class, but that just seems like more places I have to make changes when I extend/modify the code.
Thanks for any help ahead of time...
Personally, I would modify your existing classes to extend models.Model and maintain separate versions of these classes for use outside of Django.
This will keep your classes lean and maintainable within their respective environments.
You could also create a new class that extends both models.Model and your python model through multiple inheritance. However this will result in duplicate fields for the same data.
If you would like, post an example Model as a new question and tag me in a link to it here, and I can help you convert it.
One of greatest django strengths is its ORM, if you want import i recommend you use it, and yes you would probably need rewrite the part that interacts with the database, but if you already have isolated this functions in a Models folder~classes, the modification won't be really hard
Although in your case i would recommending checking out Tornado/Aiohttp Since looks like you are just trying to create a interface for your functions
I'm developing a framework of sorts. I'm providing a base class, that will be subclassed by other developers to add behavior to the system. The instances of those classes will have attributes that my framework doesn't necessarily expect, except by inspecting those instances' __dict__. To make things even more interesting, some of those classes can be created dynamically, at any time.
I'd like some things to be handled by the framework, namely, I will need to persist those instances, display their attribute values to the user, and let her search/filter instances using those values.
I have to use a relational database. I know there are some decent python OO database out there, but unfortunately they're not an option in this case.
I'm not looking for a full-blown ORM too... and it may not even be an option, given that some of the classes can be created dynamically.
So, my question is, what state of a python instance do I need to serialize to ensure that I can deserialize it later on? Is it enough to look at __dict__, or are there other private attributes that I should be using?
Pickling the instances is not enough, because I'll need to unpickle them to search/filter the attribute values, and I'm afraid it's too much data to do it in-memory (instead of letting the database do it).
Just use an ORM. This is what they are for.
What you are proposing to do is create your own half-assed ORM on your own time. Save your time for your own code that does things, and use the effort other people put for free into solving this problem for you.
Note that all class creation in python is "dynamic" - this is not an issue, for, well, anything at all. In fact, if you are assembling classes programmatically, it is probably slightly easier with an ORM, because they provide reifications of fields.
In the worst case, if you really do need to store your objects in a fake nosql-type schema, you will still only have to write your own backend driver if you use an existing ORM, rather than coding the whole stack yourself. (As it happens, you're not the first person to face this - solutions exist. Goole "python orm store dynamically created models" and "sqlalchemy store dynamically created models")
Candidates include:
Django ORM
SQLAlchemy
Some others you can find by googling "Python ORM".
I've noticed I have the same piece of code sitting at the top of several of my controllers. They tend to look like this:
def app_description(app):
""" Dictionary describing an app. """
return {'name': app.app,
'id': app.id,
'is_new': app.is_new(),
'created_on': app.created_on.strftime("%m/%d/%Y"),
'configured': app.configured }
I'll call this from a couple different actions in the controller, but generally not outside that controller. It accesses properties. It calls methods. It formats opaque objects (like dates).
My question is: is this controller code, or model code?
The case for controller:
It defines my API.
It's currently only used in that module.
There doesn't seem to be any logic here.
The case for model:
It seems like a description of the data, which the model should be responsible for.
It feels like I might want to use this in other controllers. Haven't gotten there yet, but these functions are still pretty new, so they might.
Attaching a function to the object it clearly belongs to seems better than leaving it as a module-level function.
It could be more succinctly defined on the model. Something like having the top-level model object define .description(), and the subclasses just define a black/whitelist of properties, plus override the method itself to call functions. I'm pretty sure that would be fewer lines of code (as it would save me the repetition of things like 'name': app.name), which seems like a good thing.
Not sure which framework you are using, but I would suggest creating this helper functionality in its own class and put it in a shared folder like lib/
Alternatively you could have an application helper module that just has a bunch of these helpful application-wide functions.
Either way, I'd keep it away from both the model and the controller.
The answer I finally decided on:
In the short term, having these methods is fine in the controllers. If they define the output, then, OK, they can stay there. They're only used in the model.
Theres a couple things to watch out for, which indicate they've grown up, and need to go elsewhere:
In one case, I needed access to a canonical serialization of the object. At that point, it moved into the model, as a model method.
In another case, I found that I was formatting all timestamps the same. I have a standard #ajaxify decorator that does things like sets Content-Type headers, does JSON encoding, etc. In this case, I moved the datetime standard formatting into there -- when the JSON encoder hits a datetime (formerly unserializable), it always treats it the same (seconds since the epoch, for me).
In yet a third case, I realized that I was re-using this function in a couple controllers. For that, I pulled it out into a common class (like another answer suggested) and used that to define my "Web API". I'd used this pattern before -- it makes sense for grouping similarly-used data (like timeseries data, or top-N lists).
I suspect there's more, but basically, I don't think there all as similar as I thought they were initially. I'm currently happy thinking about them as a convention for simple objects in our (small-ish, new-ish) codebase, with the understanding that after a few iterations, a better solution may present itself. In the meantime, they stay in the controller and define my AJAXy-JSON-only interface.
surfing on the web, reading about django dev best practices points to use pickled model fields with extreme caution.
But in a real life example, where would you use a PickledObjectField, to solve what specific problems?
We have a system of social-networks "backends" which do some generic stuff like "post message", "get status", "get friends" etc. The link between each backend class and user is django model, which keeps user, backend name and credentials. Now imagine how many auth systems are there: oauth, plain passwords, facebook's obscure js stuff etc. This is where JSONField shines, we keep all backend-specif auth data in a dictionary on this model, which is stored in db as json, we can put anything into it no problem.
You would use it to store... almost-arbitrary Python objects. In general there's little reason to use it; JSON is safer and more portable.
You can definitely substitute a PickledObjectField with JSON and some extra logic to create an object out of the JSON. At the end of the day, your use case, when considering to use a PickledObjectField or JSON+logic, is serializing a Python object into your database. If you can trust the data in the Python object, and know that it will always be serialize-able, you can reasonably use the PickledObjectField. In my case (I don't use django's ORM, but this should still apply), I have a couple different object types that can go into my PickledObjectField, and their definitions are constantly mutating. Rather than constantly updating my JSON parsing logic to create an object out of JSON values, I simply use a PickledObjectField to just store the different objects, and then later retrieve them in perfectly usable form (calling their functions). Caveat: If you store an object via PickledObjectField, then you change the object definition, and then you retrieve the object, the old object may have trouble fitting into the new object's definition (depending on what you changed).
The problems to be solved are the efficiency and the convenience of defining and handling a complex object consisting of many parts.
You can turn each part type into a Model and connect them via ForeignKeys.
Or you can turn each part type into a class, dictionary, list, tuple, enum or whathaveyou to your liking and use PickledObjectField to store and retrieve the whole beast in one step.
That approach makes sense if you will never manipulate parts individually, only the complex object as a whole.
Real life example
In my application there are RQdef objects that represent essentially a type with a certain basic structure (if you are curious what they mean, look here).
RQdefs consist of several Aspects and some fixed attributes.
Aspects consist of one or more Facets and some fixed attributes.
Facets consist of two or more Levels and some fixed attributes.
Levels consist of a few fixed attributes.
Overall, a typical RQdef will have about 20-40 parts.
An RQdef is always completely constructed in a single step before it is stored in the database and it is henceforth never modified, only read (but read frequently).
PickledObjectField is more convenient and much more efficient for this purpose than would be a set of four models and 20-40 objects for each RQdef.