Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
I'm currently writing a chat-application for my Website and storing all the Messages in one database, by referring to sender and receiver with a Foreign-key.
Is this really the smartest Idea? I think the site could become slow when everybody is trying to access the same database or when the number of sent messages gets big. Would it be smarter/faster, to use one database per user, storing his/her messages there? If yes, am i right that this is not really possible to achieve this with Django?
Thanks for your Answer!
Would it be smarter/faster, to use one database per user
Probably not, because you will still have the same quantity of data and just have the overhead of have multiple database instances
this is not really possible to achieve this with Django?
Django is not a database manager, so the question does not really make sense. For example, Oracle or SQLite allow a single query to access multiple databases. So if you used Oracle with Django, you could have a different database per user. Anyway, even if by default Django only uses one single database connection, it can be configured to use multiple databases (thanks to #brunodesthuilliers for the info).
Database design is a rather complex operation: you have to design the tables to structure the data, the indexes, to speed up some accesses and/or enforce uniqueness, and optionaly views to easy request writing. Once this is done (and the database is used), you will have to define recurrent tasks to cleanup the database (index maintenance, archivage and purge of archived data).
If you intend to use a large farm of servers to be able to handle thousands of simultaneous accesses, a SQL database can indeed become a bottleneck, and you should considere NoSQL databases to dispatch the load on independant servers. But in this use case, Django would just be a casting error. Anyway, I know no example of real world use case where multiple databases are used just for performance reasons.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
TL DR: should you use graylog instead mariadb as database for a moderate amount of data? Would this be a good idea?
Longversion:
There is a python script which puts some nmap monitoring data (maybe 1000 to 10000 rows) into a maria database each day. It is using sql alchemy, a flask website is using the same datatypes to display the results and everything is working fine. The model is not too complicated, 3 Tables with a one to many relation.
ip range -> hosts -> ports (with results)
Now my boss wants to put everything into graylog with a python binding and to abadon sql alchemy and the mariadb completly.
My question is: would you recommend to keep the existing structure and just export each night everything from the mariadb into the graylog server (the data is collected just once a night).
Replace the database / mysql alchemy and use Graylog directly? I have never worked with Graylog and I am not sure if you can use it as a database replacement.
Graylog is a log agregator like Splunk its not a relational database itself. From what you describe you are using the MariaDB Database like a logging utility for this data.
Questions to ask yourself are, do you have other logs you should be aggregating and collecting or is it just this one niche set of data?
If you have multiple sources then Graylog is one answer but if its just this subset then its probably overkill.
For context I use Splunk to aggregate over 50GB of logs per day, they come from servers, networking gear, firewalls and more. I'd never think to pour that data into a SQL Database, its not the right tool for the job.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I'm using Python (3.7.4) to make a text/password manager, where the user can store text under different tabs (using tkinter for the interface) , and use a "master login" to access all data.
The only experience I've got with saving/storing data is using CSV files, and looping through them to get values.
Can anyone recommend anyway I can store text, without it being able to be opened from Windows Explorer, and need some sort of key to be opened?
The natural alternative to using csv files is the use of a database. A solution like sqlite might be enough for your solution. Directly from the documentation:
SQLite is a C library that provides a lightweight disk-based database
that doesn’t require a separate server process and allows accessing
the database using a nonstandard variant of the SQL query language.
Some applications can use SQLite for internal data storage. It’s also
possible to prototype an application using SQLite and then port the
code to a larger database such as PostgreSQL or Oracle.
Once you have learned sqlite, you can think of encrypting your database: in this post you will find many ideas for encrypting your sqlite database.
Otherwise you can switch to other more complete and complex DBMSs. The important thing is that you consider moving from the csv to using a db.
Take a look at SQLLite as a lightweight local database and use something like SQLCipher for strong AES 256-bit encryption.
I havn't read it fully but take a look at this blog for a python implementation.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
I wonder if a good habit is to have some logic in MySQL database (triggers etc.) instead of logic in Django backend. I'm aware of fact that some functionalities may be done both in backend and in database but I would like to do it in accordance with good practices. I'm not sure I should do some things manually or maybe whole database should be generated by Django (is it possible)? What are the best rules to do it as well as possible? I would like to know the opinion of experienced people.
It is true that if you used a database for your business logic you could get maximum possible performance and security optimizations. However, you would also risk many things such as
No separation of concerns
Being bound to the database vendor
etc.
Also, whatever logic you write in your database won't be version controlled with your app. Thus, whenever you change your database, you will have to create all those things once again.
Instead, use Django ORM. It will create and manage your database based on your models by itself. As a result, whenever you recreate your database, you will just have to run migrations with one single command and you are done.
This will cover most of the situations. And whenever you will need those speeds of stored procedures, Django ORM has you covered as well.
In short, I believe that business logic should be kept out of the database as much as possible.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
Being a newbie programmer, most of the programs I've written never saved or loaded any of their data, and the few that did were saved by serializing the classes and saving it in raw text files or binary forms.
I am now in the process of learning Python with Django and I have a fundamental lack of understanding how things work behind the scenes. In all of the programs I've written, I know that if I have a class A and the class holds a linked-list data member list then the list exists in the memory (heap/stack).
Now assume I write a very large Django application and I have 10,000 instances of A, where exactly are they saved? How do I control them? I guess holding a list of 10,000 instances isn't rational, but how do I manage what is being loaded to the memory of the application, and what is being accessed directly through the database?
I hope I'm clear, being a newbie I don't know the right definitions that describes that things I mean and makes it hard to communicate so please feel free to edit and correct me.
I'm gonna divert from what you are asking to recommend reading about data storage, data structures, tables, and such before using an ORM, such as the one provided with django.
If I understand correctly, you don't mean memory management, but persistence of the data. How is data stored in nowadays business apps? Mostly in databases.
You cand find hundreds of tutorials on this matter, and based on the Relational Database Modelling Systems Django supports, I would start by learning relational databases.
I found this article to be pretty "straightforward"
Feel free to ask any questions if you don't understand the concepts. Once you get a grasp of how databases work, you will understand what does Django do and how your models persist over time.
It seems like your question is directed towards memory management, such as accessing the actual memory space in which objects are saved. Generally this is avoided, as python should handle this all for you, and it is not critical that you as a programmer need to manipulate the actual memory space in which your objects are stored. See Accessing object memory address
Django supports the use of various databases such as SQLite, MySQL, and PostgreSQL; all of which have different back-end operations.
It may be worth your time exploring the Django documentation on databases if you have not done so already: https://docs.djangoproject.com/en/dev/ref/databases/
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
For my link scraping program (written in python3.3) I want to use a database to store around 100.000 websites:
just the URL,
a time stamp
and for each website a list of several properties
I don't have knowledge about databases, but found the following may fit my purpose:
Postgresql
SQLite
Firebird
I'm interested in speed (to access the database and to get the wanted information). For example: for website x does property y exist and if yes read it. The speed of writing is of course also important.
My question: Are there big differences in speed or does it not matter for my small program? Maybe someone can tell which database fits my requirements (and is easy to handle with Python).
The size and scale of your database is not particularly large, and it's well within the scope of almost any off-the-shelf database solution.
Basically, what you're going to do is install the database server on your machine and it will come up on a given port. You then can install a library in Python to access it.
For example, if you want to use Postgresql, you'll install it on your machine and it will come up attached to some port like 5000, or port 5432.
But if you just have the information you're talking about to store and retrieve, you probably want to go with a NoSQL solution because it's very easy.
For example, you can install mongodb on your server, then install pymongo. The tutorial for pymongo will teach you pretty much everything you need for your application.
If speed is the main criteria, then i would suggest to go with a in-memory database.
Take a look at http://docs.python.org/2/library/sqlite3.html
it can be used as a normal database too, for the in-memory mode use the below and the db should get created in the RAM itself and hence much faster run-time access.
import sqlite3
conn = sqlite3.connect(':memory:')