Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I'm using Python (3.7.4) to make a text/password manager, where the user can store text under different tabs (using tkinter for the interface) , and use a "master login" to access all data.
The only experience I've got with saving/storing data is using CSV files, and looping through them to get values.
Can anyone recommend anyway I can store text, without it being able to be opened from Windows Explorer, and need some sort of key to be opened?
The natural alternative to using csv files is the use of a database. A solution like sqlite might be enough for your solution. Directly from the documentation:
SQLite is a C library that provides a lightweight disk-based database
that doesn’t require a separate server process and allows accessing
the database using a nonstandard variant of the SQL query language.
Some applications can use SQLite for internal data storage. It’s also
possible to prototype an application using SQLite and then port the
code to a larger database such as PostgreSQL or Oracle.
Once you have learned sqlite, you can think of encrypting your database: in this post you will find many ideas for encrypting your sqlite database.
Otherwise you can switch to other more complete and complex DBMSs. The important thing is that you consider moving from the csv to using a db.
Take a look at SQLLite as a lightweight local database and use something like SQLCipher for strong AES 256-bit encryption.
I havn't read it fully but take a look at this blog for a python implementation.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
I'm currently writing a chat-application for my Website and storing all the Messages in one database, by referring to sender and receiver with a Foreign-key.
Is this really the smartest Idea? I think the site could become slow when everybody is trying to access the same database or when the number of sent messages gets big. Would it be smarter/faster, to use one database per user, storing his/her messages there? If yes, am i right that this is not really possible to achieve this with Django?
Thanks for your Answer!
Would it be smarter/faster, to use one database per user
Probably not, because you will still have the same quantity of data and just have the overhead of have multiple database instances
this is not really possible to achieve this with Django?
Django is not a database manager, so the question does not really make sense. For example, Oracle or SQLite allow a single query to access multiple databases. So if you used Oracle with Django, you could have a different database per user. Anyway, even if by default Django only uses one single database connection, it can be configured to use multiple databases (thanks to #brunodesthuilliers for the info).
Database design is a rather complex operation: you have to design the tables to structure the data, the indexes, to speed up some accesses and/or enforce uniqueness, and optionaly views to easy request writing. Once this is done (and the database is used), you will have to define recurrent tasks to cleanup the database (index maintenance, archivage and purge of archived data).
If you intend to use a large farm of servers to be able to handle thousands of simultaneous accesses, a SQL database can indeed become a bottleneck, and you should considere NoSQL databases to dispatch the load on independant servers. But in this use case, Django would just be a casting error. Anyway, I know no example of real world use case where multiple databases are used just for performance reasons.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I'm currently using a mix of smart view and power query(sql) to load data into Excel models however my excel always crashes when smart view is used. I'm required to work in Excel but I'm know looking at finding a way to periodically load data from Essbase into my SQL server database and only use power query(sql) for all my models. What would be my best options in doing this? Being a Python enthusiast I found essbasepy.py however there isn't much documentation on it. Please help
There are a couple of ways to go. The most straightforward is to export all of the data from your Essbase database using column export, then designing a process to load the data into SQL Server (such as using the import functionality or BULK IMPORT, or SSIS...).
Another approach is to use the DataExport calc script command to export either to a file (that you then load into SQL) or directly to the relational database (DataExport can be configured to export data directly to relational).
In either case, you will need privileges that are greater than normal user privileges, and either approach involves Essbase automation that may require you to coordinate with the Essbase admin.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 years ago.
Improve this question
could you tell me which is the best python driver independent of database platform similar to PDO in PHP or JDBC in Java. Thank you in advance.
I believe "best" it's a matter of your preferences. But I find the SQLAlchemy Core convenient. They support quite some database dialects and offer a ORM layer that's optional to use. It's easy to swap database without making any code change (I'm running SQLite3 in-memory for my test suit but Oracle and Postgres in production). You also get connection pooling and other stuff for free.
Python has a ODBC driver which might fit your needs, especially if you are already familiar with JDBC.
Furthermore, the python standard defines an API for database modules, which might help somewhat to abstract from the actual database implementation as well (I can't really tell how many implementations adhere to this standard as I only ever worked with the sqlite module, but the standard claims 'most').
A third option is mentioned here: using JDBC with Jython.
Which would be the best for you depends on what you actually want to achieve, but from your comparison with JDBC I would suspect the first option using the ODBC driver might be the best for you.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
For my link scraping program (written in python3.3) I want to use a database to store around 100.000 websites:
just the URL,
a time stamp
and for each website a list of several properties
I don't have knowledge about databases, but found the following may fit my purpose:
Postgresql
SQLite
Firebird
I'm interested in speed (to access the database and to get the wanted information). For example: for website x does property y exist and if yes read it. The speed of writing is of course also important.
My question: Are there big differences in speed or does it not matter for my small program? Maybe someone can tell which database fits my requirements (and is easy to handle with Python).
The size and scale of your database is not particularly large, and it's well within the scope of almost any off-the-shelf database solution.
Basically, what you're going to do is install the database server on your machine and it will come up on a given port. You then can install a library in Python to access it.
For example, if you want to use Postgresql, you'll install it on your machine and it will come up attached to some port like 5000, or port 5432.
But if you just have the information you're talking about to store and retrieve, you probably want to go with a NoSQL solution because it's very easy.
For example, you can install mongodb on your server, then install pymongo. The tutorial for pymongo will teach you pretty much everything you need for your application.
If speed is the main criteria, then i would suggest to go with a in-memory database.
Take a look at http://docs.python.org/2/library/sqlite3.html
it can be used as a normal database too, for the in-memory mode use the below and the db should get created in the RAM itself and hence much faster run-time access.
import sqlite3
conn = sqlite3.connect(':memory:')
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I would like to add some interactive capability to a python CLI application I've writen that stores data in a SQLite3 database. Currently, my app reads-in a certain type of file, parses and analyzes, puts the analysis data into the db, and spits the formatted records to stdout (which I generally pipe to a file). There are on the order of a million records in this file. Ideally, I would like to eliminate that text file situation altogether and just loop after that "parse and analyze" part, displaying a screen's worth of records, and allowing the user to page through them and enter some commands that will edit the records. The backend part I know how to do.
Can anyone suggest a good starting point for creating that pager frontend either directly in the console (like the pager "less"), through ncurses, or some other system?
You might want to take a look at urwid. It is a console user interface library for Python. The examples should be more than enough to convince you that this is what you want, if you really want to go text-console UI.
I'd use something like pygtk instead though.
After looking around a bit, I found that less and other pagers actually use curses. When I thought of curses I always imagined the blue-boxed interface with menus and mouse interaction. These are library add-ons for curses, which offer exactly the basic terminal selection and editing control functionality I'm looking for.
Tutorial on Python Curses Programming
Curses Programming with Python
On the backend, when the user attempts to move the cursor above or below the currently displayed records, I'll have sqlite fetch me the next appropriate set of records for display.