I spinup postgres container and its data path /var/lib/postgresql/data is mapped to my local using volumes. As soon as container is up and database is setup the local path populates with all db data. I need to some how check programatically (using Python) if local location is proper postgres db data. This is needed if I need to create tables or not. I create if local directory is blank or invalid postgres data and I don't if it is valid. The reason I am trying to achieve this is if I want to hook up local db created due to postgers_container_1 to postgres_container_2
If the file /var/lib/postgresql/data/PG_VERSION exists, then it's probably a valid data directory. This is the first thing Postgres will check when you try to start the server.
Of course, there are many, many other things required to make it a valid data directory - too many to check by yourself. If you need to be 100% sure, the only practical way is to start the Postgres server and try to connect to it.
Related
I am using an embedded monetdb database in python using Monetdbe.
I can see how to create a new connection with the :memory: setting
But i cant see a way to persist the created database and tables for use later.
Once an in memory session ends, all data is lost.
So i have two questions:
Is there a way to persist an in memory db to local disk
and
Once an in memory db has been saved to local disk, is it possible to load the db to memory at a later point to allow fast data analytics. At the moment it looks like if i create a connection from a file location, then my queries are reading from local disk rather memory.
It is a little bit hidden away admittedly, but you can check out the following code snipet from the movies.py example in the monetdbe-examples repository:
import monetdbe
database = '/tmp/movies.mdbe'
with monetdbe.connect(database) as conn:
conn.set_autocommit(True)
conn.execute(
"""CREATE TABLE Movies
(id SERIAL, title TEXT NOT NULL, "year" INTEGER NOT NULL)""")
So in this example the single argument to connect is just the desired path to your database directory. This is how you can (re)start a database that stores its data in a persistent way on a file system.
Notice that I have intentionally removed the python lines from the example in the actual repo that start with the comment # Removes the database if it already exists. Just to make the example in the answer persistent.
I haven't run the code but I expect that if you run this code twice consecutively the second run wil return a database error on the execute statement as the movies table should already be there.
And just to be sure, don't use the /tmp directory if you want your data to persist between restarts of your computer.
hey I'm currently working on a website (Photo selling services) and now I wanna deploy it on a public host,
I didn't change the database and I'm using Django's SQLite as my database, Is it gonna be a problem or it's fine?
and also I'm handling the downloads with my views and template and the files (photos) will be downloaded from my database and I wanted to know do I need one host for my application and another for putting my photos in? or I can just run the whole website on one host without a problem ( same as I'm running it on my local host).
I prefer not to use SQLite in production because:
It is just a single file and it may get deleted suddenly by anyone that has access to that.
No user management.
Limited data types.
For serving files on a heavy-traffic website it's good to have CDN to serve files from.
Currently trying to use pg_dump and pg_restore to be able to dump select rows from a production server to a testing server. The goal is to have a testing server and database that contains the subset of data selected, moreover through a python script, I want the ability to restore the database that original subset after testing and potentially modifying the contents of the database.
From my understanding of pg_dump and pg_restore, the databases that they interact with must be of the same dbname. Moreover, a selection criteria should be made with a the COPY command. Hence, my idea is to have 2 databases in my production server, one with the large set of data and one with the selected set. Then, name the smaller set db 'test' and restore it to the 'test' db in the test server.
Is there a better way to do this considering I don't want to keep the secondary db in my production server and will need to potentially make changes to the selected subset in the future.
From my understanding of pg_dump and pg_restore, the databases that they interact with must be of the same dbname.
The databases being worked with only have to have the same name if you are using --create. Otherwise each programs operates in whatever database was specified when it was invoked, which can be different.
The rest of your question is too vague to be addressable. Maybe pg_dump/pg_restore are the wrong tools for this, and just using COPY...TO and COPY...FROM would be more suitable.
I use python and SQL-server to manage a database, but I do not know "good practices" about database management and know few about security information.
Is it secure to save Database credentials in Windows as a environment variable and use it into scripts with os.environ? Like this:
import os
DB_HOST = os.environ['DBHOST']
DB_USER = os.environ['DBUSER']
...
How is the proper way to store credentials to automate uses of databases?
If you are asking if you should permanently set environment variables for your laptop - I’d avoid that because any process could list all environment variables on the PC and the associated stored values quite easily.
Instead - I’d recommend checking out Keyring. This will use the Windows Credential Locker (or other OS specific keyring services).
Usually secure credentials are stored in a .env file that relates to your current environment and then are grabbed from within your code. E.g DB_HOST = env('DBHOST').
Basically what you're doing right now but stored in a file (as secure as you need it, possibly encrypted) rather than directly as environment variables as they're accessible from the entire machine.
By using Encryptedbypassphrase('key','Your_Password') method in sqlserver,
Example,
create table #temp(id int identity(1,1),Password varbinary(max))
insert into #temp(Password) values(encryptbypassphrase('12','Passw0rd'))
select * from #temp
In that code we are provide the original password but it stored in the database
table by encrypted value.
Screenshot of my output:
I'm building a web application in Pyramid and it needs user logins. Database backend is a MySQL DB connected via SQLAalchemy.
Pyramid has an introduction on using beaker for sessions, but it only shows how to configure it using files. I couldn't find out how to store session data in the database (I think it should be possible), since then I would have only one place were my varying data is stored.
I found it. Put something like this in your configuration file (development.ini/production.ini)
session.type=ext:database
session.secret=someThingReallyReallySecret
session.cookie_expires=true
session.key=WhatEver
session.url=mysql://user:password#host/database
session.timeout=3000
session.lock_dir=%(here)s/var/lock
I don't is if it is possible (or sensible) to put locking to the DB, too, but the sessions should live in the DB like this. You'll need to take care to delete old sessions from the DB yourself (but I think that's the case when using files, too).