When creating an mlflow tracking server and specifying that a SQL Server database is to be used as a backend store, mlflow creates a bunch of table within the dbo schema. Does anyone know if it is possible to specify a different schema in which to create these tables?
It is possible to alter mlflow/mlflow/store/sqlalchemy_store.py to change the schema of the tables that are stored.
It is very likely that this is the wrong solution for you, since you will go out of sync with the open source and lose newer features that alter this, unless you maintain the fork yourself. Could you maybe reply with your use case?
You can use postgres uri options:
Postgres URI options sample:
"postgresql://postgres:postgres#localhost:5432/postgres?options=-csearch_path%3Ddbo,mlflow_schema"
In your Mlflow Code:
mlflow.set_tracking_uri("postgresql://postgres:postgres#localhost:5432/postgres?options=-csearch_path%3Ddbo,mlflow_schema")
! Don't forget to create 'mlflow_schema' schema.
how-to-specify-schema-in-psycopg2-connection-method
I'm using MSSQLServer as the backend store. I could use a different schema than dbo by specifying the default schema for the SQLServer user being used by MLFlow.
In my case, if the MLFlow tables (e.g: experiences) exist in dbo, then those tables will be used. If not, MLFlow will create those tables in the default schema.
Related
I have a bigquery table about 200 rows, i need to insert,delete and update values in this through a web interface(the table cannot be migrated to any other relational or non-relational database).
The web application will be deployed in google-cloud on app-engine and the user who acts as admin and owner privileges on Bigquery will be able to create and delete records and the other users with view permissions on the dataset in bigquery will be able to view records only.
I am planning to use the scripting language as python,
server(django or flask or any other)-> not sure which one is better
The web application should be displayed as a data-grid like appearance with buttons create,delete or view visiblility according to their roles.
I have not done anything like this in python,bigquery and django. I am already familiar with calling bigquery from python-client but to call in a web interface and in a transactional way, i am totally new.
I am seeing examples only related to django with their inbuilt model and not with big-query.
Can anyone please help me and clarify whether this is possible to implement and how?
I was able to achieve all of "C R U D" on Bigquery with the help of SQLAlchemy, though I had make a lot of concessions like if i use sqlalchemy class i needed to use a false primary key as Bigquery does not use any primary key and for storing sessions i needed to use file based session On Django for updates and create sqlalchemy does not allow without primary key, so i used raw sql part of SqlAlchemy. Thanks to the #mhawke who provided the hint for me to carry out this exericse
No, at most you could achieve the "R" of "CRUD." BigQuery isn't a transactional database, it's for querying vast amounts of data and preparing the results as an immutable view.
It doesn't provide a method to modify the source data directly and even if you did you'd need to run the query again. Also important to note are that queries are asynchronous and require much longer to perform than traditional databases.
The only reasonable solution would be to export the table data to GCS and then import it into a normal database for querying. Alternatively if you can't use another database and since you said there are only 1,000 rows you could perform your CRUD actions directly on that exported CSV.
Is it possible to create a new database from pony ORM? We couldn't find it within pony ORM docs, it's always assumed the database exists or is using SQLite file.
We would like to create a testing database and drop it afterwards.
No. Per:
https://docs.ponyorm.org/api_reference.html#sqlite
Supported databases
If you look at the .bind() API for the various databases, SQLite is the only one with create_db. This is because in SQLite creating a database is just creating a single file. The other engines need to go through their own program to initialize a database. You will need to create an independent script that creates the database.
If you have your sqlite database file, you can try using pgloader.
When using an SQLite database where does Django store the database that it uses when running tests?
Is there way to define this path?
I would like to be able to manually look at the contents of the test database following each test.
The command python manage.py test --keepdb is supposed to keep the database, which it does, but I cannot seem to find where this database is stored.
The dev database is stored in the root of the project but the test database is not found there.
from the docs:
When using SQLite, the tests will use an in-memory database by default (i.e., the database will be created in memory, bypassing the filesystem entirely!). The TEST dictionary in DATABASES offers a number of settings to configure your test database. For example, if you want to use a different database name, specify NAME in the TEST dictionary for any given database in DATABASES.
for more info see:
https://docs.djangoproject.com/en/2.1/topics/testing/overview/#the-test-database
I'm using a shared postgres instance which hosts data from multiple groups, managing permissions via postgres schemas. Effectively, this means all of my tables need to be prefixed with my schema name like so:
my_schema.my_table
For the results backend of Celery, I found this doc which mentions that database_table_names may be used to set the name of the tables used in the results backend.
Is it possible to use this setting to set the broker table names too?
Looking at the code, it doesn't appear to be possible. But, maybe there's another way to do this.
I am trying to use sqlalchemy to connect with mysql database. I have set up charset=utf-8$use_unicode=0. This worked with almost all databases, but not with a particular one. I believe it is because it has 'init-connect' variable set to 'SET NAMES latin2;' I have no privileges to change that.
It works for me if I send explicit query SET NAMES utf8, however if there is a temporal disconnection, then after reconnecting my program breaks again as it gets lati2-encoded data from the server.
Is it possible to create some hook to always send the SET NAMES when sqlalchemy connects? Or any other way to solve this problem?
Sounds like what you want is a custom PoolListener. This SO answer explains how to write one in the context of SQLite's PRAGMA foreign_keys=ON
Sqlite / SQLAlchemy: how to enforce Foreign Keys?