I have built an an app with Flask, without using SQLAlchemy (so that I don't get an ORM, which for me give less transparency, and understanding), so just plain SQL with SQLite.
Now that I want to publish it online (now I want to test it with a free Heroku), all the tutorials I find seem to use SQLAlchemy to be able to use PostgreSQL instead of SQLite.
I am also using a schema.sql file containing all the code to create the schema (CREATE TABLE user, etc.) which is launched with the command flask init-db.
I am also a bit worried that I will need to change way more code than simply the database link.
Using the db.py code that is in the Flask tutorial, I changed the connect URL to use the one which Heroku gave me.
def get_db():
if 'db' not in g:
if os.environ.get('DATABASE_URL') is None:
g.db = sqlite3.connect(
current_app.config['DATABASE'],
detect_types=sqlite3.PARSE_DECLTYPES
)
g.db.row_factory = sqlite3.Row
else:
DATABASE_URL = os.environ['DATABASE_URL']
g.db = psycopg2.connect(DATABASE_URL, sslmode='require')
return g.db
When pushing the code to Heroku with the initial commit, on the URL of my website I get a "Application error" message. And logs say:
2019-08-21T15:12:50.333541+00:00 app[init.1]: Usage: flask [OPTIONS] COMMAND [ARGS]...
2019-08-21T15:12:50.333564+00:00 app[init.1]: Try "flask --help" for help.
2019-08-21T15:12:50.333567+00:00 app[init.1]:
2019-08-21T15:12:50.333642+00:00 app[init.1]: Error: No such command "init_db".
2019-08-21T15:12:50.434989+00:00 heroku[init.1]: Process exited with status 2
2019-08-21T15:12:54.909027+00:00 heroku[init.1]: Starting process with command `FLASK_APP=myapp flask init_db`
2019-08-21T15:12:55.504819+00:00 heroku[init.1]: State changed from starting to up
2019-08-21T15:12:57.997833+00:00 heroku[init.1]: State changed from up to crashed
2019-08-21T15:12:57.888599+00:00 app[init.1]: Usage: flask [OPTIONS] COMMAND [ARGS]...
2019-08-21T15:12:57.888627+00:00 app[init.1]: Try "flask --help" for help.
2019-08-21T15:12:57.888630+00:00 app[init.1]:
2019-08-21T15:12:57.888639+00:00 app[init.1]: Error: No such command "init_db".
2019-08-21T15:12:57.978596+00:00 heroku[init.1]: Process exited with status 2
So can I easily deploy a Flask app to Heroku without using SQLAlchemy, or at least not an ORM, so that I can use my plain SQL commands?
You're conflating SQLAlchemy and PostgreSQL.
SQLAlchemy provides
a unified, relatively low-level interface for working with many different database backends via SQLAlchemy Core, and
an optional ORM
It can be used with both PostgreSQL and SQLite (and others).
PostgreSQL is a relational database management system, much like SQLite is.
You can use Postgres directly, e.g. with psycopg2, or with something like SQLAlchemy.
You can't use SQLite on Heroku, with or without SQLAlchemy. SQLite saves its data in a file on the filesystem, and this doesn't play nicely with Heroku's ephemeral filesystem.
To use Postgres without SQLAlchemy, simply
provision the add-on (this should happen automatically if you add psycopg2 to your dependencies),
and connect to it via the connection string given in the DATABASE_URL environment variable.
I am also using a schema.sql file containing all the code to create the schema (CREATE TABLE user, etc.) which is launched with the command flask init-db.
As long as the schema.sql is part of your application you should be able to do
heroku run flask init-db
If that doesn't work you should be able to do something like
cat schema.sql | heroku pg:psql
though personally I'd dig into why heroku run isn't doing what you want, first.
Finally, I strongly recommend switching to PostgreSQL in development as well.
SQLite and PostgreSQL are different products and don't always behave the same way. Some measure of uniformity can be achieved through an ORM or SQLAlchemy's Core (though even then I recommend using the same database everywhere), but if you're writing raw SQL the differences become more important.
Related
I am in the process of migrating a Python flask web app from Heroku to Railway. The app uses a set of Flask CLI commands to initialize and update a Postgres database schema.
In the development environment, for example, I would run "flask db migrate" in order to create a db model to be copied to other locations.
In order to copy the model into staging or production environments, I would then need to run the command "flask db upgrade" on the staging/production app. Heroku's CLI "run" command allows you to do this without SSH by running the following command:
heroku run flask db upgrade --app NAME-OF-STAGING/PRODUCTION-APP
As far as I know, Railway's CLI run command does not allow you to select an online environment on which to run the command in the same way. Nor do they allow SSH access.
Is there any way to run a CLI command or Python file on a Railway live app, so that I can run Flask-migrate's "db upgrade" and get my Postgres DB working?
If you're still looking for an answer, here is what I did: I didn't use flask migrate, but instead I downloaded my database as a CSV file and created my own migration script. The script reads through the CSV file row by row and adds the data to the new database (you need to delete the old one). I called the script from my main.py, and then I removed the script after the database was migrated. This is definitely not the best way to do this, especially if your DB has a lot of data in it, but it's a good workaround.
I need help switching my database engine from sqlite to mysql. manage.py datadump is returning the same error that pops up when I try to do anything else with manage.py : ImproperlyConfigured: Error loading MySQL module, No module named MySQLdb.
This django project is a team project. I pulled new changes from bitbucket and our backend has a new configuration. This new configuration needs mysql (and not sqlite) to work. Our lead dev is sleeping right now. I need help so I can get started working again.
Edit: How will I get the data in the sqlite database file into the new MySQL Database?
According to the Database setup paragraph in Django docs:
By default, the configuration uses SQLite.
If you wish to use another database, install the appropriate database
bindings...
And Get your database running page says:
If you’re using MySQL, you’ll need the MySQL-python package, version
1.2.1p2 or higher.
To use MySQL backend you need a tool that would help Django talk to the database, an adapter.
In other words, MySQL-python (aka MySQLdb) package needs to be installed.
Try the followings steps:
1. Change DATABASES in settings.py to MYSQL engine
2. Run $ ./manage.py syncdb
I am trying to install an app in Heroku cloud. This python based app is built on postgresql. I have set up this app in my local machine(Ubuntu). I have following queries.
How to install postgresql for this app on heroku?
Do I have to make same set up again on Heroku as I did on local machine? Like setting up of Flask etc.?
Can I submit code in many stages? e.g I have pushed code, after some time modified same code and pushed again.
I have already gone through following url https://devcenter.heroku.com/articles/git#deploying-code but didn't got any satisfactory answer.
Thanks
Sanjeev Yadav
Assuming you have heroku belt installed on your computer, create git repository inside your project(which you probably already have) then create Heroku app:
heroku create --stack cedar <name_of_your_app>
First add Postgresql to your project:
heroku addons:add heroku-postgresql
after that check if it has been added succesfully:
heroku pg:info
This should return something like:
HEROKU_POSTGRESQL_WHITE_URL <== Database name
Plan: Dev
Status: available
Then promote the database so heroku knows which one you will be using (you can have several databases in one project).
heroku pg:promote HEROKU_POSTGRESQL_WHITE_URL
At this point if you want to configure your database connections you can either get all the connection info from Heroku page of your project or from environment variable DATABASE_URL which was created when you promoted the database. If you use the latter you don't have to provide any info yourself, the function that i use frequently is something in this lines:
if "DATABASE_URL" in os.environ:
urlparse.uses_netloc.append("postgres")
url = urlparse.urlparse(os.environ.get("DATABASE_URL"))
DATABASE = {
"database" : url.path[1:],
"user" : url.username,
"password": url.password,
"host": url.hostname,
"port": url.port
}
return PostgresqlDatabase(**DATABASE)
This is for peewee ORM , but you can adapt it to any other ORM like SQLAlchemy, just take what's in DATABASE and plug it in to whatever you want.
A far as setting up your application, you need to write requirements.txt file with all dependencies that your app uses, this will automatically be parsed and all libraries installed when you first push your project to Heroku, also you need Procfile file inside your project, this tells Heroku which processes to run at which stage, the most important part is web part for instance:
web: gunicorn main:app
will start gunicorn server serving your site. If you need to run any scripts, for instance script that creates the tables in your database simply type:
heroku run create_tables.py
from within your repository.
As far as last question goes - yes you can push your code in many stages, your app will be restarted and all new code should work same as on your local machine. Hope this helps.
I'm trying to setup a simple Flask app using a postgresql database. I'm connecting to the database using psycopg2. Psycopg2 expect a dsn string for the connection, something like this :
dbname=mydb user=myuser password=1234 host=myhost.com
but heroku gives me, in my app config, a database url like this :
postgres://myuser:mypassword#myhost.com/mydb
I cannot directly change those config variables and I didn't find any way to specify their format in the heroku postgres dashboard.
Maybe there's a way to process it in python?
I had a Django site running on Dreamhost. Although I used SQLite when developing locally, I initially used MySQL on Dreamhost, because that’s what the wiki page said to do, and because if I’m using an ORM, I might as well take advantage of it by running against a different database.
After a while, I switched the settings on the server to use SQLite, to make it easier to keep my development database in sync with the server one. python manage.py syncdb worked on the server, but when I tried to access the site, I got an OperationalError. The Django error page said that one of my tables didn’t exist.
I checked the database using sqlite on the command line on the server, and using python manage.py shell, and both worked fine.
It turned out that on the server, the DATABASE_NAME setting required a full path, e.g.
DATABASE_NAME = '/home/USERNAME/SITE/DJANGOPROJECT/DATABASE.db'
Locally (and I guess for manage.py on the server), just a filename was fine, e.g.
DATABASE_NAME = 'DATABASE.db'