I am in the process of migrating my python application from Google App Engine to Heroku. I have managed to get webapp2 (google app engine's default request handler) working.
What I am experiencing an issue with, is the database. In GAE, I was using the built in Datastore, and my application contains all the database syntax according to that. But after migrating to Heroku, I have a lot of database choices as "addons".
I wanted to know, as to which database I should choose in Heroku, which resembles the Google App engine's datastore the most, so that I have to change the least amount of code in my Application.
Just as a reference, here's the link showing the vast amount of options available on Heroku for database choices: Heroku Database Addons.
Something like Cassandra: https://addons.heroku.com/cassandraio would be closest in performance to the GAE datastore given it's column-structured nature.
However, depending on your usage patterns and how close you stuck to the Django-like API, any of the standard datastores (Postgres, ClearDB MySQL) would be about the same amount of effort to switch over to from a code perspective.
Related
So, I am working on a project that has a backend optimization algorithm written in GAMS.
The ideal solution is to be able to use this GAMS code that is already written with the python API for GAMS. I want to be able to call this code through an HTTP request and run the algorithm, so I wanted to make a Flask server for this. Ideally, it could run in Google's App Engine, but GAMS software must be installed.
I am not sure if this is possible on App Engine, or if it can be done in a Google instance.
The data input would be from CSV's in google cloud storage and the output would be put there as well.
I was wondering if anyone has tried this before or if you know more about Google cloud and think this will or will not work. I could not find much about this online.
Your question would benefit from more detail and including references e.g. Is this GAMS?
App Engine standard is opinionated and it may not (don't know) be possible for you to bundle GAMS as part of a Python deployment to App Engine standard.
However, if you were willing to bundle a Python (Flask) server and GAMS together, it's possible you could containerize the solution and run it on App Engine flexible as a custom runtime.
Alternatives exist and may be better matches to your needs including detaching the (Flask) frontend server from the GAMS backend (?) and deploying these on App Engine (or possibly Cloud using Functions, Cloud Run, Compute Engine etc.).
Someone who has done this may be willing to provide additional pointers|guidance.
The best solution to this problem, I found, was to get rid of GAMS entirely.
There were many ways I wished to integrate this model into my web app, so I translated the model to Python using PyOMO. The performance was slightly affected, but the model can now be run in Google Cloud.
I set up the model in a Flask server and now built a REST API around it for use in the web app.
I have a web application written in Python-Flask with database created using MySQL. I have deployed my app using Heroku but it's only the frontend and backend part. I also need to upload database so that the app can interact with the database. I have searched many videos on YouTube but nothing came as useful. Most of them are using Postgre SQL both in local and heroku app but I've an MySQL database with me. I need help how can I connect my app to my database.
Heroku has native support for postgessql not mysql, so you might want to change to that or use the Clear DB addon as explained in the docs.
From heroku
"Heroku provides three managed data services to all customers:
Heroku Postgres, Heroku Redis, Apache Kafka on Heroku"
Also have a look at this previous answer that targets the same issue.
I'm trying to figure out how to force a flush of the GAE datastore to disk in the dev appserver.
I understand that some time ago writes were disabled until dev_appserver shutdown from here:
App Engine local datastore content does not persist
My app is running 24/7 doing things and generating data, and I do not want to have to restart things from zero if the VM hosting my dev_appserver goes down or something. So, I'd like to schedule a periodic flush of the datastore.
I found this method for 1.6.4:
GAE SDK 1.6.4 dev_appserver datastore flush
That doesn't work anymore, and I haven't found a variant using the API server that I am permitted to invoke from my app. Anyone know how to flush to disk on demand in 1.9?
No, dev_appserver is not designed to be used in production and provides no reliability or durability guarantees.
Ideally you'd be using the service on GCP, however if for some reason that is not possible you can look at the open source project AppScale that is designed for feature parity of Google App Engine / Cloud Datastore.
I want to deploy an application with sqlite3 as the database on Heroku. However, it seems to be that Heroku doesn't support applications with sqlite3 as the database. Is it true? Is there no way to deploy my sqlite3-backed application on Heroku?
PS: I have successfully deployed my application using PythonAnywhere, but would now like to know whether there's any possible way to deploy it using Heroku.
As Heroku's dynos don't have a filesystem that persists across deploys, a file-based database like SQLite3 isn't going to be suitable. It's a great DB for development/quick prototypes, though.
Heroku do have a Postgres offering however that will suit - with a free tier and a basic $9/month tier that are good for hobby/small projects. The biggest benefit over SQLite is that you get backups that you wouldn't get otherwise (plus all the other Postgres features).
There's a guide to updating your settings.py to use Postgres here: https://devcenter.heroku.com/articles/getting-started-with-django#django-settings
Heroku has a detailed article explaining "Why is SQLite a bad fit for running on Heroku" https://devcenter.heroku.com/articles/sqlite3
I am looking for a simple python web framework which runs both as standalone and on Appengine.
I would like to write the app in such a way that i can switch between standalone and Appengine based on the deployment configuration. Standalone will be using RDBMS.
When i checked web.py looked simple and promising. I have the following questions:
Does web.py work well on Appengine?
Is there any major known gotchas?
Can you share your experience with using web.py on Appengine?
Any suggestions to achieve the above mentioned goal is appreciated.
Does web.py work well on Appengine?
Yes it does work pretty well, it's compact and very easy to use.
Is there any major known gotchas?
Web.py does not offer any Data Abstraction Layer compatible with GAE; that means that in order to switch from a RDBMS to a NoSQL database, you have to create by yourself a layer above both the web.db database API and the Google App Engine datastore API.
Can you share your experience with using web.py on Appengine?
I have deployed a couple of simple applications with web.py on top of GAE and other several pet projects with Sqlite as database; here a mini review.
Any suggestions to achieve the above mentioned goal is appreciated.
If the switch from Appengine to RDBMS is your primary concern, have a look to web2py.
Although with some limitations, to the best of my knowledge is the only one Python web framework GAE friendly that offers a DAL above the App Engine Datastore and other several RDBMS solutions.
Switching from one db to another is just a matter of initializing the DAL with the proper connection string:
db = DAL('sqlite://storage.db')
db = DAL('mysql://username:password#localhost/test')
db = DAL('gae')
No webapp framework on its own is going to be able to do this - the App Engine environment consists of a lot more than just a database interface and a CGI environment. Even if it could, any DAL that supports both the datastore and a relational DB is likely to sacrifice most of the flexibility of both in pursuit of that.
Your best option, if you want to take advantage of more than just the datastore, is to write your app for App Engine, and use AppScale or TyphoonAE to run your app outside the App Engine environment.
Consider using webapp2. It is similar to web.py, provides basically the same featureset, and runs outside of App Engine out-of-the-box. Besides, you get stellar compatibility with App Engine SDK and libraries (I explain it better here).
You'd only need to add a relational database library. But this is an easy choice: SQLAlqchemy is pretty much the standard in Python land, with lots of documentation and a thriving community.
I would also suggest web2py. Great framework, great community, plays out-of-the-box outside of AppEngine. It also supports a lot of databases (SQLite, PostgreSQL, MySQL, MSSQL, FireBird, Oracle, IBM DB2, Informix, Ingres, and Google App Engine).
On top of that, it's trivial to install it on Linux, Windows, Mac.