I'm trying to transfer an Openshift-hosted Django website to Google App Engine. I wanted to connect it to an existing PostgreSQL database. However, I found this in Google's documentation:
A small percentage of native C python modules, and subsets of native C python modules are not available with Google App Engine. The disabled modules fall in to the following categories: (...)
Please keep in mind that third party packages which use any of the above features will not function with Google App Engine (packages such as PostgreSQL, etc).
My local GAE server produced the following error:
ImproperlyConfigured: Error loading psycopg2 module: No module named psycopg2._psycopg
Further research seems to indicate that this error was raised due to the limitations of GAE, rather than due to a configuration error. Surely there must be some reasonable way to connect my Django website with the externally hosted PostgreSQL server. What are my options?
Edit: so to clarify, I think I need a pure-Python replacement for psycopg2 that works with Django. In other words, it should also implement a Django database backend I can use.
You might be able to use pg8000.
Related
So, I am working on a project that has a backend optimization algorithm written in GAMS.
The ideal solution is to be able to use this GAMS code that is already written with the python API for GAMS. I want to be able to call this code through an HTTP request and run the algorithm, so I wanted to make a Flask server for this. Ideally, it could run in Google's App Engine, but GAMS software must be installed.
I am not sure if this is possible on App Engine, or if it can be done in a Google instance.
The data input would be from CSV's in google cloud storage and the output would be put there as well.
I was wondering if anyone has tried this before or if you know more about Google cloud and think this will or will not work. I could not find much about this online.
Your question would benefit from more detail and including references e.g. Is this GAMS?
App Engine standard is opinionated and it may not (don't know) be possible for you to bundle GAMS as part of a Python deployment to App Engine standard.
However, if you were willing to bundle a Python (Flask) server and GAMS together, it's possible you could containerize the solution and run it on App Engine flexible as a custom runtime.
Alternatives exist and may be better matches to your needs including detaching the (Flask) frontend server from the GAMS backend (?) and deploying these on App Engine (or possibly Cloud using Functions, Cloud Run, Compute Engine etc.).
Someone who has done this may be willing to provide additional pointers|guidance.
The best solution to this problem, I found, was to get rid of GAMS entirely.
There were many ways I wished to integrate this model into my web app, so I translated the model to Python using PyOMO. The performance was slightly affected, but the model can now be run in Google Cloud.
I set up the model in a Flask server and now built a REST API around it for use in the web app.
Im making a webpage through google app engine that querys based on the person needs from an existing SQL database located on a local server. Im having trouble importing third party python libraries because most of them are not written in native python code (google app engine only supports fully python code). I have searched everywere for a way to connect to this database but I cant find a way around this issue.
You can use the MySQLdb library, which is a built-in library for App Engine. See the set up instructions in the docs for details.
Specify the MySQLdb library in the app.yaml libraries element:
libraries:
- name: MySQLdb
version: "latest"
What #BrettJ said, some of the common third-party libs for python are not pure python, and thus cannot be imported into the project unless google whitelists them end enables them to be imported in a safe manner, basically, what you need to do is to have MySQLdb library set in the app.yaml file.
libraries:
- name: MySQLdb
version: "latest"
I highly suggest you go through several tutorials, this is basic stuff and will be cleared to you in minimum time.
I am looking for a simple python web framework which runs both as standalone and on Appengine.
I would like to write the app in such a way that i can switch between standalone and Appengine based on the deployment configuration. Standalone will be using RDBMS.
When i checked web.py looked simple and promising. I have the following questions:
Does web.py work well on Appengine?
Is there any major known gotchas?
Can you share your experience with using web.py on Appengine?
Any suggestions to achieve the above mentioned goal is appreciated.
Does web.py work well on Appengine?
Yes it does work pretty well, it's compact and very easy to use.
Is there any major known gotchas?
Web.py does not offer any Data Abstraction Layer compatible with GAE; that means that in order to switch from a RDBMS to a NoSQL database, you have to create by yourself a layer above both the web.db database API and the Google App Engine datastore API.
Can you share your experience with using web.py on Appengine?
I have deployed a couple of simple applications with web.py on top of GAE and other several pet projects with Sqlite as database; here a mini review.
Any suggestions to achieve the above mentioned goal is appreciated.
If the switch from Appengine to RDBMS is your primary concern, have a look to web2py.
Although with some limitations, to the best of my knowledge is the only one Python web framework GAE friendly that offers a DAL above the App Engine Datastore and other several RDBMS solutions.
Switching from one db to another is just a matter of initializing the DAL with the proper connection string:
db = DAL('sqlite://storage.db')
db = DAL('mysql://username:password#localhost/test')
db = DAL('gae')
No webapp framework on its own is going to be able to do this - the App Engine environment consists of a lot more than just a database interface and a CGI environment. Even if it could, any DAL that supports both the datastore and a relational DB is likely to sacrifice most of the flexibility of both in pursuit of that.
Your best option, if you want to take advantage of more than just the datastore, is to write your app for App Engine, and use AppScale or TyphoonAE to run your app outside the App Engine environment.
Consider using webapp2. It is similar to web.py, provides basically the same featureset, and runs outside of App Engine out-of-the-box. Besides, you get stellar compatibility with App Engine SDK and libraries (I explain it better here).
You'd only need to add a relational database library. But this is an easy choice: SQLAlqchemy is pretty much the standard in Python land, with lots of documentation and a thriving community.
I would also suggest web2py. Great framework, great community, plays out-of-the-box outside of AppEngine. It also supports a lot of databases (SQLite, PostgreSQL, MySQL, MSSQL, FireBird, Oracle, IBM DB2, Informix, Ingres, and Google App Engine).
On top of that, it's trivial to install it on Linux, Windows, Mac.
I have a written a very small web-based survey using cgi with python(This is my first web app. ).The questions are extracted from a MySQL database table and the results are supposed to be saved in the same database. I have created the database along with its table locally. My app works fine on my local computer(localhost). To create db,table and other transaction with the MySQL i had to do import MySQLdb in my code.
Now I want to upload everything on my personal hosting. As far as I know my hosting supports Python,CGI and has MySQL database. And I know that I have to change some parameters in the connection string in my code, so I can connect to the database, but I have two problems:
I remember that I installed MySQLdb as an extra to my Python, and in my code i am using it, how would I know that my hosting's python interpretor has this installed, or do I even need it, do I have to use another library?
How do I upload my database onto my hosting?
Thanks
If you have shell access, you can fire up the python interpreter by running python and type import MySQLdb at the >>> prompt. If you get no errors in return, then its installed.
Likewise, if you have shell access, this page will help you with importing and exporting using the mysql command. I found it by googleing "import export mysql".
You can write a simple script like
import MySQLdb and catch any errors
to see if the required package is
installed. If this fails you can ask
the hosting provider to install your
package, typically via a ticket
The hosting providers typically also provide URL's to connect to the MySQL tables they provision for you, and some tools like phpmyadmin to load database dumps into the hosted MySQL instance
To check if MySQLdb library is installed on your hosting, simply open a python shell and type: import MySQLdb. If everthing goes ok, you're readt to go. If you get: ImportError: No module named MySQLdb, that means the the library is not installed and you nedd to install it.
You need that library or some library that provides similar support, because Python does not support native access to MySQL databases.
To transfer you database to your hosting check mysqldump.
I'm trying to use a MongoDB Database from a Google App Engine service is that possible? How do I install the PyMongo driver on Google App Engine? Thanks
No, it is not possible. Read the Runtime Environment section. On App Engine you'll have to use the datastore, or a 'database service' you can access using HTTP calls.
You might want to check out TyphoonAE.
It's not possible because you don't have access to networks sockets in App Engine. As long as you cannot access the database via HTTP, it's impossible.
The new runtime environment section says:
an app cannot write data to the local file system or make arbitrary network connections.
I am not sure to which level this is still being enforced (seeing that it's 7 years since the question was asked and answered), but we at Tam have successfully used the PyMongo driver in the standard Google App Engine Python runtime without issues. We simply followed the official PyMongo tutorial and it worked out.