I am investigating into if I can use a library like GHMM with my python web service in which runs on AppEngine.
Short answer: no
https://developers.google.com/appengine/kb/commontasks
What third party libraries can I use in my application?
You can use any pure Python third party libraries in your Google App Engine application. In order to use a third party library, simply include the files in your application's directory, and they will be uploaded with your application when you deploy it to our system. You can import the files as you would any other Python files with your application.
As #gahooa has said, the generic answer is no.
For more popular libraries that have C dependencies your best option right now is to file a ticket[1], get other to upvote (star) your ticket and have the App Engine add it as a supported library.
[1] http://code.google.com/p/googleappengine/issues/entry?template=Feature%20request
In 2021 yes you can.
The flexible version of AppEngine allows you to do this, standard does not.
If, like me, you cannot justify the full time running costs of flexible, an alternative is to host the C library on Cloud Run and make API calls to it. Then you have costs of AppEngine standard and Cloud Run, but both are on-demand only.
Related
I am new to Google Cloud Platform and in my whole I have been working on Python 3. I am trying to find out which version of Python is more complete for Google App Engine: Python 2.7 or Python 3.
As I'm starting to work with Google App Engine I have realised that continuing using Python 3 seems too painful as basic tools like dev_appserver.py are written for Python 2 only. Now I am hitting the opposite problem: cloudstorage module seems to exist only for python3. Again, when I install it, seems the only way I can test read/write to google bucket locally is by authenticating with google.appengine.ext, which in turn only works within dev_appserver.py or remotely. This leaves me confused which environment to chose.
What is a general agreement / what is the focus of Google App Engine: Python 2 or Python 3?
In App Engine, you have to options: the Standard environment and the Flexible environment.
Python 2.7 is available in both Standard and Flexible, while Python 3.6 is only available in Flexible.
Also, the choice between Standard and Flexible depends on what you want to do/what libraries you need:
There are some third-party libraries already built-in in the Standard Environment, and you can include other libraries, but, those libraries can't include C extensions, they must be written in pure Python. If you need libraries with C extensions, you will have to move to Flexible.
In Standard, you can use propietary libraries (like google.appengine.ext, as you mentioned) to do tasks like accessing databases, while in Flexible you can use other libraries (like the client you mentioned).
There are also another important differences, like pricing, scaling, etc. The choice will depend, as I said, in your needs for your application.
EDIT
dev_appserver.py is only used when developing in Standard. There is a tutorial in here, with Flask. If you are in Flexible, you can test the app locally as if you were running as usual a python file, like in this other example.
You can use buckets in both Standard and Flexible
The python3-only cloudstorage support assumption based on the SO post you referenced is not correct:
the import appears to be done in a regular python shell or as a standalone script, not from a standard environment GAE app - different things, see import cloudstorage, ImportError: No module named google.appengine.api.
it is not specified where that library comes from
GCS is definitely supported in the standard env GAE (i.e. on python 2), you just need to follow the steps from the official documentation: Setting Up Google Cloud Storage and Reading and Writing to Google Cloud Storage.
Both were good. But the question is what kind of environment do you want? Standard environment or Flexible environment.
Find your answer in this document: https://cloud.google.com/appengine/docs/python/
It kind of depends on what you're using it for. If you're doing data science, for example, I'm seeing a few notices of Python libraries that are (finally) dropping support for Python 2. numpy is one that is dropping support.
Generally speaking, I would recommend Python 3 over Python 2. Why spend time developing in an aging version when its replacement has matured nicely and is more consistent?
I'm using app engine with cloud endpoints to deploy APIs. All service in the following description is about APIs.
I know in app engine flex env, I could deploy several independent service which talk to each other only through REST.
It's a great idea to make services as independent as possible, but there are some opportunities for code reuse I don't know how to achieve.
For example, there are same procedures I will need to use regardless which API under which service. Usually I'll write a helper library so all these small helper functions would be in one place and I could reuse them across various APIs.
The purpose is to write once and use at many places.
So my questions:
Is this shared library still good idea under the context of app engine? If not, wouldn't it has too much overhead to make all those small function as api?
If shared helper library still make sense, how to achieve it in app engine flex environment? I know in standard env you could use includes directive to include files, I don't see how in flex environment.
I know we could use 3rd party library by declaring them in the requirement and pip support install from github repo, but my helper library would be a private repo, how to allow app engine to pip install private repo?
Thanks in advance.
You can specify private dependencies in requirements.txt files. You would do this in the same way you would handle private dependencies for any other python package. You might directly specify a github url, or you might use a private repository such as an instance of devpi.
Can the pandas data analysis module run on Google App Engine?
My first inclination is no: the web page states "critical code paths compiled to C". So since this is not a purely python package, you cannot simply copy a directory or ZIP file into your app engine project.
Is it possible to "disable" the C extensions and have the module run in pure python (albeit slower)?
As of today, Google App Engine Python 2.7 runtime doesn't currently allow running arbitrary C extensions.
But it does allow inclusion of third party library that have been hand selected by the engineering team based on their popularity and developer demand.
You can request for a specific library to be included by filling a feature request on the public issue tracker. If it gets enough ☆☆☆ it is likely to be considered for inclusion in an upcoming release.
I am starting on developing a django application on a shared webhosting server(alwaysdata.com).
I would like to understand what are the packaing options available to package a django application (preferably in compiled form)
I would like to setup the source code repository on my system and build using the python packaging and deploy the package on alwaysdata.com.
I would like the option of not having to share the source code on alwaysdata.com.
Looking at pip , I am not able to find this option.
EDIT
A few more clarification: I would need the ability to not share the sourcecode since it contains the "API secret key" which I would not want to compromise. The more I look into it , the more i believe that there is no way for me to distribute binary only distribution.
I've found fabric to be a pretty nice tool for deploying Django projects. To compile your python code you can use compileall:
python -m compileall <dir>
How is this API key used? Is it a google maps api? Is it provided in scripts that go to the browser? If so, it's already out in the open, anyone using your site will see it, so you're trying to provide a $100 lock for a $0.01 piece of information. If it's a google maps api, it's not secured by keeping it hidden, but rather it's tied to a domain/directory (IIRC).
Can you share a little more what the API key is and is for, then maybe we can help you find a better solution to keep it secure.
Do you think you have to share your source code if you host your application on a 'shared hosting' provider? That's not the case. Your source code should still be private to you but the administrators of your hosting provider can get it too. Other normal Joe Users of the service shouldn't have access to your source code, or your database too. If they do, then get another shared hosting provider!
How to check available Python libraries on Google App Engine & add more?
Is SQLite available or we must use GQL with their database system only?
Thank you in advance.
SQLite is there (but since you cannot write to files, you must use it in a read-only way, or on a :memory: database).
App engine docs do a good job at documenting what's there. You can add any other pure-python library, typically as a zipfile of .py (NOT .pyc) files to upload in the main directory of your app (you can directly import from inside the zipfile, of course).
A few more pure-Python third-party libraries included with app engine are listed and documented here -- the paragraph on zipimport at this URL has a bit more details on the ways and limitations of using zipfiles to add more third-party pure-Python libs to your app.
Afaik, you can only use the GAE specific database.