I'm using app engine with cloud endpoints to deploy APIs. All service in the following description is about APIs.
I know in app engine flex env, I could deploy several independent service which talk to each other only through REST.
It's a great idea to make services as independent as possible, but there are some opportunities for code reuse I don't know how to achieve.
For example, there are same procedures I will need to use regardless which API under which service. Usually I'll write a helper library so all these small helper functions would be in one place and I could reuse them across various APIs.
The purpose is to write once and use at many places.
So my questions:
Is this shared library still good idea under the context of app engine? If not, wouldn't it has too much overhead to make all those small function as api?
If shared helper library still make sense, how to achieve it in app engine flex environment? I know in standard env you could use includes directive to include files, I don't see how in flex environment.
I know we could use 3rd party library by declaring them in the requirement and pip support install from github repo, but my helper library would be a private repo, how to allow app engine to pip install private repo?
Thanks in advance.
You can specify private dependencies in requirements.txt files. You would do this in the same way you would handle private dependencies for any other python package. You might directly specify a github url, or you might use a private repository such as an instance of devpi.
Related
We working on a project which involves ML/AI integration to the native mobile application. We are programing our ML/AI code in python. Python code has dependencies, that we need to include in our mobile application.
We have tried with kivy but they only create .apk files and apk files can't be called from other apks. So, we need to create libraries that can be included in the android and ios projects.
Also, we tried chequopy but that doesn't support mediapipe which is in heart of our implementation.
Any guidance in that direction will go long way for us.
If your app was entirely self-contained in python including dependencies using recipes should be possible. If rewriting the native app is not an option maybe one idea is to serve the ML over an HTTP API running on a local server (eg flask). Quite cumbersome as users would need to install two apps
My Python App Engine Flex application needs to connect to an external Oracle database. Currently I'm using the cx_Oracle Python package which requires me to install the Oracle Instant Client.
I have successfully run this locally (on macOS) by following the Instant Client installation steps. The steps required me to do the following:
Make a directory called /opt/oracle
Create a symlink from /opt/oracle/instantclient_12_2/libclntsh.dylib.12.1 to ~/lib/
However, I am confused about how to do the same thing in App Engine Flex (instructions). Specifically, here's what I'm confused about:
The instructions say I should run sudo yum install libaio to install the libaio package. How do I do this on GAE Flex? Or is this package already available?
I think I can add the Instant Client files to GAE (a whopping ~100MB!), then set the LD_LIBRARY_PATH environment variable in app.yaml to export LD_LIBRARY_PATH=/opt/oracle/instantclient_12_2:$LD_LIBRARY_PATH. Will this work?
Is this even feasible without using custom Docker containers on App Engine Flex?
Overall I'm not sure if I'm on the right track. Would love to hear from someone who has managed this before :)
If any of your dependencies is not available in the base GAE flex images provided by Google and cannot be installed via pip (because it's not a python package or it's not available in PyPI or whatever other reason) then you can't use the requirements.txt file to get it installed in your GAE flex app.
The proper way to satisfy such dependencies would be to build your own custom runtime. From About Custom Runtimes:
Custom runtimes allow you to define new runtime environments, which
might include additional components like language interpreters or
application servers.
Yes, that means providing a custom Docker file. In your particular case you'd be installing the Instant Client and libaio inside this Dockerfile. See also Building Custom Runtimes.
Answering your first question, I think that the instructions in the oracle website just show that you have to install said library for your application to work.
In the case of App engine flex, they way to ensure that the libraries are present in the deployment is with the requirements.txt textfile. There is a documentation page which does explain how to do so.
On the other hand, I will assume that "Instant Client Files" are not libraries, but necessary data for your App to run. You should use Google Cloud Storage to serve them, or any other alternative of Storage within Google Cloud.
I believe that, if this is all what you need for your App to work, pushing your own custom container should not be necessary.
Can somebody explain me how to use the setuptools inside python in google app engine to implement WSGIProxy for a webapp.
How do i utilize it, if i dont have access to the filesystem? Specifically,easy steps on how install package from python egg on GAE.
This should be relatively easy for someone who has used setuptools or installed 3rd party packages on GAE python.
I just answered almost the same question, but about a different library. The concept behind installing thirdparty libraries is exactly the same though, you need to either put a copy of the actual code in your app folder, or use a softlink to in.
GAE - Including external python modules without adding them to the repository?
I am investigating into if I can use a library like GHMM with my python web service in which runs on AppEngine.
Short answer: no
https://developers.google.com/appengine/kb/commontasks
What third party libraries can I use in my application?
You can use any pure Python third party libraries in your Google App Engine application. In order to use a third party library, simply include the files in your application's directory, and they will be uploaded with your application when you deploy it to our system. You can import the files as you would any other Python files with your application.
As #gahooa has said, the generic answer is no.
For more popular libraries that have C dependencies your best option right now is to file a ticket[1], get other to upvote (star) your ticket and have the App Engine add it as a supported library.
[1] http://code.google.com/p/googleappengine/issues/entry?template=Feature%20request
In 2021 yes you can.
The flexible version of AppEngine allows you to do this, standard does not.
If, like me, you cannot justify the full time running costs of flexible, an alternative is to host the C library on Cloud Run and make API calls to it. Then you have costs of AppEngine standard and Cloud Run, but both are on-demand only.
I am starting on developing a django application on a shared webhosting server(alwaysdata.com).
I would like to understand what are the packaing options available to package a django application (preferably in compiled form)
I would like to setup the source code repository on my system and build using the python packaging and deploy the package on alwaysdata.com.
I would like the option of not having to share the source code on alwaysdata.com.
Looking at pip , I am not able to find this option.
EDIT
A few more clarification: I would need the ability to not share the sourcecode since it contains the "API secret key" which I would not want to compromise. The more I look into it , the more i believe that there is no way for me to distribute binary only distribution.
I've found fabric to be a pretty nice tool for deploying Django projects. To compile your python code you can use compileall:
python -m compileall <dir>
How is this API key used? Is it a google maps api? Is it provided in scripts that go to the browser? If so, it's already out in the open, anyone using your site will see it, so you're trying to provide a $100 lock for a $0.01 piece of information. If it's a google maps api, it's not secured by keeping it hidden, but rather it's tied to a domain/directory (IIRC).
Can you share a little more what the API key is and is for, then maybe we can help you find a better solution to keep it secure.
Do you think you have to share your source code if you host your application on a 'shared hosting' provider? That's not the case. Your source code should still be private to you but the administrators of your hosting provider can get it too. Other normal Joe Users of the service shouldn't have access to your source code, or your database too. If they do, then get another shared hosting provider!