We Keep Coding

How to deploy Google Cloud Function with shared local dependencies?

I am having some trouble with local modules when deploying on Cloud Functions - any ideas or best practices would be appreciated!
I am trying to deploy a piece of my project as a Cloud Function. It uses some local code from the project, which is shared with other modules - and I use an absolute import for that. I am using a Cloud Repository for deployment, and there I state the folder where the function resides (parent\cloud_function\). The problem is the parent package is not available with that setup.
This is an example of the project structure:
├── parent_repo
│   ├── parent
│   │   ├── __init__.py
│   │   ├── config.conf
│   │   ├── config.py
│   │   ├── cloud_function
│   │   │   ├── __init__.py
│   │   │   ├── main.py
│   │   │   └── requirements.txt
│   │   ├── shared_module
│   │   │   ├── __init__.py
│   │   │   ├── package1.py
│   │   │   └── package2.py
│   │   ├── other_module
│   │   │   ├── __init__.py
│   │   │   ├── some_script.py
│   │   │   └── another_script.py
│   │   └── utils.py
inside parent.cloud_function.main.py AND in parent.other_module.some_script.py I use:
from parent.shared_module.package1 import some_func
from parent.shared_module.package2 importsome_class
to access shared code. However, when trying to deploy the function on Cloud Functions, since I assume it only looks at the folder alone, the parent module is unavailable.
Of course I could simply nest all required code inside the cloud_function folder - but from a project perspective that isn't ideal - as that code is shared across other resources, and does not logically belong there.
Does anyone have a good idea how to this better?
Thanks in advance!

Very shortly - it is difficult.
Here is Python runtime - Specifying dependencies description -
requirements.txt file or packaging local dependencies alongside your
function
You probably can play with Private dependencies in the Cloud Build
Some ideas are provided in the Deploy a Python Cloud Function with all package dependencies SO question.

heroku can't detect buildpack

I have a Flask app that I am trying to put on heroku. I have a requirements.txt file with the requirements for my project, and heroku says that this should be enough to let heroku detect python but it does not. I can manually set the buildpack to python like so
heroku buildpacks:set heroku/python but then I get this error: (from running git push heroku master)
remote: -----> App not compatible with buildpack: https://buildpack-registry.s3.amazonaws.com/buildpacks/heroku/python.tgz
remote: More info: https://devcenter.heroku.com/articles/buildpacks#detection-failure
No default language could be detected for this app.`
What is wrong with my project layout?
Here is my file tree:
.
├── faceParser
│   ├── __init__.py
│   ├── recolor.py
│   ├── static
│   │   ├── libs
│   │   │   ├── bootstrap.min.css
│   │   │   ├── bootstrap.min.js
│   │   │   ├── jquery.min.js
│   │   │   ├── notify.js
│   │   │   └── webcam.min.js
│   │   ├── sketch.js
│   │   └── style.css
│   └── templates
│   ├── base.html
│   └── index.html
├── main.sh
├── README.md
├── requirements.txt
└── venv
(virtual environment files omitted because there are a lot of them)
This is how I run it locally:
export FLASK_APP=faceParser
export FLASK_ENV=development
flask run
Thanks!

I changed my project layout to contain an app.py file in the outermost directory, and added a Procfile file and a runtime.txt file (python-3.5.2)
I think that Heroku needs these files to understand that it is a python project.
Now it works.

Openshift Find Static Files From Django App

I am trying to add a css file to my django application which I am deploying on openshift. Currently, I am able to have the css file work when running locally with debug and all that. However, when I deploy my application to openshift, the css file does not have any effect. Many of the other questions online seem to be referring to old versions of django and openshift (lots of references to a wsgi folder that doesn't seem to be standard anymore and triggering collectstatic manually)
My understanding with the current django/openshift relationship is that I do not need to do collecstatic manually as I am using the django-ex template from openshift, and during the build process the collectstatic command is run automatically, the ouput of which I can see in the logs. Notably, I can see that my style.css file for my application gets copied into STATIC_ROOT in that log.
My project structure from my project root is thus:
.
├── db.sqlite3
├── djangoWrapper
│   ├── __init__.py
│   ├── settings.py
│   ├── templates
│   │   └── djangoWrapper
│   │   └── index.html
│   ├── urls.py
│   ├── views.py
│   └── wsgi.py
├── gitlab-ci.yml
├── ldap
│   ├── admin.py
│   ├── apps.py
│   ├── forms.py
│   ├── __init__.py
│   ├── migrations
│   ├── models.py
│   ├── static
│   │   └── ldap
│   │   └── style.css
│   ├── templates
│   │   └── ldap
│   │   ├── apiOffline.html
│   │   ├── index.html
│   │   ├── results.html
│   │   └── search.html
│   ├── tests.py
│   ├── urls.py
│   └── views.py
├── manage.py
├── openshift
├── README.md
├── requirements.txt
Here's what I am doing in my settings.py:
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
STATIC_URL = '/static/'
STATICFILES_DIRS = [os.path.join(BASE_DIR, "ldap", "static"),]
STATIC_ROOT = os.path.join(BASE_DIR, 'static')
In my html files, I am then trying to load the css file with
{% load static %}
<link rel="stylesheet" type="text/css" href="{% static 'ldap/style.css' %}">
in the <head> section.
I have attempted to do some debugging here because it would be assumed that the problem before I got to deployment was that my static files were not being put in the correct place/djgano couldn't find them. So I put some prints into my app and deployed it on openshift and got back the following info:
BASE_DIR: /opt/app-root/src
STATIC_ROOT: /opt/app-root/src/static
STATIC_URL: /static/
APP_VIEW_DIR: /opt/app-root/src/ldap/views.py
as well putting <p>"{%static 'ldap/style.css' %}"</p> into an html file, which displays as /static/ldap/style.css
From these tidbits, I know that the root directory for openshift is /opt/app-root/src/, that my static files are in /opt/app-root/src/static, and that my html is looking for the css file in /static/ldap/style.css. This seems like it should be working then as all of those directories line up. Is there something else about serving static files with django/openshift that I am missing here?

Finally got it working! I did use whitenoise, following the steps here.

Importing app when using Alembic raises ImportError

I am trying to study how to use alembic in flask, I want to import a method in flask app:
tree .
.
├── README.md
├── alembic
│   ├── README
│   ├── env.py
│   ├── env.pyc
│   ├── script.py.mako
│   └── versions
│   ├── 8f167daabe6_create_account_table.py
│   └── 8f167daabe6_create_account_table.pyc
├── alembic.ini
├── app
│   ├── __init__.py
│   ├── main
│   │   ├── __init__.py
│   │   ├── errors.py
│   │   ├── forms.py
│   │   └── views.py
│   ├── models.py
│   └── templates
│   ├── 404.html
│   ├── 500.html
│   ├── base.html
│   ├── index.html
│   └── user.html
├── config.py
├── data.sqlite
├── manage.py
└── requirements.txt
in app/__init__.py:
def create_app(config_name):
app = Flask(__name__)
I want to import create_app in env.py:
from app import create_app
but the error shows as below when I run the command alembic upgrade head:
File "alembic/env.py", line 5, in <module>
from app import create_app
ImportError: No module named app
Any idea for this?

I guess you are trying to run
python env.py
In this case, your app directory is not in PYTHONPATH.
solution 1
Run the app from parent dir:
python alembic/env.py
solution 2
Set the PYTHONPATH before running the app
PYTHONPATH=/path/to/parent/dir python env.py
edit
I read about alembic. As #mrorno said, just set the PYTHONPATH before running alembic:
PYTHONPATH=. alembic upgrade head

alembic just tries to load your env.py source code. It's not in your package, so it can't access your app module.
Use solution 2 #tomasz-jakub-rup suggested, you can execute like
$ PYTHONPATH=. alembic upgrade head
and you should get your result.

Create file .env and insert PYTHONPATH=.

Flask url_for incorrectly links to static pages outside of templates

I'm currently in the process of moving all of my static files to S3, as well as allowing some image uploads to my site. In general this is going fantastically. I got all of my existing css and js files up to S3 with no hassle, however I'm having some trouble with uploading images and saving them to S3.
Specifically this is how I'm handling file uploads within my view:
image_file = request.files["imageurl"]
if image_file and allowed_file(image_file.filename):
fn = "products/%s" % secure_filename(image_title + "." + image_file.filename.split(".")[-1])
image_file.save(url_for('static', filename=fn))
else:
response = app.make_response(render_template(
'admin/newImage.html',
title=title,
error="That Image file is invalid.")
)
return response
Which is all wrapping in a POST request handler. The problem here is that the url_for('static') fails to link to the correct path, and so I get an IOError anytime I try to save an image like this.
Normally I would assume I was just doing something silly with my directory structure, but the same pattern of url_for works perfectly for files in my static directory. Any ideas on how to remedy this? Here's my directory structure (trimmed down for viewing)
├── SpoolEngine
│   ├── admin.py
│   ├── __init__.py
│   ├── templates
│   │   ├── admin
│   │   │   ├── base.html
│   │   ├── base.html
│   │   ├── _forms.html
│   │   ├── posts
│   │   │   ├── complete.html
│   │   └── super_user
│   │   ├── base.html
│   ├── users.py
│   └── views.py
└── static
├── css
│   ├── admin.css
│   └── zebraTable.css
├── img
│   └── subtle_grunge.png
├── js
│   ├── compatibility.js
│   ├── list.js
│   ├── login.js
│   └── profiler.js
└── products
And for reference, url_for works perfectly within /static/css and links to the wrong url from admin.py Any ideas?

url_for returns an url path. Not a filesystem path.
So you're trying to save your file at /static/something which, for the system, means a path starting from the root of your filesystem, not your application path.
You can create the static path for your file with something like this
static_path_to_save = os.path.join([app.root_path, '/static', fn])
Just a side note, when dealing with uploads, remember to sanityze all the paths and to double check the destination. Best practices applies, like stripping slashes if you use the filename provided by the user (best is if you generate the filename).
In your code I see also a problem if 2 users uploads a file with the same name, overwriting each other. But this is probably safe in your context.