We Keep Coding

Google Cloud Buildpack custom source directory for Python app

I am experimenting with Google Cloud Platform buildpacks, specifically for Python. I started with the Sample Functions Framework Python example app, and got that running locally, with commands:
pack build --builder=gcr.io/buildpacks/builder sample-functions-framework-python
docker run -it -ePORT=8080 -p8080:8080 sample-functions-framework-python
Great, let's see if I can apply this concept on a legacy project (Python 3.7 if that matters).
The legacy project has a structure similar to:
.gitignore
source/
main.py
lib
helper.py
requirements.txt
tests/
<test files here>
The Dockerfile that came with this project packaged the source directory contents without the "source" directory, like this:
COPY lib/ /app/lib
COPY main.py /app
WORKDIR /app
... rest of Dockerfile here ...
Is there a way to package just the contents of the source directory using the buildpack?
I tried to add this config to the project.toml file:
[[build.env]]
name = "GOOGLE_FUNCTION_SOURCE"
value = "./source/main.py"
But the Python modules/imports aren't set up correctly for that, as I get this error:
File "/workspace/source/main.py", line 2, in <module>
from source.lib.helper import mymethod
ModuleNotFoundError: No module named 'source'
Putting both main.py and /lib into the project root dir would make this work, but I'm wondering if there is a better way.
Related question, is there a way to see what project files are being copied into the image by the buildpack? I tried using verbose logging but didn't see anything useful.
Update:
The python module error:
File "/workspace/source/main.py", line 2, in <module>
from source.lib.helper import mymethod
ModuleNotFoundError: No module named 'source'
was happening because I moved the lib dir into source in my test project, and when I did this, Intellij updated the import statement in main.py without me catching it. I fixed the import, then applied the solution listed below and it worked.

I had been searching the buildpack and Google cloud function documentation, but I discovered the option I need on the pack build documentation page: option --path.
This command only captures the source directory contents:
pack build --builder=gcr.io/buildpacks/builder --path source sample-functions-framework-python
If changing the path, the project.toml descriptor needs to be in that directory too (or specify with --descriptor on command line).

Neo4j graph academy - building neo4j applications with python - no .env file

I'm doing the building Neo4j applications with Python course on Neo4j's GraphAcademy and am stuck early in the process with Setting Environment Variables. I've installed the dependencies (FLASK etc.) but don't seem to have an .env file for the next part...
Setting Environment Variables
This project will read environment variables from the .env file located in the project root.
The project contains an example file at .env.example. You can run the following command in your terminal window to copy the example file to .env.
cp .env.example .env
But when I try to run this in the shell I get the following error:
cp: .env.example: No such file or directory
I don't seem to have a .env file in any of the newly created folders in the sandbox. Can anyone help with this?

For me, this worked:
Clone the git repository that provides the scaffolding
git clone https://github.com/neo4j-graphacademy/app-python.git
Change directory to be in the newly checked project root folder.
This step is not explicitly mentioned in the graph academy course.
cd app-python
Copy the template env file
cp .env.example .env
Inspect the file to make sure it looks right
cat .env
which printed
FLASK_APP=api
FLASK_ENV=development
FLASK_RUN_PORT=3000
NEO4J_URI=neo4j://localhost:7687
NEO4J_USERNAME=neo4j
NEO4J_PASSWORD=neo
JWT_SECRET=secret
SALT_ROUNDS=10
If you still can't get it to work, let me know which step fails and with which error.

git ignore not ignoring a file when using GCloud's gcloudignore

I have a rule to ignore secret.py (production.py in my case) and once I added .gcloudignore, github stopped following that rule...
Is there some sort of a rule overriding between gitignore and gcloudignore that I am not aware of?
my-project/
.git
.gitignore
my-project/
.gcloudignore
settings/
base.py
local.py
production.py
my .gitignore:
my-project/my-project/settings/production.py
my-project/my-project/settings/local.py
my .gcloudignore:
# This file specifies files that are *not* uploaded to Google Cloud Platform
# using gcloud. It follows the same syntax as .gitignore, with the addition of
# "#!include" directives (which insert the entries of the given .gitignore-style
# file at that point).
#
# For more information, run:
# $ gcloud topic gcloudignore
#
.gcloudignore
# If you would like to upload your .git directory, .gitignore file or files
# from your .gitignore file, remove the corresponding line
# below:
.git
.gitignore
*.sqlite3
settings/local.py
End result is that the 'local.py' is NOT pushed to google cloud NOR github.
However, 'production.py' IS pushed to github AND gcloud.

If you had previously (perhaps accidentally) submitted a change to git that included my-project/my-project/settings/production.py, then it will remain a part of the repository even if it is subsequently added to .gitignore.
Assuming you are at the root of your project, you can use
$ git log my-project/my-project/settings/production.py
to see its git history. If it is present in your repo, you can do
$ git rm --cached my-project/my-project/settings/production.py
to remove it from the repo, but keep it in your local (working) environment.

Looks like you are enforcing to ignore the rules applied before.
Though make sure that the cloudignore is enable via
gcloud config set gcloudignore/enabled true
And make sure to place .gcloudignore - > in the root of your project / basically where you .git .gitignore reside.

How to use Travis CI with some files in gitignore?

I have a Flask app that has its configurations in a file called settings.py. I've put this file in .gitignore because the project is in a public repo. Travis-CI was working before I added tests into my project even though settings.py was in .gitignore. After adding tests to the project, the build started failing with the following output:
Debugged import:
- 'settings' not found.
Original exception:
ImportError: No module named 'settings'
My .travis.yml file looks like this:
language: python
python:
- "3.4"
- "3.5"
# command to install dependencies
install:
- pip install -r requirements.txt
# command to run tests
script: python tests.py
Does this mean that in order to use travis-ci, we have to include all necessary files in the repo? Or is there a workaround? The repo on GitHub can be found here.

#dirn's comment of using a default settings.py file and then overriding some settings with encrypted environment variables on Travis is a good idea, certainly worth it if there are only a couple of differences.
However, if you can't be bothered or it's too complicated breaking up your settings, you could install the Ruby Travis command line client gem, which is useful for quite a few things.
With the client on your machine you can use Travis' file encryption feature to encrypt your whole settings.py file, and then commit the encrypted version (which will have an .enc file extension) to GitHub. Travis will then be able to decrypt the file during the CI run, as long as you add the right commands to the .travis.yml file, say in a before_install step. Detailed instructions are on the file encryption page.

I did a trick in .travis.yml
After commit and before Travis build, create the ignored file like this:
before_install:
- cp .ignored.file.copy ignored.file
This way, the build will succeed without the actual gitignore-ed file.

How to add all catalog using GIT without .pyc file?

How to add all catalog using GIT witout .pyc file?
git add catalog_name
git commit -m "Update"
git push
What add and where?

use .gitignore file. add '*.pyc' to it.
The file is located in root directory of your repo i.e. where you did git init
Here is a good .gitignore file for python projects, containing common extension to ignore in git commit.
https://github.com/github/gitignore/blob/master/Python.gitignore