I'd like to shorten the process of deploying to Heroku (i.e. a git push)
I use git-flow to organise my codebase - so typically the process would be:
start a new feature branch
Do the coding
Push this branch up to my dev heroku instance - git push develop feature/somefeature:master)
Merge into the develop branch
Create a new release branch
Push this to the production heroku instance - git push production release/1.2.3:master
What I'd like to do is be able to run a Fab command like:
fab dev_deploy
which would just deploy whatever the current working branch is to my dev instance
and
fab prod_deploy
which would do the same but push to the production instance.
I could include some sanity checks here to make sure I'm inside a release branch etc.
my fab commands would do other things (like push static assets up to the right S3 bucket etc, run south migrate commands and so on)
So all I really want to know is how to get the current working branch as a variable inside fabric...!?
Thanks,
Guy
OK - a bit more digging got me this:
from fabric.api import local
my_branch = local('git rev-parse --abbrev-ref HEAD', capture=True)
which does exactly what I wanted.
import subprocess
my_branch = subprocess.check_output(['git','branch'])
or:
from fabric.api import local
my_branch = local('git branch')
Related
Generally, my question is about being able to access ssh keys during docker-compose build.
I'm able to access my ssh keys when running docker-compose up using volume mapping in my docker-compose.yml file, looks like:
services:
flask:
volumes:
- ~/.ssh:/root/.ssh
But I cannot access them during docker-compose build
More Specifics
I am running a python flask app. I want to install a private git repo as a pip package. So I added this line to requirements.txt
git+ssh://git#github.com/username/repo_name.git#branch_name
If I run bash in the service through docker-compose run flask bash then I can manually run pip install git+ssh://git#github.com/username/repo_name.git#branch_name and that works, because I have the volume mapping to the ssh keys.
But when I run docker-compose build, it cannot access the private git repo because it doesn't have access to the ssh keys.
Anyone know if there's a way to give docker-compose build access to ssh keys, or another way around this problem?
volumes are attached at run time of your container, NOT at build time.
Solution:
Copy your .ssh next to your Dockerfile and do the following in your Dockerfile:
COPY ./.ssh /root/.ssh
Be careful:
Like this, your .ssh directory will be available for everyone who has access to your Docker image. So either create a technical user and copy his .ssh into the image or (better) do something like this:
FROM baseimage AS builder
COPY ./.ssh /root/.ssh
RUN your commands
FROM baseimage
COPY --from=builder some-directory some-directory
Edit:
Another option is to use username:password instead of ssh key authentication. This way, you would use build args in your Dockerfile like:
FROM baseimage
ARG GIT_USER
ARG GIT_PASS
RUN git clone http://${GIT_USER}:${GIT_PASS}#your-git-url.git
and build it with args docker build --build-args GIT_USER=<user> --build-args GIT_PASS=<pass> .
ssh is harder to setup than just using the username:password. Here is the line I added to requirements.txt that got it to work:
-e git+https://<username>:<password>#github.com/<path to git repo>.git#egg=<package_name>
If you want to get a specific tag, then you can do this:
-e git+https://<username>:<password>#github.com/<path to git repo>.git#<tag_name>#egg=<package_name>
You can use any username and password that has access to the git repo, but I recommend that you don't use your main git account for obvious security reasons. Create a new user specifically for this project, and grant them access rights.
I have a django project under development on my windows computer (dev-machine). I am using pyCharm for development.
I have set up a server (server-machine) running ubuntu. And now want to push my project to the server.
So in my project folder on the dev-machine I have done the git init:
$ git init
$ git add .
$ git commit -m"Init of Git"
And on the server-machine I have made a project folder: /home/username/projects
In this folder I init git as well
$ git init --bare
Back on my dev-machine, I set the connection to the server-machine by doing this
$ git remote add origin username#11.22.33.44:/home/username/projects
And finally pushing my project to server-machine by typing this command on my dev-machine
$ git push origin master
It starts to do ome transfer. And here's the problem.
On the server-machine when I check what's been transferred, it's only stuff like this
~/projects$ ls
branches config description HEAD hooks info objects refs
Not a single file from the project is transferred. This looks much like what the .git folder contains on the dev-machine.
What am I doing wrong?
What you see is the directory structure git uses to store your files and meta data. This is not a checked-out copy of the repository.
To check whether the data made it into the repository use git log in ~/project
Okay, so I understand now where I went wrong.
With git push I am not setting up the project.
To set up the project I need to do git clone.
This is how I did it.
1.
So I made a folder for git repositories on the server-machine. I called it /home/username/gitrepos/
2.
Inside there, I made a folder for my project, where I push the git repository into. So path would look like this for me /home/username/gitrepos/projectname/
3.
Being inside that folder I do a 'git init' like this
$ git init --bare
4.
Then I push the git repo to this location. First setting the remote adress from my dev-machine with this command. If adding a remote destination new use this:
$ git remote set nameofconnection username#ip.ip.ip.ip:/home/username/gitrepos/projectname
if changing the adress for a remote destination use this:
$ git remote set-url nameofconnection username#ip.ip.ip.ip:/home/username/gitrepos/projectname
To se with remote destinations you have set type this:
$ git remote -v
5.
Now go back to server-machine and clone the project into a project folder. I made a folder like this /home/username/projects/
When being inside that folder I clone from the gitrepo ike this:
$ git clone /home/username/gitrepos/projectname
Thank you all for the help! <3
I'm working to modify a cookiecutter Flask app.
locally I have deleted the migration folder and sqllite db a couple of times during development. I've pushed my chnages to heroku.
When trying to migrate the heroku postgresdb :
$ heroku run python manage.py db upgrade
.....
alembic.util.CommandError: Multiple head revisions are present for given argument 'head'; please specify a specific target revision, '<branchname>#head' to narrow to a specific head, or 'heads' for all heads
following http://alembic.readthedocs.org/en/latest/branches.html I tried:
$ heroku run python manage.py db merge heads
Running python manage.py db merge heads on myapp... up, run.9635
Generating /app/migrations/versions/35888775_.py ... done
Then I tried:
$ heroku run python manage.py db upgrade
Running python manage.py db upgrade on myapp... up, run.7021
INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO [alembic.runtime.migration] Will assume transactional DDL.
....
"%s#head" % branch_label if branch_label else "head")
alembic.util.CommandError: Multiple head revisions are present for given argument 'head'; please specify a specific target revision, '<branchname>#head' to narrow to a specific head
, or 'heads' for all heads
How can I merge the revision heads into one and make sure this is synced with my development version?
I contacted heroku support and got the following back (which worked for me):
Hi,
To remove a folder from your local repository, git rm needs to be run. Could you please try something like below?
$ git rm -r migrations
$ git commit -m 'Remove migrations directory'
$ git push heroku master
To see differences between actual files and what are registered onto your local repository, git status may be useful.
Please let us know if you have any difficulty here.
After accidentally damaging my flask app on openshift I deleted it and am trying to rebuild it. I believe I have installed it correctly by creating a new python app, then performing:
$ git remote set-url origin ssh://55ddee2489f5.......#myapp-mydomain.rhcloud.com/~/git/myapp.git/
$ git push -f origin master
then
remote: Activation status: success
remote: Deployment completed with status: success
To ssh://55ddee248........c#myflaskapp-mydomain.rhcloud.com/~/git/myflaskapp.git/
+ 068620c...00df6fb master -> master (forced update)
Next I want to add a redis cartridge.
$ rhc add-cartridge http://cartreflect-claytondev.rhcloud.com/reflect\?github\=smarterclayton/openshift-redis-cart
The cartridge 'http://cartreflect-claytondev.rhcloud.com/reflect?github=smarterclayton/openshift-redis-cart' will be downloaded and installed
Adding http://cartreflect-claytondev.rhcloud.com/reflect?github=smarterclayton/openshift-redis-cart to application 'myflaskapp' ... Application '5585ab144.......'
not found.
As you can see the cartridge is being deployed to the old location '5585ab144.......', not ssh://55ddee248........c#myflaskapp-mydomain.rhcloud.com/~/git/myflaskapp.git/
How can I fix this?
If you use the same DNS (application) name (app-domain.rhcloud.com) that your old application was using, you need to wait for the DNS to update and point to the new application. It could take up to 24 hours, but usually it just takes a couple of hours.
I am using Circle CI for tests and pushing a python application to heroku in order to also run Web GUI tests on the machine.
What is your recommended way of filling up the heroku instance with a certain database content?
I do not think that circle CI should access the heroku instance or the database directly?
The heroku deploy hooks only seem to be able to call a web hook. But I would like to run a command to reset the database.
If you're using the built-in heroku deployments, you won't be able to do this, e.g. if your configuration looks like this:
deployment:
staging:
branch: master
heroku:
appname: foo-bar-123
You can instead configure your deployment to run several commands:
deployment:
production:
branch: production
commands:
- "[[ ! -s \"$(git rev-parse --git-dir)/shallow\" ]] || git fetch --unshallow"
- git push git#heroku.com:foo-bar-123.git $CIRCLE_SHA1:refs/heads/master
- heroku run rake db:migrate --app foo-bar-123:
timeout: 400 # if your deploys take a long time
The comment from Two-Bit Alchemist would be a good command to reset the database.
The above examples are taken from: https://circleci.com/docs/continuous-deployment-with-heroku#heroku-setup