I am creating a post-commit script in Python and calling git commands using subprocess.
In my script I want to stash all changes before I run some commands and then pop them back. The problem is that if there was nothing to stash, stash pop returns a none-zero error code resulting in an exception in subprocess.check_output(). I know how I can ignore the error return code, but I don't want to do it this way.
So I have been thinking. Is there any way to get the number of items currently in stash? I know there is a command 'git stash list', but is there something more suited for my needs or some easy and safe way to parse the output of git stash list?
Also appreciate other approaches to solve this problem.
Don't do that!
Suppose that git stash save saves nothing, but there are already some items in the stash. Then, when you're all done, you pop the most recent stash, which is not one you created.
What did you just do to the user?
One way to do this in shell script code is to check the result of git rev-parse refs/stash before and after git stash save. If it changes (from failure to something, or something to something-else), you have created a new stash, which you can then pop when you are done.
More recent versions of Git have git stash create, which creates the commit-pair as usual but does not put them into the refs/stash reference. If there is nothing to save, git stash create does nothing and outputs nothing. This is a better way to deal with the problem, but is Git-version-dependent.
You can simply try calling git stash show stash#{0}. If this returns successfully, there is something stashed.
Related
I'm trying to generate a simple Python code that:
checks if it is running inside a git folder
if so, fetch the latest commit, else skip
it should work under the three platforms: Linux, Windows, and Mac
I have this code that works correctly under Linux:
from subprocess import call, STDOUT
import os
if call(["git", "branch"], stderr=STDOUT, stdout=open(os.devnull, 'w')) != 0:
# Not a git folder
commit = ''
else:
# Inside a git folder.: fetch latest commit
commit = subprocess.check_output(['git', 'rev-parse', '{}'.format('HEAD')])
print(commit)
but I have no way of checking if itwill work under Windows and Mac.
Does it work? Is there any way of checking/knowing this sort of things when one has no access to the other operating system?
You don't want to run git branch to detect whether you're in a Git repository, because you may or may not have any branches. To detect whether you're able to use Git commands, you'll want to run something like git rev-parse --git-dir, which will exit non-zero if you're not within a Git repository.
However, there are a couple of other issues with your code. First of all, in a new repository (one created fresh with git init), there will be a .git directory and the above command will succeed, but HEAD will not point anywhere. Therefore, your git rev-parse HEAD command will fail and print HEAD and an error.
Finally, if you want parse a revision, you should usually use --verify so that you don't print the dummy HEAD value on failure. So your invocation should look like git rev-parse --verify HEAD.
Ultimately, it's up to you to figure out what you want to do in a newly initialized repository, whether that's fail or fall back to an empty string.
The behaviors I've described here are consistent across platforms; they're built into Git and well defined.
There's a method check_output in subprocess library
from subprocess import check_output
try:
# use python to parse this log for info. This is your entire last commit
logs = check_output(['git', 'log', '-1', '--stat']).decode("UTF-8")
except Exception as e:
# Do whatever you wanna do otherwise if not git repository
print(e)
Git has a command called "git log".
"-1" indicates the last commit and
--stat will give you the files that were changed, commit ID, TIME ETC
then you can use python to parse this log and retrive any information you want
Check this out for more info on git log
I have a few RUN commands in my Dockerfile that I would like to run with -no-cache each time I build a Docker image.
I understand the docker build --no-cache will disable caching for the entire Dockerfile.
Is it possible to disable cache for a specific RUN command?
There's always an option to insert some meaningless and cheap-to-run command before the region you want to disable cache for.
As proposed in this issue comment, one can add a build argument block (name can be arbitrary):
ARG CACHEBUST=1
before such region, and modify its value each run by adding --build-arg CACHEBUST=$(date +%s) as a docker build argument (value can also be arbitrary, here it is current datetime, to ensure its uniqueness across runs).
This will, of course, disable cache for all following blocks too, as hash sum of the intermediate image will be different, which makes truly selective cache disabling a non-trivial problem, taking into account how docker currently works.
Use
ADD "https://www.random.org/cgi-bin/randbyte?nbytes=10&format=h" skipcache
before the RUN line you want to always run. This works because ADD will always fetch the file/URL and the above URL generates random data on each request, Docker then compares the result to see if it can use the cache.
I have also tested this and works nicely since it does not require any additional Docker command line arguments and also works from a Docker-compose.yaml file :)
If your goal is to include the latest code from Github (or similar), one can use the Github API (or equivalent) to fetch information about the latest commit using an ADD command.
docker build will always fetch an URL from an ADD command, and if the response is different from the one received last time docker build ran, it will not use the subsequent cached layers.
eg.
ADD "https://api.github.com/repos/username/repo_name/commits?per_page=1" latest_commit
RUN curl -sLO "https://github.com/username/repo_name/archive/main.zip" && unzip main.zip
As of February 2016 it is not possible.
The feature has been requested at GitHub
Not directly but you can divide your Dockerfile in several parts, build an image, then FROM thisimage at the beginning of the next Dockerfile, and build the image with or without caching
the feature added a week ago.
ARG FOO=bar
FROM something
RUN echo "this won't be affected if the value of FOO changes"
ARG FOO
RUN echo "this step will be executed again if the value of FOO changes"
FROM something-else
RUN echo "this won't be affected because this stage doesn't use the FOO build-arg"
https://github.com/moby/moby/issues/1996#issuecomment-550020843
Building on #Vladislav’s solution above I used in my Dockerfile
ARG CACHEBUST=0
to invalidate the build cache from hereon.
However, instead of passing a date or some other random value, I call
docker build --build-arg CACHEBUST=`git rev-parse ${GITHUB_REF}` ...
where GITHUB_REF is a branch name (e.g. main) whose latest commit hash is used. That means that docker’s build cache is being invalidated only if the branch from which I build the image has had commits since the last run of docker build.
I believe that this is a slight improvement on #steve's answer, above:
RUN git clone https://sdk.ghwl;erjnv;wekrv;qlk#gitlab.com/your_name/your_repository.git
WORKDIR your_repository
# Calls for a random number to break the cahing of the git clone
# (https://stackoverflow.com/questions/35134713/disable-cache-for-specific-run-commands/58801213#58801213)
ADD "https://www.random.org/cgi-bin/randbyte?nbytes=10&format=h" skipcache
RUN git pull
This uses the Docker cache of the git clone, but then runs an uncached update of the repository.
It appears to work, and it is faster - but many thanks to #steve for providing the underlying principles.
Another quick hack is to write some random bytes before your command
RUN head -c 5 /dev/random > random_bytes && <run your command>
writes out 5 random bytes which will force a cache miss
For a python script I'm writing (using Python 2.7 on Windows 7) I should be able to modify a branch with a given commit, that is adding it (cherry pick) if the commit is missing, or reverting it if it's already present.
Apparently revert has not been wrapped in gitpython's Repo class, so I tried to use Git directly with:
repo.git.revert(reference)
where reference is one of the commits returned by repo.iter_commits("master")
What happens is that the script locks on that command and becomes idle; I then have to kill the command prompt window.
If I go in the working directory and explore the repository, I can see (with git diff) that after the execution, the changes have been applied even tho' no new commit is visibile if I git log.
Any ideas about if and what I'm doing wrong?
I solved the mistery by trying to git commit the applied changes manually. Git complained about a swap file in the working directory.
So, the problem was that the command was being executed as if it was run from a terminal, hence waiting for me to somehow edit the commit message! So I needed to run the revert command with the no-edit option.
I changed the method invocation to:
repo.git.revert(reference.hexsha, no_edit = True)
(notice that gitpython requires the underscore as a separator. Also, using explicitly the hexsha property is not required, since reference would be converted to its str() representation anyway.)
It seems to work.
Is there a way to use use python to rebase a repo from one on github, then push the result. As well as detecting if the rebase failed as a result of conflicts that need to be resolved?
Git is primarily a command-line tool. Once installed, you should be able to open-up a console, command prompt, powershell, c-shell, bash shell, etc. and just type git and get a list of available git commands.
Once you have Git setup and working this way, then from Python it would be possible to execute git commands in the same way you would execute any other shell commands. I'm not a Python expert, but ElpieKay suggests in the comments to use:
commands.getstatusoutput("git <command>")
You will need to do a separate search for git rebase specifically and figure out how the output is formatted and parse it to determine success, or possibly there is an error code or StdErr output that you can get through the .getstatusoutput or a similar command from commands in python.
Another thing that may help is looking at the man page for rebase with git rebase --help.
Summary
I recommend doing a search to find out more about the python commands library or just shell interaction in general for python, and then a separate set of searches/research to determine exactly how to implement the git rebase commands and its output format to determine what you need to parse to determine success or failure.
I am stuck since 2 days trying to set up a small automatic deployment script.
The thing is: I have been using Git for some months now, but I always used it locally just by myself, just with the purpose of easily saving version of my code. All good until here.
Now I have to find a way to "publish" the code as soon as new functionalities are implemented and I think the code is stable enough.
Searching around I've discovered these 'hooks', which are scripts that are executed by Git in certain situations. Basically the idea is to have my master branch sync'd with my published code, so that everytime I merge a branch to the master and 'push', the files are automatically copied into '/my/published/folder'.
That said, I've found this tutorial that explains to do exactly what I want using a 'hooks' post-receive script, which is written in Ruby. Since at my studio I don't have and don't want to use Ruby at this time, I've found a Python version of the same script.
I tested and tested, but I couldn't make it work. I keep getting the same error:
remote: GIT_WORK_TREE is not recognized as as internal or external command,
Consider this is based on the tutorial I've shared above. Same prj name, same structure, etc.
I even installed Ruby on my personal laptop and tried the original script, but it still doesn't work...
I'm using Windows, and the Git env variable is set and accessible. But nevertheless it seems like it's not recognizing the GIT_WORK_TREE command. If I run it from the Git Bash it works just fine, but if I use the Windows Shell I get the same error message.
I suppose that when in my py script use the call() function, it runs the cmd using the Windows Shell. That's my guess, but I don't really know how to solve it. Google didn't help, as if no one ever had this problem before.
Maybe I'm just not seeing something obvious here, but I spent the whole day on this and I cannot get out of this bog!
Does anyone know how to solve it, or at least have an idea for a workaround?
Hope someone can help...
Thanks a lot!
The Ruby script you are talking about generates "bash" command:
GIT_WORK_TREE=/deploy/path git checkout -f ...
It means: define environment variable "GIT_WORK_TREE" with value "/deploy/path" and execute "git checkout -f ...".
As I understand it doesn't work for Windows command line.
Try to use something like:
set GIT_WORK_TREE=c:\temp\deploy && git checkout -f ...
I've had this problem as well - the best solution I've found is to pass the working tree across as one of the parameters
git --work-tree="/deploy/path" checkout -f ...