We Keep Coding

How can I check poetry lockfile freshness without importing poetry

We've been using pipenv for dependency management for a while, and using micropipenv's protected functionality to check lock freshness - the idea here being that micropipenv is lightweight, so this is a cheap and cheerful way of ensuring that our dependencies haven't drifted during CI or during a docker build.
Alas, micropipenv has no such feature for poetry (it skips the hash check completely), and I am therefore left to "reverse-engineer" the feature on my own. Ostensibly this should be super easy - I've assembled the code posted later from what I traced through the poetry and poetry-core repos (Locker, Factory, core.Factory, and PyProjectTOML, primarily). This absolutely does not do the trick, and I'm at a loss as to why.
_relevant_keys = ["dependencies", "group", "source", "extras"]
def _get_content_hash(pyproject):
content = pyproject["tool"]["poetry"]
print(content)
relevant_content = {}
for key in _relevant_keys:
relevant_content[key] = content.get(key)
print(json.dumps(relevant_content, sort_keys=True).encode())
content_hash = sha256(
json.dumps(relevant_content, sort_keys=True).encode()
).hexdigest()
print(f"Calculated: {content_hash}")
return content_hash
def is_fresh(lockfile, pyproject):
metadata = lockfile.get("metadata", {})
print(f"From file: {lockfile['metadata']['content-hash']}")
if "content-hash" in metadata:
return _get_content_hash(pyproject) == lockfile["metadata"]["content-hash"]
return False
Would love to figure out what exactly the heck I'm missing here - i'm guessing that the poetry locker _local_config gets changed at some point and I've failed to notice it.
References:
Locker: https://github.com/python-poetry/poetry/blob/a1a5bce96d85bdc0fdc60b8abf644615647f969e/poetry/packages/locker.py#L454
core.Factory: https://github.com/python-poetry/poetry-core/blob/afaa6903f654b695d9411fb548ad10630287c19f/poetry/core/factory.py#L24

Naturally, this ended up being a PEBKAC error. I was using the hash generation function from the master branch but using an earlier version of poetry on the command line. Once I used the function from the correct code version, everything was hunky dory.
I think this functionality actually exists in micropipenv now anyways lol

HEAD detatched from origin/master when merging with pygit2

I am using pygit2 to merge some branches of a project, however whenever I merge them I end up with:
def avoid_walls(directions, board, snake):
<<<<<<< HEAD
return directions
=======
z = 1
moves = []
for direction in directions:
new_x = snake[0][0] + dirs[direction][0]
new_y = snake[0][1] + dirs[direction][1]
if new_x >= 0 and new_x < len(board[0]) and new_y >= 0 and new_y < len(board):
moves.append(direction)
return moves
>>>>>>> 357f58b6d1682760b2fa9bf7b2418da347ca353c
in my code. When I check the repo with 'git status' I find this:
HEAD detached from origin/master
All conflicts fixed but you are still merging.
(use "git commit" to conclude merge)
As far as I can tell I am doing everything properly, so I cant figure out why the HEAD is detached. From my understanding, the HEAD is detached when you check out a specific commit, rather than a branch, which I am not doing:
# clone repo to local directory
repo = pygit2.clone_repository(my_repo, my_dir)
# checkout master branch (checked out by default, but just to be safe)
repo.checkout('refs/remotes/origin/master')
# merge with other branches
repo.merge(repo.branches['origin/branch1'].target)
# commit changes
index = repo.index
index.add_all()
index.write()
author = pygit2.Signature("ME", "me#domain.com")
commiter = pygit2.Signature("ME", "me#domain.com")
tree = index.write_tree()
oid = repo.create_commit('refs/heads/master', author, commiter, "init commit", tree, [repo.head.target, repo.branches['origin/branch1'].target])
#some tests i tried to fix the issue
repo.head.set_target(oid)
repo.apply(oid)
Am I missing something after the merge that will complete the commit and resolve this issue?

refs/remotes/origin/master is not a branch. All branches start with refs/heads/:
if name.startswith('refs/heads/'):
print('{} is a branch name'.format(name))
else
print('{} is not a branch name'.format(name))
In this case, since it starts with refs/remotes/, it's a remote-tracking name (Git documentation generally calls this a remote-tracking branch name, but I think this is too misleading since that contains the word branch even though it's not a branch name).
When you check out a remote-tracking name, or a tag name, or any name that is not a branch name, you are really checking out a specific commit and will get a detached HEAD.

Git merge fails unexpected

I've implemented automated merging script in Python. It should merge branches automatically and create pull requests if there are some merging conflicts.
Part of my script looks like:
from subprocess import check_call, CalledProcessError
# E.g. destination_branch = "master", source_branch = "release"
try:
check_call(['git', 'checkout', '%s' % destination_branch])
check_call(['git', 'merge', '--no-ff', '%s' % source_branch])
except CalledProcessError:
# Creating pull request.
It looks like everything is good, but there are some issues here.
After some automated merges I've got following errors:
error: you need to resolve your current index first
Dockerfile: needs merge. Also I'm printing status code of these two step. Status code is 1, which is not good.
As a result, I can see too many pull requests, most of them do not have any merging conflicts.
What is wrong here?
UPDATE:
Before merging something I also have a stuff for branch updating (to keep up-to-date version).
It looks like:
try:
check_call(['git', 'checkout', str(source_branch)])
# branch successfully checked out
check_call(['git', 'pull', 'origin', str(source_branch)])
except CalledProcessError:
# Logging an errors.
One important thing to add:
As guys: #torek, #MarkAdelsberger mentioned in their comments, I've tried also their solution with adding git merge --abort command after failed merging for some reason.
So, it doesn't help. It fails with another error:
check_call([GIT_CMD, 'merge', '--abort'])
File "/usr/lib/python2.7/subprocess.py", line 540, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['git', 'merge', '--abort']' returned non-zero exit status 128
So, I'm looking for some solution again....
Ideas?

An error message of the form you need to resolve your current index first means you, earlier, started a regular (non-fast-forward) merge and it failed with a merge conflict.
When git merge fails with a merge conflict, Git produces a status 1 exit. The check_call code will raise CalledProcessError if that particular failed merge occurred as part of this code, but all we know for sure is that there was such a failed merge earlier, not that it happened right here:
subprocess.CalledProcessError: Command '...' returned non-zero exit status 1
Once this happens—whether as part of your Python code, or by some other means—the repository's work-tree and index are left in this "merge failed, hand treatment required" state. If you don't know the repository's state in advance, you should find it out before you start, e.g., using git status --porcelain or git status --porcelain=v2 (see the documentation for git status; note that this web version does not show the new v2 format, which was first release in Git 2.11).
If you have put the work-tree and index into this partially merged, intermediate, requires-hand-treatment state and wish to restore it to the pre-merge state, simply run git merge --abort. Do not attempt a commit or a pull request from this half-vast1 state: the merge must be either finished, or aborted entirely.
Aside: '%s' % expr is silly, just use str(expr), or expr itself if it is already known to be a string.
1Say "half-vast" aloud, three times, fast. :-)

git merge --abort command after failed merging for some reason
Git 2.38 (Q3 2022) hould help: when "git merge"(man) finds that it cannot perform a merge, it should restore the working tree to the state before the command was initiated, but in some corner cases it didn't (before 2.38).
See commit c23fc07, commit 034195e, commit aa77ce8, commit 1369f14, commit 8f240b8, commit e4cdfe8, commit 24ba8b7, commit 11f4290 (23 Jul 2022) by Elijah Newren (newren).
(Merged by Junio C Hamano -- gitster -- in commit 966ff64, 03 Aug 2022)
merge: do not exit restore_state() prematurely
Reported-by: ZheNing Hu
Signed-off-by: Elijah Newren
Previously, if the user:
Had no local changes before starting the merge
A merge strategy makes changes to the working tree/index but returns with exit status 2
Then we'd call restore_state() to clean up the changes and either let the next merge strategy run (if there is one), or exit telling the user that no merge strategy could handle the merge.
Unfortunately, restore_state() did not clean up the changes as expected; that function was a no-op if the stash was a null, and the stash would be null if there were no local changes before starting the merge.
So, instead of "Rewinding the tree to pristine..." as the code claimed, restore_state() would leave garbage around in the index and working tree (possibly including conflicts) for either the next merge strategy or for the user after aborting the merge.
And in the case of aborting the merge, the user would be unable to run "git merge --abort"(man) to get rid of the unintended leftover conflicts, because the merge control files were not written as it was presumed that we had restored to a clean state already.
Fix the main problem by making sure that restore_state() only skips the stash application if the stash is null rather than skipping the whole function.
However, there is a secondary problem -- since merge.c forks subprocesses to do the cleanup, the in-memory index is left out-of-sync.
While there was a refresh_cache(REFRESH_QUIET) call that attempted to correct that, that function would not handle cases where the previous merge strategy added conflicted entries.
We need to drop the index and re-read it to handle such cases.
(Alternatively, we could stop forking subprocesses and instead call some appropriate function to do the work which would update the in-memory index automatically.
For now, just do the simple fix.)
Also, add a testcase checking this, one for which the octopus strategy fails on the first commit it attempts to merge, and thus which it cannot handle at all and must completely bail on (as per the "exit 2" code path of commit 98efc8f ("octopus: allow manual resolve on the last round.", 2006-01-13, Git v1.2.0 -- merge)).

Diff commit messages of two branches with gitpython

At work, we have a workflow where each branch is "named" by date. During the week, at least once, the latest branch gets pushed to production. What we require now is the summary/commit messages of the changes between the latest branch in production vs the new branch via gitpython.
What I have tried to do:
import git
g = git.Git("pathToRepo")
r = git.Repo("pathToRepo")
g.pull() # get latest
b1commits = r.git.log("branch1")
b2commits = r.git.log("branch2")
This give me all of the commit history from both branches but I can't figure out how to compare them to just get the newest commit messages.
Is this possible to do in gitPython? Or is there a better solution?

I figured it out:
import git
g = git.Git(repoPath+repoName)
g.pull()
commitMessages = g.log('%s..%s' % (oldBranch, newBranch), '--pretty=format:%ad %an - %s', '--abbrev-commit')
Reading through the Git documentation I found that I can compare two branches with this syntax B1..B2. I tried the same with gitpython and it worked, the other parameters are there for a custom format.

This solution uses GitPython
import git
def get_commit_from_range(start_commit, end_commit):
repo = git.Repo('path')
commit_range = f"{start_commit}...{end_commit}"
result = repo.iter_commits(commit_range)
for commit in result:
print(commit.message)

How to do a git reset --hard using gitPython?

Well the title is self explanatory. What will be the python code equivalent to running git reset --hard (on terminal) using GitPython module?

You can use:
repo = git.Repo('c:/SomeRepo')
repo.git.reset('--hard')
Or if you need to reset to a specific branch:
repo.git.reset('--hard','origin/master')
Or in my case, if you want to just hard update a repo to origin/master (warning, this will nuke your current changes):
# blast any current changes
repo.git.reset('--hard')
# ensure master is checked out
repo.heads.master.checkout()
# blast any changes there (only if it wasn't checked out)
repo.git.reset('--hard')
# remove any extra non-tracked files (.pyc, etc)
repo.git.clean('-xdf')
# pull in the changes from from the remote
repo.remotes.origin.pull()

I searched for reset in the documentation and found this:
class git.refs.head.HEAD(repo, path='HEAD')
reset(commit='HEAD', index=True, working_tree=False, paths=None, **kwargs)
Reset our HEAD to the given commit optionally synchronizing the index and working tree. The reference we refer to will be set to commit as well.

You can use:
repo = git.Repo('repo')
# ...
# Remove last commit
repo.head.reset('HEAD~1', index=True, working_tree=True)