I have been at this for hours now, and although I have a feeling I'm close I can't seem to figure this out.
I'm trying to make a script that takes a git repository, updates a submodule in that repository to a specified version, and commits that change.
What works:
I can find the repository, get the submodule and check out the commit I want.
What doesn't work:
I can't seem to add the updated submodule hash so I can commit it.
My Code:
repos = Repo('path/to/repos')
submodule = repos.submodule('submodule-name')
submodule.module().git.checkout('wanted commit')
diff = repos.index.diff(None)
At this point I can see the submodule-change. If I check sourcetree, I can see the changed submodule in the 'unstaged files'.
The thing is, I have no clue how to stage the change so I can commit it.
What I have tried:
If I commit using repos.index.commit(''), it creates an empty commit.
If I try to add the path of the submodule using repos.index.add([submodule.path]), all files in the submodule are added to the repository, which is definately not what I want.
If I try to add the submodule itself (which should be possible according to the docs) using repos.index.add([submodule]), nothing seems to happen.
There are two ways to add new submodule commits to the parent repository. One will use the git command directly, the other one will be implemented in pure-python.
All examples are based on the code presented in the question.
Simple
repos.git.add(submodule.path)
repos.index.commit("updated submodule to 'wanted commit'")
The code above will invoke the git command, which is similar to doing a git add <submodule.path> in a shell.
Pythonic
submodule.binsha = submodule.module().head.commit.binsha
repos.index.add([submodule])
repos.index.commit("updated submodule to 'wanted commit'")
The IndexFile.add(...) implementation adds whichever binsha it finds in the submodule object. This one would be the one in the parent repository's current commit, and not the commit that was checked out in the submodule. One can see the Submodule object as a singular snapshot of the submodule, which does not change, nor does not it know about changes to the submodule's repository.
In this case it seems easiest to just overwrite the binsha field of the submodule object to the one that is actually checked out in its repository, so adding it to the index will have the desired effect.
Related
Similar questions like this were raised many times, but I was not able to find a solution for my specific problem.
I was playing around with setuptools_scm recently and first thought it is exactly what I need. I have it configured like this:
pyproject.toml
[build-system]
requires = ["setuptools_scm"]
build-backend = "setuptools.build_meta"
[project]
...
dynamic = ["version"]
[tool.setuptools_scm]
write_to = "src/hello_python/_version.py"
version_scheme = "python-simplified-semver"
and my __init__.py
from ._version import __version__
from ._version import __version_tuple__
Relevant features it covers for me:
I can use semantic versioning
it is able to use *.*.*.devN version strings
it increments minor version in case of feature-branches
it increments patch/micro version in case of fix-branches
This is all cool. As long as I am on my feature-branch I am able to get the correct version strings.
What I like particularly is, that the dev version string contains the commit hash and is thus unique across multiple branches.
My workflow now looks like this:
create feature or fix branch
commit, (push, ) publish
merge PR to develop-branch
As soon as I am on my feature-branch I am able to run python -m build which generated a new _version.py with the correct version string accordingly to the latest git tag found. If I add new commits, it is fine, as the devN part of the version string changes due to the commit hash. I would even be able to run a python -m twine upload dist/* now. My package is build with correct version, so I simply publish it. This works perfectly fine localy and on CI for both fix and feature branches alike.
The problem that I am facing now, is, that I need a slightly different behavior for my merged PullRequests
As soon as I merge, e.g. 0.0.1.dev####, I want to run my Jenkins job not on the feature-branch anymore, but instead on develop-branch. And the important part now is, I want to
get develop-branch (done by CI)
update version string to same as on branch but without devN, so: 0.0.1
build and publish
In fact, setuptools_scm is changing the version to 0.0.2.dev### now, and I would like to have 0.0.1.
I was tinkering a bit with creating git tags before running setuptools_scm or build, but I was not able to get the correct version string to put into the tag. At this point I am struggling now.
Is anyone aware of a solution to tackle my issue with having?:
minor increment on feature-branches + add .devN
patch/micro increment on fix-branches + add .devN
no increment on develop-branch and version string only containing major.minor.patch of merged branch
TLDR: turning off to write the version number to a file every time setuptools_scm runs could maybe solve your problem, alternatively add the version file to .gitignore.
Explanation:
I also just started using setuptools_scm, so I am not very confident in using it yet.
But, as far as I understand the logic to derive the version number is incremented according to the state of your repository (the detailed logic is documented here: https://github.com/pypa/setuptools_scm/#default-versioning-scheme).
When I am not mistaken, the tool now does exactly what it is expected to: it does NOT set a version name only derived from the tag, but also adds a devSomething because the tag you've set is not referencing the most current commit on the develop branch head in your case.
Also I had the problem that when letting setuptools_scm generate a version and also configuring it to write it to a file, this would lead to another state since the last commit, again generating a dev version number.
To get a "clean" (e.g. v0.0.1) version number I hat to do the tagging after merging (with merge commit) since the merge commit was also taken into account for the version numbering logic.
Still my setup is currently less complex than yours. Just feature and fix branches and just a main branch without develop. So fewer merge commits (I chose to do merge commits, so no linear history). Now after merging with commit I create a Tag manually and formulate its name myself.
And this also only works for me in case I opt-out for writing the version number into a file. This I have done by inserting the following into pyproject.toml:
[tool.setuptools_scm]
# intentionally empty/commented out
# write_to option leads to an unclean workspace during build
# which again is leading setuptools_scm to interpret this during build and producing wheels with unclean version numbers
# write_to = "version.txt"
Since setuptools_scm runs during build, a new version file is also generated, which pollutes your worktree. Since your worktree will never be clean this way, you always get a dev version number. To still have a version file and to let it ignore during build, add the file to your .gitignore.
My approach is not perfect, some manual steps, but for now it works for me.
Certainly not 100% applicable in your CI scenario, but maybe you could change the order of doing merges and tags. I hope this helps somehow.
I clone the 'Apache/tomcat' git repo to use some info about commit.
However, when i use git.repo('repo local address').iter_commits(), i can't get some commits.
Besides, I can't search these in github search engine.
For example, commit 69c56080fb3355507e1b55d014ec0ee6767a6150 is in the 'Apache tomcat' repo, however, search '69c56080fb3355507e1b55d014ec0ee6767a6150' in 'in this repository' get nothing.
It's amazing for me.
It seems like that the commit isn't in the master branch, so can't be searched?
I want to know the theory behind this and how to get info about these 'missing' commits in Python.
Thanks.
repo.iter_commits(), with no arguments, gives you the commits which can be reached by tracing back through the parent(s) of the current commit. In other words, if you are in the master branch, it will only give you commits that are part of the master branch.
You can give it a rev argument which, among other things, can be a branch name. For example, iter_commits(rev='8.5.x') ought to give you all commits in the 8.5.x branch, which will include 69c5608. You can use another function, repo.branches(), if you need to get a list of branches.
Alternatively, if you already know the hash of a single commit that you want to look up, you can use repo.commit(), again with a rev parameter which in this case is the full or abbreviated commit hash: commit(rev='69c5608').
I believe the issue here is that this commit is in the branch 8.5.x and not master. You can see this in the first link. It will show which branches include it. The GitHub search algorithm only searches the master/main/trunk branch.
To find it via git python library, try changing to that branch. See these instructions on how to switch branches: https://gitpython.readthedocs.io/en/stable/tutorial.html#switching-branches
I want to use django-achievements (link) module in my app, but it lack some fields in it's model. For example, I want to add CharField to it with path to picture of the badge/achievement. Also I will need to modify module's engine.py file for that.
What is the right way to do that? Download that module to my main app' folder and modify original files, or i can somehow redefine some methods/classes of original models.py and engine.py locally without modifing original files?
I'd say fork it and make your own modifications directly to the source. If it's an improvement you can create a Pull Request and contribute your code to the actual repository (not required, you can always just keep it for your own use).
I'm looking for a way to extend Python Commit Hooks such that I can ONLY find out all the files that were modified excluding all the revision properties changed.
Is there a SVN.Core or SVN.fs or another SVN import lib function that I could use?
I'm currently looking into
svn_fs_txn_prop
but haven't had much luck.
Thanks in Advance.
Using Subversion Core library in Python you can create a class to read through different Subversion Properties associated with a commit.
you can call something like:
changeCollector = svn.repos.ChangeCollector(self.fs_obj,self.fsroot,self.pool);
for path, change in changeCollector.changes.items():
# this is the property that i want
change.text_changed
I want to try out a patch on gist that modifies the source code of Django:
gist: 550436
How do I do it? I have never used git so a step by step instruction would be greatly appreciated.
You can use patch to apply diffs. Make sure you're in your django source directory (or wherever you want to apply the patch), and run something like patch -p1 < downloaded-patch.diff.
You may want to experiment with the -p argument if it fails; -p tells patch to strip some of the directory prefix for each file in the diff (look at the first line in the diff).