We have multiple branches in SVN and use Hudson CI jobs to maintain our builds. We use SVN revision number as part of our application version number. The issue is when a Hudson job check out HEAD of a brach, it is getting HEAD number of SVN not last committed revision of that brach. I know, SVN maintains revision numbers globally, but we want to reflect last committed number of particular brach in our version.
is there a way to get last committed revision number of a brach using python script so that I can checkout that branch using that revision number?
or better if there a way to do it in Hudson itself?
Thanks.
Getting the last committed revision of a path using python:
from subprocess import check_output as run # >=2.7
path = './'
cmd = ['svn', '--username', XXXX, '--password', XXXX, '--non-interactive', 'info', path]
out = run(cmd).splitlines()
out = (i.split(':', 1) for i in out if i)
info = {k:v.strip() for k,v in out}
# you can access the other svn info fields in a similar manner
rev = info['Last Changed Rev']
with open('.last-svn-commit', 'w') as fh:
fh.write(rev)
I don't think the subversion scm plugin can give you the information you need (it exports SVN_URL and SVN_REVISION only). Keep in mind that there is no difference between checking out the 'Last changed Rev' and the HEAD revision - they both refer to the same content in your branch.
You might want to consider using a new job for every branch you have. This way, the commit that triggers a build will be the 'Last changed Rev' (unless you trigger it yourself). You can do this manually by cloning the trunk job and changing the repository url, or you could use a tool like jenkins-autojobs to do it automatically.
Except svn info you can also use svn log -q -l 1 URL or svn ls -v --depth empty URL
Related
If I execute the command git log --no-walk --tags --reverse in the Windows shell, I get results such as:
commit e83dcf42f2a52183cf21642c7b8deb42961663f3 (tag: my_tag)
Author: an_author
Date: a_date
while if I execute the same Git command from a Python script:
subprocess.check_output(['git', 'log', '--no-walk', '--tags', '--reverse'], stderr=subprocess.PIPE).decode(sys.stdout.encoding)
I get the following result:
commit e83dcf42f2a52183cf21642c7b8deb42961663f3
Author: an_author
Date: a_date
As you can see, the tag information is missing and as you guess, I need it. Is there anything I can do to get the tag information from the Python subprocess.check_output?
When you want to parse git output, you should use the form of its commands intended for scripting. In this case it means the --pretty=format:1 option. The %D specifier provides the ref names pointing to the commit, and of course you need to explicitly specify all the fields you want.
You might also want to use the lower level git rev-list command instead of log; it has similar options, but is promised to have stable interface for script use.
Add option --decorate:
subprocess.check_output(['git', 'log', '--decorate', '--no-walk', '--tags', '--reverse'], stderr=subprocess.PIPE).decode(sys.stdout.encoding)
I'm writing a git pre-commit hook in Python, and I'd like to define a blacklist like a .gitignore file to check files against before processing them. Is there an easy way to check whether a file is defined against a set of .gitignore rules? The rules are kind of arcane, and I'd rather not have to reimplement them.
Assuming you're in the directory containing the .gitignore file, one shell command will list all the files that are not ignored:
git ls-files
From python you can simply call:
import os
os.system("git ls-files")
and you can extract the list of files like so:
import subprocess
list_of_files = subprocess.check_output("git ls-files", shell=True).splitlines()
If you want to list the the files that are ignored (a.k.a, untracked), then you add the option '--other':
git ls-files --other
This is rather klunky, but should work:
create a temporary git repository
populate it with your proposed .gitignore
also populate it with one file per pathname
use git status --porcelain on the resulting temporary repository
empty it out (remove it entirely, or preserve it as empty for the next pass, whichever seems more appropriate).
This does, however, smell like an XY problem. The klunky solution to Y is probably a poor solution to the real problem X.
Post-comment answer with details (and side notes)
So, you have some set of files to lint, probably from inspecting the commit. The following code may be more generic than you need (we don't really need the status part in most cases) but I include it for illustration:
import subprocess
proc = subprocess.Popen(['git',
'diff-index', # use plumbing command, not user diff
'--cached', # compare index vs HEAD
'-r', # recurse into subdirectories
'--name-status', # show status & pathname
# '--diff-filter=AM', # optional: only A and M files
'-z', # use machine-readable output
'HEAD'], # the commit to compare against
stdout=subprocess.PIPE)
text = proc.stdout.read()
status = proc.wait()
# and check for failure as usual: Git returns 0 on success
Now we need something like pairwise from Iterating over every two elements in a list:
import sys
if sys.version_info[0] >= 3:
izip = zip
else:
from itertools import izip
def pairwise(it):
"s -> (s0, s1), (s2, s3), (s4, s5), ..."
a = iter(it)
return izip(a, a)
and we can break up the git status output with:
for state, path in pairwise(text.split(b'\0')):
...
We now have a state (b'A' = added, b'M' = modified, and so on) for each file. (Be sure to check for state T if you allow symlinks, in case a file changes from ordinary file to symlink, or vice versa. Note that we're depending on pairwise to discard the unpaired empty b'' string at the end of text.split(b'\0'), which is there because Git produces a NUL-terminated list rather than a NUL-separated list.)
Let's assume that at some point we collect up the files-to-maybe-lint into a list (or iterable) called candidates:
>>> candidates
[b'a.py', b'dir/b.py', b'z.py']
I will assume that you have avoided putting .gitignore into this list-or-iterable, since we plan to take it over for our own purposes.
Now we have two big problems: ignoring some files, and getting the version of those files that will actually be linted.
Just because a file is listed as modified, doesn't mean that the version in the work-tree is the version that will be committed. For instance:
$ git status
$ echo foo >> README
$ git add README
$ echo bar >> README
$ git status --short
MM README
The first M here means that the index version differs from HEAD (this is what we got from git diff-index above) while the second M here means that the index version also differs from the work-tree version.
The version that will be committed is the index version, not the work-tree version. What we need to lint is not the work-tree version but rather the index version.
So, now we need a temporary directory. The thing to use here is tempfile.mkdtemp if your Python is old, or the fancified context manager version if not. Note that we have byte-string pathnames above when working with Python3, and ordinary (string) pathnames when working with Python2, so this also is version dependent.
Since this is ordinary Python, not tricky Git interaction, I leave this part as an exercise—and I'll just gloss right over all the bytes-vs-strings pathname stuff. :-) However, for the --stdin -z bit below, note that Git will need the list of file names as b\0-separated bytes.
Once we have the (empty) temporary directory, in a format suitable for passing to cwd= in subprocess.Popen, we now need to run git checkout-index. There are a few options but let's go this way:
import os
proc = subprocess.Popen(['git', 'rev-parse', '--git-dir'],
stdout=subprocess.PIPE)
git_dir = proc.stdout.read().rstrip(b'\n')
status = proc.wait()
if status:
raise ...
if sys.version_info[0] >= 3: # XXX ugh, but don't want to getcwdb etc
git_dir = git_dir.decode('utf8')
git_dir = os.path.join(os.getcwd(), git_dir)
proc = subprocess.Popen(['git',
'--git-dir={}'.format(git_dir),
'checkout-index', '-z', '--stdin'],
stdin=subprocess.PIPE, cwd=tmpdir)
proc.stdin.write(b'\0'.join(candidates))
proc.stdin.close()
status = proc.wait()
if status:
raise ...
Now we want to write our special ignore file to os.path.join(tmpdir, '.gitignore'). Of course we also need tmpdir to act like its own Git repository now. These three things will do the trick:
import shutil
subprocess.check_call(['git', 'init'], cwd=tmpdir)
shutil.copy(os.path.join(git_dir, '.pylintignore'),
os.path.join(tmpdir, '.gitignore'))
subprocess.check_call(['git', 'add', '-A'], cwd=tmpdir)
as we will now be using Git's ignore rules with the .pylintignore file we copied to .gitignore.
Now we just would need one more git status pass (with -z for b'\0' style output, likegit diff-index`) to deal with ignored files; but there's a simpler method. We can get Git to remove all the non-ignored files:
subprocess.check_call(['git', 'clean', '-fqx'], cwd=tmpdir)
shutil.rmtree(os.path.join(tmpdir, '.git'))
os.remove(os.path.join(tmpdir, '.gitignore')
and now everything in tmpdir is precisely what we should lint.
Caveat: if your python linter needs to see imported code, you won't want to remove files. Instead, you'll want to use git status or git diff-index to compute the ignored files. Then you'll want to repeat the git checkout-index, but with the -a option, to extract all files into the temporary directory.
Once done, just remove the temp directory as usual (always clean up after yourself!).
Note that some parts of the above are tested piecewise, but assembling it all into full working Python2 or Python3 code remains an exercise.
i use CVS to code on server by eclipse local ( eclipse on win 8)
when i create a file is test.py in project CVS
but i see it is test.py,v on sever and
on local:
print 11111
on server:
head 1.1;
access;
symbols;
locks; strict;
comment ## #;
1.1
date 2013.12.07.03.35.50; author username; state Exp;
branches;
next ;
desc
##
1.1
log
#test
#
text
#print 11111#
What you're describing isn't a problem. The test.py,v file stores the complete revision history of test.py: the current version and all past versions, including information about when each version was checked in and the comment that you wrote for each commit. That's how CVS works.
CVS is old and has some major shortcomings, by the way. You should strongly consider switching to something more modern, such as Git or Subversion.
In git command line, we can use git commit -a.
How can I do that using dulwich?
When considering the various tests for a commit, like those in test_repository.py, it doesn't seem to be possible in one line:
r.stage(['a'])
commit_sha = r.do_commit('modified a',
committer='Test Committer <test#nodomain.com>',
author='Test Author <test#nodomain.com>',
commit_timestamp=12395, commit_timezone=0,
author_timestamp=12395, author_timezone=0)
You need to find modified or deleted files first, and stage them, before calling the commit.
The other alternative would be to use git-python, which is a wrapper to git, but it doesn't offer either that feature out of the box.
EDIT: This question duplicates How to access the current Subversion build number? (Thanks for the heads up, Charles!)
Hi there,
This question is similar to Getting the subversion repository number into code
The differences being:
I would like to add the revision number to Python
I want the revision of the repository (not the checked out file)
I.e. I would like to extract the Revision number from the return from 'svn info', likeso:
$ svn info
Path: .
URL: svn://localhost/B/trunk
Repository Root: svn://localhost/B
Revision: 375
Node Kind: directory
Schedule: normal
Last Changed Author: bmh
Last Changed Rev: 375
Last Changed Date: 2008-10-27 12:09:00 -0400 (Mon, 27 Oct 2008)
I want a variable with 375 (the Revision). It's easy enough with put $Rev$ into a variable to keep track of changes on a file. However, I would like to keep track of the repository's version, and I understand (and it seems based on my tests) that $Rev$ only updates when the file changes.
My initial thoughts turn to using the svn/libsvn module built in to Python, though I can't find any documentation on or examples of how to use them.
Alternatively, I've thought calling 'svn info' and regex'ing the code out, though that seems rather brutal. :)
Help would be most appreciated.
Thanks & Cheers.
There is a command called svnversion which comes with subversion and is meant to solve exactly that kind of problem.
Stolen directly from django:
def get_svn_revision(path=None):
rev = None
if path is None:
path = MODULE.__path__[0]
entries_path = '%s/.svn/entries' % path
if os.path.exists(entries_path):
entries = open(entries_path, 'r').read()
# Versions >= 7 of the entries file are flat text. The first line is
# the version number. The next set of digits after 'dir' is the revision.
if re.match('(\d+)', entries):
rev_match = re.search('\d+\s+dir\s+(\d+)', entries)
if rev_match:
rev = rev_match.groups()[0]
# Older XML versions of the file specify revision as an attribute of
# the first entries node.
else:
from xml.dom import minidom
dom = minidom.parse(entries_path)
rev = dom.getElementsByTagName('entry')[0].getAttribute('revision')
if rev:
return u'SVN-%s' % rev
return u'SVN-unknown'
Adapt as appropriate. YOu might want to change MODULE for the name of one of your codemodules.
This code has the advantage of working even if the destination system does not have subversion installed.
Python has direct bindings to libsvn, so you don't need to invoke the command line client at all. See this blog post for more details.
EDIT: You can basically do something like this:
from svn import fs, repos, core
repository = repos.open(root_path)
fs_ptr = repos.fs(repository)
youngest_revision_number = fs.youngest_rev(fs_ptr)
I use a technique very similar to this in order to show the current subversion revision number in my shell:
svnRev=$(echo "$(svn info)" | grep "^Revision" | awk -F": " '{print $2};')
echo $svnRev
It works very well for me.
Why do you want the python files to change every time the version number of the entire repository is incremented? This will make doing things like doing a diff between two files annoying if one is from the repo, and the other is from a tarball..
If you want to have a variable in one source file that can be set to the current working copy revision, and does not replay on subversion and a working copy being actually available at the time you run your program, then SubWCRev my be your solution.
There also seems to be a linux port called SVNWCRev
Both perform substitution of $WCREV$ with the highest commit level of the working copy. Other information may also be provided.
Based on CesarB's response and the link Charles provided, I've done the following:
try:
from subprocess import Popen, PIPE
_p = Popen(["svnversion", "."], stdout=PIPE)
REVISION= _p.communicate()[0]
_p = None # otherwise we get a wild exception when Django auto-reloads
except Exception, e:
print "Could not get revision number: ", e
REVISION="Unknown"
Golly Python is cool. :)