We Keep Coding

Unable to get complete git history of file with python

I am trying to get all the content from each revision - history of a file in my local repo i am using gitpython lib and here is the code:
import git,json
from pprint import pprint
repo = git.Repo()
path = "my_file_path"
revlist = (
(commit, (commit.tree / path).data_stream.read())
for commit in repo.iter_commits(paths=path)
)
for commit, filecontents in revlist:
filecontentsjs = json.loads(filecontents)
pprint(commit)
pprint(filecontentsjs["execution_status"])
pprint(filecontentsjs["execution_end_time"])
Problem: i am comparing our bitbucket history and the history i get from this script and the script comes up short, meaning the bitbucket history has more revisions of the file but when i clone the repo locally i get like half of the revisions with the script
am i missing something here? limitation or anything like that?

so it turns out it was my fault i didn't notice that the filename changed slightly and bitbucket did what it supposed to do since it thought "if the code is the same the file is the same" which is not true
so adding the --follow flag in the git log i was seeing the full "bad" history.
The real "good" history is without the --follow since i care only about the file with the same name

Why are Git log command outputs different when launched from Windows shell and from Python subprocess?

If I execute the command git log --no-walk --tags --reverse in the Windows shell, I get results such as:
commit e83dcf42f2a52183cf21642c7b8deb42961663f3 (tag: my_tag)
Author: an_author
Date: a_date
while if I execute the same Git command from a Python script:
subprocess.check_output(['git', 'log', '--no-walk', '--tags', '--reverse'], stderr=subprocess.PIPE).decode(sys.stdout.encoding)
I get the following result:
commit e83dcf42f2a52183cf21642c7b8deb42961663f3
Author: an_author
Date: a_date
As you can see, the tag information is missing and as you guess, I need it. Is there anything I can do to get the tag information from the Python subprocess.check_output?

When you want to parse git output, you should use the form of its commands intended for scripting. In this case it means the --pretty=format:1 option. The %D specifier provides the ref names pointing to the commit, and of course you need to explicitly specify all the fields you want.
You might also want to use the lower level git rev-list command instead of log; it has similar options, but is promised to have stable interface for script use.

Add option --decorate:
subprocess.check_output(['git', 'log', '--decorate', '--no-walk', '--tags', '--reverse'], stderr=subprocess.PIPE).decode(sys.stdout.encoding)

Add files to Xcode project through command line ? Use of project.pbxproj file in Xcode?

I was trying to add a plist file to a xcode project through command line, some of the blogs suggested to edit the project.pbxproj file. I searched about the project.pbxproj file but was not able to get much information about it. Can anyone let me know what is use of the project.pbxproj file in xcode? How does one add entries to it?
I am using this repo to work with it.
the script that I wrote is as follows:
import sys
import os
from mod_pbxproj import XcodeProject
def addPlistInProject(corodova_proj_name, xcode_proj_name, plist_file_name):
print "Cordova project name : " + corodova_proj_name
present_directory = os.getcwd()
path_to_xcode_proj = present_directory + '/' + corodova_proj_name + '/platforms/ios/' + xcode_proj_name + '.xcodeproj/project.pbxproj'
print "Xcode Project Path : " + path_to_xcode_proj
project = XcodeProject.Load(path_to_xcode_proj)
project.get_or_create_group('new group')
project.add_file(plist_file_name)
if __name__ == "__main__":
corodova_proj_name = sys.argv[1]
xcode_proj_name = sys.argv[2]
plist_file_name = sys.argv[3]
print "Xcode Project Name = : " + xcode_proj_name
print "Plist File Path = : " + plist_file_name
addPlistInProject(corodova_proj_name, xcode_proj_name, plist_file_name)
I will be invoking the script as:
python myscript.py hello HelloWorld manisha-rules_camdo.plist
myscript.py is the script I wrote, hello is the existing cordova project and HelloWorld is the Xcode project created by using cordova platform add iOS.
The command Sequence I will be following will be as follows:
cordova create hello com.example.hello HelloWorld
cordova platform add iOS
py myscript.py hello HelloWorld manisha-rules_camdo.plist
Where hello is the name of cordova project and HelloWorld name of iOS target.

There is a Ruby API from Cocoapods for editing Xcode projects. It also have an active community of developers:
https://github.com/CocoaPods/Xcodeproj

Another great option, especially for Cordova projects, is to use the XCODE node module: node-xcode; you can easily add it via NPM.
Once in place, you can create an after_prepare hook to modify the pbxproj, injecting custom source files, additional frameworks, etc, on every build. In fact, Cordova itself leverages this module during its own build processes.
Within my solution, I first added the module via npm:
npm install xcode --save-dev
And then I created and after_prepare hook to add extra frameworks into my XCode project:
var xcode = require('xcode'),
fs = require('fs'),
rootdir = process.argv[2],
projectPath = rootdir + '/platforms/ios/my-project/project.pbxproj',
proj = new xcode.project(projectPath);
proj.parse(function(err) {
if (err) {
console.log("Oh noes! XCODE project failed to parse:");
console.log(err);
} else {
proj.addFramework('Fabric.framework', {customFramework:true});
proj.addFramework('Crashlytics.framework', {customFramework:true});
proj.addFramework('AdSupport.framework');
proj.addFramework('FacebookSDK.framework', {customFramework:true});
fs.writeFileSync(projectPath, proj.writeSync());
console.log("Updated XCODE project with references to social libs!");
}
});
The XCODE module is smart enough to know if the frameworks / files / etc that you are trying to add are already present, and won't try to add them again.

What you are asking to do is not the most straightforward thing. The Xcode pbxproj file format looks like XML, but I think there's quite a few proprietary / undocumented pieces to it (much like everything iOS). As far as I can tell, Xcode doesn't have any way to add files from the command line.
I did find a Python script that you might be able to use to modify Xcode's project files, but it's a few years old and it might be out of date.
Here is the Blog post that talks about it and here is the current GitHub repo (last updated five months ago, as of the date of me typing this answer).
Give this a try and let me know if it works for you.

Using actual branch head revision number in Hudson

We have multiple branches in SVN and use Hudson CI jobs to maintain our builds. We use SVN revision number as part of our application version number. The issue is when a Hudson job check out HEAD of a brach, it is getting HEAD number of SVN not last committed revision of that brach. I know, SVN maintains revision numbers globally, but we want to reflect last committed number of particular brach in our version.
is there a way to get last committed revision number of a brach using python script so that I can checkout that branch using that revision number?
or better if there a way to do it in Hudson itself?
Thanks.

Getting the last committed revision of a path using python:
from subprocess import check_output as run # >=2.7
path = './'
cmd = ['svn', '--username', XXXX, '--password', XXXX, '--non-interactive', 'info', path]
out = run(cmd).splitlines()
out = (i.split(':', 1) for i in out if i)
info = {k:v.strip() for k,v in out}
# you can access the other svn info fields in a similar manner
rev = info['Last Changed Rev']
with open('.last-svn-commit', 'w') as fh:
fh.write(rev)
I don't think the subversion scm plugin can give you the information you need (it exports SVN_URL and SVN_REVISION only). Keep in mind that there is no difference between checking out the 'Last changed Rev' and the HEAD revision - they both refer to the same content in your branch.
You might want to consider using a new job for every branch you have. This way, the commit that triggers a build will be the 'Last changed Rev' (unless you trigger it yourself). You can do this manually by cloning the trunk job and changing the repository url, or you could use a tool like jenkins-autojobs to do it automatically.

Except svn info you can also use svn log -q -l 1 URL or svn ls -v --depth empty URL

How does one add a svn repository build number to Python code?

EDIT: This question duplicates How to access the current Subversion build number? (Thanks for the heads up, Charles!)
Hi there,
This question is similar to Getting the subversion repository number into code
The differences being:
I would like to add the revision number to Python
I want the revision of the repository (not the checked out file)
I.e. I would like to extract the Revision number from the return from 'svn info', likeso:
$ svn info
Path: .
URL: svn://localhost/B/trunk
Repository Root: svn://localhost/B
Revision: 375
Node Kind: directory
Schedule: normal
Last Changed Author: bmh
Last Changed Rev: 375
Last Changed Date: 2008-10-27 12:09:00 -0400 (Mon, 27 Oct 2008)
I want a variable with 375 (the Revision). It's easy enough with put $Rev$ into a variable to keep track of changes on a file. However, I would like to keep track of the repository's version, and I understand (and it seems based on my tests) that $Rev$ only updates when the file changes.
My initial thoughts turn to using the svn/libsvn module built in to Python, though I can't find any documentation on or examples of how to use them.
Alternatively, I've thought calling 'svn info' and regex'ing the code out, though that seems rather brutal. :)
Help would be most appreciated.
Thanks & Cheers.

There is a command called svnversion which comes with subversion and is meant to solve exactly that kind of problem.

Stolen directly from django:
def get_svn_revision(path=None):
rev = None
if path is None:
path = MODULE.__path__[0]
entries_path = '%s/.svn/entries' % path
if os.path.exists(entries_path):
entries = open(entries_path, 'r').read()
# Versions >= 7 of the entries file are flat text. The first line is
# the version number. The next set of digits after 'dir' is the revision.
if re.match('(\d+)', entries):
rev_match = re.search('\d+\s+dir\s+(\d+)', entries)
if rev_match:
rev = rev_match.groups()[0]
# Older XML versions of the file specify revision as an attribute of
# the first entries node.
else:
from xml.dom import minidom
dom = minidom.parse(entries_path)
rev = dom.getElementsByTagName('entry')[0].getAttribute('revision')
if rev:
return u'SVN-%s' % rev
return u'SVN-unknown'
Adapt as appropriate. YOu might want to change MODULE for the name of one of your codemodules.
This code has the advantage of working even if the destination system does not have subversion installed.

Python has direct bindings to libsvn, so you don't need to invoke the command line client at all. See this blog post for more details.
EDIT: You can basically do something like this:
from svn import fs, repos, core
repository = repos.open(root_path)
fs_ptr = repos.fs(repository)
youngest_revision_number = fs.youngest_rev(fs_ptr)

I use a technique very similar to this in order to show the current subversion revision number in my shell:
svnRev=$(echo "$(svn info)" | grep "^Revision" | awk -F": " '{print $2};')
echo $svnRev
It works very well for me.
Why do you want the python files to change every time the version number of the entire repository is incremented? This will make doing things like doing a diff between two files annoying if one is from the repo, and the other is from a tarball..

If you want to have a variable in one source file that can be set to the current working copy revision, and does not replay on subversion and a working copy being actually available at the time you run your program, then SubWCRev my be your solution.
There also seems to be a linux port called SVNWCRev
Both perform substitution of $WCREV$ with the highest commit level of the working copy. Other information may also be provided.

Based on CesarB's response and the link Charles provided, I've done the following:
try:
from subprocess import Popen, PIPE
_p = Popen(["svnversion", "."], stdout=PIPE)
REVISION= _p.communicate()[0]
_p = None # otherwise we get a wild exception when Django auto-reloads
except Exception, e:
print "Could not get revision number: ", e
REVISION="Unknown"
Golly Python is cool. :)