I am trying to have 2 simultaneous versions of a single package on a server. A production and a testing one.
I want these 2 to be on the same git repository on 2 different branches (The testing would merge into production) however i would love to keep them in the same directory so its not needed to change any imports or paths.
Is it possible to dynamically change the package name in setup.py, depending on the git branch?
Or is it possible to deploy them with different names using pip?
EDIT : i may have found a proper solution for my problem here : Git: ignore some files during a merge (keep some files restricted to one branch)
Gitattributes can be setup to ignore merging of my setup.py, ill close this question after i test it.
This could be done with a setup script that looks like this:
#!/usr/bin/env python3
import pathlib
import setuptools
def _get_git_ref():
ref = None
git_head_path = pathlib.Path(__file__).parent.joinpath('.git', 'HEAD')
with git_head_path.open('r') as git_head:
ref = git_head.readline().split()[-1]
return ref
def _get_project_name():
name_map = {
'refs/heads/master': 'ThingProd',
'refs/heads/develop': 'ThingTest',
}
git_ref = _get_git_ref()
name = name_map.get(git_ref, 'ThingUnknown')
return name
setuptools.setup(
# see 'setup.cfg'
name=_get_project_name(),
)
It reads the current git ref directly from the .git/HEAD file and looks up the corresponding name in a table.
Inspired from: https://stackoverflow.com/a/56245722/11138259.
Using .gitattributes file with content "setup.py merge=ours" and also setting up the git config --global merge.ours.driver true. Makes the merge "omit" the setup.py file (it keeps our file instead). This works only if both master and child branch have changed the file since they firstly parted ways. (it seems)
Related
I am working on a python package and in order to format imports I run isort on all the python files. I would like iSort to skip __init__.py files as in some (rare) cases the order of the imports is critical and does not line up with isort's ordering scheme.
I tried playing around with different configuration options in my pyproject.toml file and I believe I am looking for the extend_skip_glob configuration option. However, the issue is I am unable to figure out a glob pattern that matches any __init__.py file located in any subdirectory of src (where the code lives) located at any depth (I believe that's part of the issue I am experiencing).
I have tried a few combinations listed below, but none of them appear to be working.
[tool.isort]
# option 1
extend_skip_glob = ["__init__.py"]
# option 2
extend_skip_glob = ["src/**/__init__.py"]
# option 3
extend_skip_glob = ["src/**/*init__.py"]
Has anybody encountered something like this before and figured out a way to solve it?
I think you should use skip instead. In pyproject.toml file for example,
[tool.isort]
skip = ["__init__.py"]
The repo structure looks like this:
- folder_a
- folder_b
- folder_c
- ...
I am particularly interested in the files that changed in a specific commit, but only the ones in folder_a. My solution is
for filename, details in commit.stats.files.items():
if not filename.startswith('folder_a'):
continue
# ...
but it seems the performance is not quite good if there are a great number of files in the other folders. Is there any better way to skip the files I don't care about?
If I understand correctly : you want stats on modifications from a commit, only on one specific subfolder.
Using plain git :
git show [commit] --stat folder_a
will display exactly what you want.
Have a look at what : git.show('<commit identifier>', '--stat', 'folder_a'); returns in your python script.
I am using gitpython library for performing git operations, retrieve git info from python code. I want to retrieve all revisions for a specific file. But couldn't find a specific reference for this on the docs.
Can anybody give some clue on which function will help in this regard? Thanks.
A follow-on, to read each file:
import git
repo = git.Repo()
path = "file_you_are_looking_for"
revlist = (
(commit, (commit.tree / path).data_stream.read())
for commit in repo.iter_commits(paths=path)
)
for commit, filecontents in revlist:
...
There is no such function, but it is easily implemented:
import git
repo = git.Repo()
path = "dir/file_you_are_looking_for"
commits_touching_path = list(repo.iter_commits(paths=path))
Performance will be moderate even if multiple paths are involved. Benchmarks and more code about that can be found in an issue on github.
I have a subversion repo ie "http://crsvn/trunk/foo" ... I want to walk this directory or for starters simply to a directory list.
The idea is to create a script that will do mergeinfo on all the branches in "http://crsvn/branches/bar" and compare them to trunk to see if the branch has been merged.
So the first problem I have is that I cannot walk or do
os.listdir('http://crsvn/branches/bar')
I get the value label syntax is incorrect (mentioning the URL)
You can use PySVN. In particular, the pysvn.Client.list method should do what you want:
import pysvn
svncl = pysvn.Client()
entries = svncl.list("http://rabbitvcs.googlecode.com/svn/trunk/")
# Gives you a list of directories:
dirs = (entry[0].repos_path for entry in entries if entry[0].kind == pysvn.node_kind.dir)
list(dirs)
No checkout needed. You could even specify a revision to work on, to ensure your script can ignore other people working on the repository while it runs.
listdir takes a path and not a url. It would be nice if python could be aware of the structure on a remote server but i don't think that is the case.
If you were to checkout your repository locally first you could easly walk the directories using pythons functions.
How do I specify the header files in a setup.py script for a Python extension module? Listing them with source files as follows does not work. But I can not figure out where else to list them.
from distutils.core import setup, Extension
from glob import glob
setup(
name = "Foo",
version = "0.1.0",
ext_modules = [Extension('Foo', glob('Foo/*.cpp') + glob('Foo/*.h'))]
)
Add MANIFEST.in file besides setup.py with following contents:
graft relative/path/to/directory/of/your/headers/
Try the headers kwarg to setup(). I don't know that it's documented anywhere, but it works.
setup(name='mypkg', ..., headers=['src/includes/header.h'])
I've had so much trouble with setuptools it's not even funny anymore.
Here's how I ended up having to use a workaround in order to produce a working source distribution with header files: I used package_data.
I'm sharing this in order to potentially save someone else the aggravation. If you know a better working solution, let me know.
See here for details:
https://bitbucket.org/blais/beancount/src/ccb3721a7811a042661814a6778cca1c42433d64/setup.py?fileviewer=file-view-default#setup.py-36
# A note about setuptools: It's profoundly BROKEN.
#
# - The header files are needed in order to distribution a working
# source distribution.
# - Listing the header files under the extension "sources" fails to
# build; distutils cannot make out the file type.
# - Listing them as "headers" makes them ignored; extra options to
# Extension() appear to be ignored silently.
# - Listing them under setup()'s "headers" makes it recognize them, but
# they do not get included.
# - Listing them with "include_dirs" of the Extension fails as well.
#
# The only way I managed to get this working is by working around and
# including them as "packaged data" (see {63fc8d84d30a} below). That
# includes the header files in the sdist, and a source distribution can
# be installed using pip3 (and be built locally). However, the header
# files end up being installed next to the pure Python files in the
# output. This is the sorry situation we're living in, but it works.
There's a corresponding ticket in my OSS project:
https://bitbucket.org/blais/beancount/issues/72
If I remember right you should only need to specify the source files and it's supposed to find/use the headers.
In the setup-tools manual, I see something about this I believe.
"For example, if your extension requires header files in the include directory under your distribution root, use the include_dirs option"
Extension('foo', ['foo.c'], include_dirs=['include'])
http://docs.python.org/distutils/setupscript.html#preprocessor-options