We are using perforce for our version control. So when i go to build a package
python setup.py sdist
it creates a versioned folder with everything in it and then tries to delete it and fails because it contains a bunch of read-only files. It also fails when trying to write to setup.cfg when that isn't checked out.
What is the proper way to do this from a perforce repo or any package that may have read-only files? Do i have to check out everything first and revert unchanged?
Thanks
Related
I am attempting to reupload everything in my local site-packages folder into a docker container running pypi-server. This is ultimately to download them again into some container using pip instead of some internal tooling that is not idea for python.
The content of my site-packages is mostly items like:
package_a
package_a.egg-info
package_b
package_b.egg-info
Is there a way to convert or simply copy these into the pypi-server? I think I'm missing setup.py and setup.cfg and was curious if there's a way to recover those in site-packages?
Docker image: https://hub.docker.com/r/pypiserver/pypiserver
I grabbed my project (which contains a setup.py) and used python setup.py sdist and twine upload --repoisotory-url <PATH> dist/* per the docker instructions. That puts a tar.gz into the pypi server. Great first step that shows it is working, but not what I need to pull things out of my site-packages.
I tried just copying site-packages content over. This fails because it's not a tar.gz.
I've tried to make a tar.gz by putting pacakge_a under package_a.egg-info/src and making that a tar.gz archive I cp over to the volume I've mounted for pypi to check for packages. This approximates the structure of the tar.gz I made with sdist. However I get an error about the package missing setup.py or pyproject.toml.
I can confirm that setup.py nor pyproject.toml are present in the site-packages.
I need create a requierment.txt for all scripts in specific folder(s). For your informations, scripts are used by an other program and run inside it... I can't use venv because program use it own install python. I run pip in *.bat to specify python path in PATH.
Do you know if a package python exist and can output a list of packages use in scripts grouped in specific folder it contains python (an other but ignore it) file with retroingenieering source file?
for example:
pip_local freeze -py-env="c:\program\bin\py27\python.exe" -script-path="c:\program\scripts"
"scripts" contains module script and "local lib class package"
The requirements.txt convention isn't used for bare modules in a directory. There is no way to reference a module by path name in a requirements file. Requirements files are meant to state a module's globally unique name and version number, where "globally unique" is relative to all the repositories you've enabled for pip -- PyPi by default, and others via for example --find-links.
You could create your modules as installable packages (via setup.py or some other mechanism) and making them available to install from PyPi, a private PyPi-ish server (like Gemfury), or directly from GitHub, Bitbucket or other SCC. Those all work with the normal requirements file mechanism.
I had developed a classic Python Django web application on my local enviroment.
Then, I used Cython to compile some my app's modules as dynamic library (*.so files) to "protect" the source code.
Now, I need to distribute my Django app to one customer (for testing purpose) and I want to provide you all my Django app (with its files) but without some specific files and folders (ex: .git folder, *.pyc, *.py, javascript unobfuscated sources, etc...).
I would like to have some bash command (ex: python setup.py local_deploy) to automatic copy entire app's folder to another folder (ex: build) and remove a specific list of folders or file patterns.
Do you have any suggestion to do that?
I recommend you package your application for deployment, as you said, by running python setup.py sdist or another setuptools script command.
You can specify which package/modules/data to include and/or use a MANIFEST.in file to include/exclude files/directories matching specified patterns.
NB if you need to perform additional steps (besides selecting the correct files and packaging them), you may also add a custom command to setup.py. Though personally, I'd use a bash script / fabric to perform those, outside of packaging..
I have a project that is constantly undergoing development. I have installed a release of the project in my python distribution's site-packages directory using the setup.py script for the project.
However, when I make changes to the project I would like my test scripts to find the files that are under the project's directory and not those that it finds in site-packages. What is the proper way to do this? I only know of one approach which is to modify the search path in the test script itself using sys.path, but then it means that I cannot use the same scripts to test the "installed" version of my codes without editing the sys.path again.
I'm not quite sure what you are asking but you could use
python setup.py develop to create a develop version of your project
https://pythonhosted.org/setuptools/setuptools.html#development-mode
Under normal circumstances, the distutils assume that you are going to
build a distribution of your project, not use it in its “raw” or
“unbuilt” form. If you were to use the distutils that way, you would
have to rebuild and reinstall your project every time you made a
change to it during development.
Another problem that sometimes comes up with the distutils is that you
may need to do development on two related projects at the same time.
You may need to put both projects’ packages in the same directory to
run them, but need to keep them separate for revision control
purposes. How can you do this?
Setuptools allows you to deploy your projects for use in a common
directory or staging area, but without copying any files. Thus, you
can edit each project’s code in its checkout directory, and only need
to run build commands when you change a project’s C extensions or
similarly compiled files. You can even deploy a project into another
project’s checkout directory, if that’s your preferred way of working
(as opposed to using a common independent staging area or the
site-packages directory).
Use "Editable" package installation like:
pip install -e path/to/SomeProject
Assuming we are in the same directory with setup.py, the command will be:
pip install -e .
Python has ability to "pseudoinstall" a package by running it's setup.py script with develop instead of install. This modifies python environment so package can be imported from it's current location (it's not copied into site-package directory). This allows to develop packages that are used by other packages: source code is modified in place and changes are available to rest of python code via simple import.
All works fine except that setup.py develop command creates an .egg-info folder with metadata at same level as setup.py. Mixing source code and temporary files is not very good idea - this folder need to be added into "ignore" lists of multiple tools starting from vcs and ending backup systems.
Is it possible to use setup.py develop but create .egg-info directory in some other place, so original source code is not polluted by temporary directory and files?
setup.py develop creates a python egg, in-place; it does not [modify the] python environment so package can be imported from it's current location. You still have to either add it's location to the python search path or use the directory it is placed in as the current directory.
It is the job of the develop command to create an in-place egg, which may include compiling C extensions, running the 2to3 python conversion process to create Python3 compatible code, and to provide metadata other python code may be relying on. When you install the package as an egg in your site-packages directory, the same metadata is included there as well. The data is certainly not temporary (it is extracted from your setup.py file for easy parsing by other tools).
The intent is that you can then rely on that metadata when using your package in a wider system that relies on the metadata being present, while still developing the package. For example, in a buildout development deployment, we often use mr.developer to automate the process of fetching the source code for a given package when we need to work on it, which builds it as a develop egg and ties it into the deployment while we work on the code.
Note that the .egg-info directory serves a specific purpose: to signal to other tools in the setuptools eco-system that your package is installed and available. If your package is a dependency of another egg in your setup, then that dependency is satisfied. pip and easy_install and buildout will not try and fetch the egg from somewhere else instead.
Apart from creating the .egg-info directory, the only other thing the command does, is to build extensions, in-place. So the command you are looking for instead is:
setup.py build_ext --inplace
This will do the exact same thing as setup.py develop but leave out the .egg-info directory. It also won't generate the .pth file.
There is no way of generating only the .pth file and leave out the .egg-info directory generation.
Technically speaking, setup.py develop will also check if you have the setuptools site.py file installed to support namespaced packages, but that's not relevant here.
The good manner is to keep all source files inside special directory which name is your project name (programmers using other languages keep their code inside src directory). So if your setup.py file is inside myproject directory then you should keep the files at myproject/myproject. This method keeps your sources separated from other files regardless what happen in main directory.
My suggestion would be to use whitelist instead of blacklist -- tell the tools to ignore all files excluding these which are inside myproject directory. I think that this is the simplest way not to touch your ignore lists too often.
Try the --install-dir option. You may also want to use --build-dir to change building dir.