We Keep Coding

Poetry install on an existing project Error "does not contain any element"

I am using Poetry for the first time.
I have a very simple project. Basically
a_project
|
|--test
| |---test_something.py
|
|-script_to_test.py
From a project I do poetry init and then poetry install
I get the following
poetry install
Updating dependencies
Resolving dependencies... (0.5s)
Writing lock file
Package operations: 7 installs, 0 updates, 0 removals
• Installing attrs (22.2.0)
• Installing exceptiongroup (1.1.0)
• Installing iniconfig (2.0.0)
• Installing packaging (23.0)
• Installing pluggy (1.0.0)
• Installing tomli (2.0.1)
• Installing pytest (7.2.1)
/home/me/MyStudy/2023/pyenv_practice/dos/a_project/a_project does not contain any element
after this I can run poetry run pytest without problem but what does that error message mean?

This is probably because Poetry tries to install your project but does not find it (there’s no a_project module inside your directory). You can tell it not to install the root project with --no-root:
poetry install --no-root
As of today there is no way to configure this to be the default (see this issue).

My issue got away after pointed correct interpreter in PyCharm. Poetry makes project environment in its own directories and PyCharm didn't link that correct.
I've added new environment in PyCharm and select poetary's just created enviroment in dialogs.

Check if your pyproject.toml contains something like:
[tool.poetry]
packages = [{include = "a_project"}]
Removing the line with packages = [{include = "a_project"}] helped in my case and should avoid including the root project. See documentation here.

create a dir with_your_package_name that u find in the file and an empty __init__.py in project root
delete the poetry.lock and install again

How do I install packages from test.pypi.org using poetry?

I want to use a pre-release version of a package (https://test.pypi.org/project/delta-spark/2.1.0rc1/) in my project.
I'm using poetry to manage my pyproject.toml. How do I do this?
In other words what is the poetry equivalent of:
pip install -i https://test.pypi.org/simple/ delta-spark==2.1.0rc1
I tried:
poetry add delta-spark==2.1.0rc1
poetry add --allow-prereleases delta-spark==2.1.0rc1
Both give: Could not find a matching version of package delta-spark
$ poetry config --local repositories.test-pypi https://test.pypi.org/
$ poetry config --list | fgrep repositories
repositories.test.url = "https://test.pypi.org/"
repositories.test-pypi.url = "https://test.pypi.org/"
$ fgrep -A 3 tool.poetry.source pyproject.toml
[[tool.poetry.source]]
name = "test-pypi"
url = "https://test.pypi.org/"
secondary = true
$ poetry add --group dev delta-spark==2.1.0rc1
Could not find a matching version of package delta-spark
$

This is described here. Basically, you can add the repository via:
poetry config repositories.test https://test.pypi.org/simple/
and then make it available in pyproject.toml via:
[[tool.poetry.source]]
name = "test"
url = "https://test.pypi.org/simple/"
secondary = true
Then adding the dependency should work.

Python poetry install failure - invalid hashes

I'm unsure exactly what information / context to provide with this post in order for it to be
properly answered - so if I can add more information please let me know.
Whilst trying to install a package I've had an error message as follows:
Invalid hashes (sha256:01d6b62184e55367ce7d770bd87d97a6cfaf783ff7ba8328e7473a339fe19807) for numpy (1.22.2)
I don't know how to go about troubleshooting this problem.
poetry --version
(install-check-clU745_7-py3.8) ➜ install_check poetry --version
Poetry version 1.1.12
poetry env info
Output of poetry env info:
(install-check-clU745_7-py3.8) ➜ install_check poetry env info
Virtualenv
Python: 3.8.12
Implementation: CPython
Path: /Users/user_name/Library/Caches/pypoetry/virtualenvs/install-check-clU745_7-py3.8
Valid: True
System
Platform: darwin
OS: posix
Python: /Users/user_name/.pyenv/versions/3.8.12
pyproject.toml
pyproject.toml config:
(install-check-clU745_7-py3.8) ➜ install_check cat pyproject.toml
[tool.poetry]
name = "install_check"
version = "0.1.0"
description = ""
authors = ["user_name <user_name#email_add.com>"]
[tool.poetry.dependencies]
python = "^3.8"
[tool.poetry.dev-dependencies]
pytest = "^5.2"
[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"
Steps to reproduce
Here is an attempt to clearly outline a reproduction of this problem:
python --version
Python 3.8.12
which python
/Users/user_name/.pyenv/shims/python
poetry new install_check
Created package install_check in install_check
cd install_check
cat pyproject.toml| grep python
python = "^3.7"
# update pyproject.toml to 3.8 instead of 3.7.
cat pyproject.toml | grep python
python = "^3.8"
python --version
Python 3.8.12
poetry env use python3.8
which python3.8
/Users/user_name/.pyenv/shims/python3.8
poetry env use python3.8
poetry install
Updating dependencies
Resolving dependencies... (5.0s)
Writing lock file
Package operations: 8 installs, 0 updates, 0 removals
• Installing pyparsing (3.0.7)
• Installing attrs (21.4.0)
• Installing more-itertools (8.12.0)
• Installing packaging (21.3)
• Installing pluggy (0.13.1)
• Installing py (1.11.0)
• Installing wcwidth (0.2.5)
• Installing pytest (5.4.3)
Installing the current project: install_check (0.1.0)
Following this, after leaving and revisiting the directory to activate the env, I have:
(install-check-clU745_7-py3.8) ➜ install_check which python
/Users/user_name/Library/Caches/pypoetry/virtualenvs/install-check-clU745_7-py3.8/bin/python
(install-check-clU745_7-py3.8) ➜ install_check python --version
Python 3.8.12
Trying to add pandas then gives the following error:
(install-check-clU745_7-py3.8) ➜ install_check poetry add pandas
Using version ^1.4.0 for pandas
Updating dependencies
Resolving dependencies... (2.3s)
Writing lock file
Package operations: 5 installs, 0 updates, 0 removals
• Installing six (1.16.0)
• Installing numpy (1.22.2): Failed
RuntimeError
Invalid hashes (sha256:01d6b62184e55367ce7d770bd87d97a6cfaf783ff7ba8328e7473a339fe19807) for numpy (1.22.2) using archive numpy-1.22.2-cp38-cp38-macosx_10_14_x86_64.whl. Expected one of sha256:03ae5850619abb34a879d5f2d4bb4dcd025d6d8fb72f5e461dae84edccfe129f, sha256:076aee5a3763d41da6bef9565fdf3cb987606f567cd8b104aded2b38b7b47abf, sha256:0b536b6840e84c1c6a410f3a5aa727821e6108f3454d81a5cd5900999ef04f89, sha256:15efb7b93806d438e3bc590ca8ef2f953b0ce4f86f337ef4559d31ec6cf9d7dd, sha256:168259b1b184aa83a514f307352c25c56af111c269ffc109d9704e81f72e764b, sha256:2638389562bda1635b564490d76713695ff497242a83d9b684d27bb4a6cc9d7a, sha256:3556c5550de40027d3121ebbb170f61bbe19eb639c7ad0c7b482cd9b560cd23b, sha256:4a176959b6e7e00b5a0d6f549a479f869829bfd8150282c590deee6d099bbb6e, sha256:515a8b6edbb904594685da6e176ac9fbea8f73a5ebae947281de6613e27f1956, sha256:55535c7c2f61e2b2fc817c5cbe1af7cb907c7f011e46ae0a52caa4be1f19afe2, sha256:59153979d60f5bfe9e4c00e401e24dfe0469ef8da6d68247439d3278f30a180f, sha256:60cb8e5933193a3cc2912ee29ca331e9c15b2da034f76159b7abc520b3d1233a, sha256:6767ad399e9327bfdbaa40871be4254d1995f4a3ca3806127f10cec778bd9896, sha256:76a4f9bce0278becc2da7da3b8ef854bed41a991f4226911a24a9711baad672c, sha256:8cf33634b60c9cef346663a222d9841d3bbbc0a2f00221d6bcfd0d993d5543f6, sha256:94dd11d9f13ea1be17bac39c1942f527cbf7065f94953cf62dfe805653da2f8f, sha256:aafa46b5a39a27aca566198d3312fb3bde95ce9677085efd02c86f7ef6be4ec7, sha256:badca914580eb46385e7f7e4e426fea6de0a37b9e06bec252e481ae7ec287082, sha256:d76a26c5118c4d96e264acc9e3242d72e1a2b92e739807b3b69d8d47684b6677.
at ~/Library/Application Support/pypoetry/venv/lib/python3.7/site-packages/poetry/installation/executor.py:632 in _download_link
628│ "Invalid hashes ({}) for {} using archive {}. Expected one of {}.".format(
629│ ", ".join(sorted(archive_hashes)),
630│ package,
631│ archive_path.name,
→ 632│ ", ".join(sorted(hashes)),
633│ )
634│ )
635│
636│ return archive
• Installing python-dateutil (2.8.2)
• Installing pytz (2021.3)
Failed to add packages, reverting the pyproject.toml file to its original content.
Attempt to solve
remove existing poetry installation
curl -sSL https://install.python-poetry.org | python3 - --uninstall
install specific poetry version
Trying to following this issue comment I installed with:
curl -sSL https://install.python-poetry.org | python3 - --version 1.1.8
Try creating another env like before (Still fails)
I'm still getting failures.
python --version
Python 3.8.12
poetry --version
Poetry version 1.1.8
poetry new install_check_2
Created package install_check_2 in install_check_2
cd install_check_2
Then within the newly created project:
poetry env use python3.8
Creating virtualenv install-check-2-1m3rSWN1-py3.8 in /Users/user_name/Library/Caches/pypoetry/virtualenvs
Using virtualenv: /Users/user_name/Library/Caches/pypoetry/virtualenvs/install-check-2-1m3rSWN1-py3.8
poetry install
Updating dependencies
Resolving dependencies... (4.9s)
Writing lock file
Package operations: 8 installs, 0 updates, 0 removals
• Installing pyparsing (3.0.7): Failed
AttributeError
'Link' object has no attribute 'is_absolute'
at ~/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/poetry/core/packages/file_dependency.py:33 in __init__
29│ self._path = path
30│ self._base = base or Path.cwd()
31│ self._full_path = path
32│
→ 33│ if not self._path.is_absolute():
34│ try:
35│ self._full_path = self._base.joinpath(self._path).resolve()
36│ except FileNotFoundError:
37│ raise ValueError("Directory {} does not exist".format(self._path))

There are several issue reports about invalid hashes. One common cause is running multiple Poetry instances simultaneously; another is aborted downloads (e.g. ^C). Poetry should be fixed to make corruption less likely to happen in such cases, and perhaps to try downloading corrupted files again automatically.
Until then, the workaround is to remove the offending file from the cache and let Poetry download a fresh copy.
You can find the full path to the offending file like this:
$ find ~/.cache/pypoetry -name numpy-1.22.2-cp38-cp38-macosx_10_14_x86_64.whl
Or, for the lazy, just blow away the entire cache:
$ rm -rf ~/.cache/pypoetry

I ctrl-C'd during a poetry install, which caused one of the cached wheels to partially download and have a hash that didn't match what was on PyPI.
On macOS with Poetry 1.1.13, the only method that worked for me was Yihua Zhou's comment:
rm -r ~/Library/Caches/pypoetry/cache
rm -r ~/Library/Caches/pypoetry/artifacts

Maybe this is a solution:
poetry cache clear . --all
rm poetry.lock
poetry install
https://github.com/python-poetry/poetry/issues/4523#issuecomment-923155835

If you're doing this in a Docker image, in addition to the answers above I also had to do:
docker builder prune
This could be necessary if taking using some of the more recent docker build caching features in the Dockerfile such as:
RUN --mount=type=cache,target=$POETRY_CACHE_DIR/cache \
--mount=type=cache,target=$POETRY_CACHE_DIR/artifacts \
poetry install
This is because the poetry cache is on the host file system and needs to be cleared for same reason as the answers above.

install local package in tox unencumbered by previous install from GitHub

Question
How to use tox to install a the local package without it refering to a random time when I installed the pacakge from GitHub?
Context
In this PR, I'm trying to build and test my package, dbt-synapse, before publishing a new minor version to PyPI. At some point (rookie move), I think I tried installing the package directly from a specific GitHub commit with the below line.
pip install -e git+https://github.com/dbt-msft/dbt-synapse.git#345d7cafcb08bac25d23867a2e22c0e9b741f603#egg=dbt_synapse
Here's a gist of the requirements.txt, tox.ini, setup.py and some other files
When I call tox -e integration-synapse to kick off the tests, having listing the actual package in the deps as -e. compared to just ., causes the package to be installed from GitHub (see the integration-synapse installed: line below, shortened for clarity)!
when deps has just ., however it lists this as having been installed (which also doesn't seem right): dbt-synapse # file:///Users/anders.swanson/repos/dbt-synapse.
Unfortunately, neither are giving me what I know should be happening.
integration-synapse recreate: /Users/anders.swanson/repos/dbt-synapse/.tox/integration-synapse
integration-synapse installdeps: -rrequirements.txt, -e.
integration-synapse installed: [...],-e git+https://github.com/dbt-msft/dbt-synapse.git#345d7cafcb08bac25d23867a2e22c0e9b741f603#egg=dbt_synapse,[...]
integration-synapse runtests: PYTHONHASHSEED='1932234374'
integration-synapse runtests: commands[0] | /bin/bash -c '/Users/anders.swanson/repos/dbt-synapse/.tox/integration-synapse/bin/python -m pytest -v test/integration/synapse.dbtspec'
======================================================================================================================================== test session starts =========================================================================================================================================
platform darwin -- Python 3.6.12, pytest-6.2.1, py-1.10.0, pluggy-0.13.1 -- /Users/anders.swanson/repos/dbt-synapse/.tox/integration-synapse/bin/python
cachedir: .pytest_cache
rootdir: /Users/anders.swanson/repos/dbt-synapse
plugins: dbt-adapter-0.4.0
collected 1 item
tox.ini
[tox]
skipsdist = True
envlist = integration-synapse
[testenv:integration-synapse]
basepython = python3
commands = /bin/bash -c '{envpython} -m pytest -v test/integration/synapse.dbtspec'
passenv = DBT_SYNAPSE_DB DBT_SYNAPSE_PORT DBT_SYNAPSE_PWD DBT_SYNAPSE_SERVER DBT_SYNAPSE_UID
deps =
-rrequirements.txt
-e.

If you want to start from a clean state, you can you can delete the hidden .tox folder and start from scratch.
Also, you can run tox -r which will recreate the environemnts.
In order to test the code from your local development environment, you do not have to include your package in the deps list (-e.). tox will build the package from your setup.py.
Well.. it would. But in your tox.ini you specified skipsdist = True which afaik means tox should not build the package.
There are some more issues in your tox.ini, e.g. unit and flake8 envs do nothing.
Why do you use the /bin/bash -c construct?
Unfortunately, I am missing some header files, so I cannot run your tox.ini file, but I would create a tox.ini file like this...
[tox]
envlist = py36,py37,py38,py39,integration-synapse,flake8
[testenv]
deps = whatever needed for your unit tests
commands = python -m pytest ...unit tests...
[testenv:integration-synapse]
deps =
pytest
pytest-dbt-adapter
commands =
python -m pytest -v test/integration/synapse.dbtspec
passenv = DBT_SYNAPSE_DB DBT_SYNAPSE_PORT DBT_SYNAPSE_PWD DBT_SYNAPSE_SERVER DBT_SYNAPSE_UID
[testenv:flake8]
skipsdist = True
deps = flake8
commands = flake8 setup.py ... (and more)

Minimal working example for conda package build

I am trying to look into conda for python package/virtual environments management. However, I can't seem to be able to build my own conda package. Could somebody help me to construct a minimal working example?
First of all some directory structure:
- src/
|- foo1.py
|- foo2.py
- conda-build/
|- meta.yaml
- setup.py
Running python setup.py install installs the packages using pip. Now if I try to cd to the conda-build directory and run conda build . I get the following output
Removing old build directory
Removing old work directory
BUILD START: example_pkg-0.5.1-abc
Fetching package metadata: ......
Solving package specifications: .
The following NEW packages will be INSTALLED:
pip: 6.1.1-py34_0
python: 3.4.3-0
setuptools: 15.0-py34_0
Linking packages ...
[ COMPLETE ]|##################################################| 100%
Removing old work directory
Copying C:\some\path\ to C:\Anaconda3\conda-bld\work
Package: example_pkg-0.5.1-abc
source tree in: C:\Anaconda3\conda-bld\work
number of files: 0
Fixing permissions
Fixing permissions
BUILD END: example_pkg-0.5.1-abc
Nothing to test for: example_pkg-0.5.1-abc
# If you want to upload this package to binstar.org later, type:
#
# $ binstar upload C:\Anaconda3\conda-bld\win-64\example_pkg-0.5.1- abc.tar.bz2
#
# To have conda build upload to binstar automatically, use
# $ conda config --set binstar_upload yes
I can indeed find the package in the directory C:\Anaconda3\conda-bld\win-64 but the package does not seem to contain any files. I can install the package using conda install --use-local .\example_pkg-0.5.1-abc.tar.bz2 and then it is listed by conda list, however I cannot import it in Python. This is my meta.yaml:
package:
name: example_pkg
version: "0.5.1"
source:
path: ../src
build:
number: 1
string: abc
script: python setup.py install
requirements:
build:
- python
run:
- python
Any help is greatly appreciated! :)

Looks like there's an issue where build/script doesn't work on Windows. Until that PR is merged, you should just create a bld.bat with
python setup.py install
if errorlevel 1 exit 1
and put it in the conda recipe.