Nightly feedstock build failed #167
Comments
|
Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary |
|
The exact same test failed with a core dump on Friday and Sunday nights, only for the linux-64 py312 np2 build (Friday, Sunday). @kounelisagis any insights into why this particular test is so flaky only on linux-64 with python 3.12 and numpy2? |
|
Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary |
|
I have never seen this test fail locally or on CI. |
|
Last night the same test |
|
Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary |
|
Last night both the py39 and py312 linux-64 tiledb-py-feedstock builds failed (py313 passed though). I restarted them |
|
Both jobs failed again on their second attempt 🤷 |
|
Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary |
|
As discussed with @jdblischak, |
|
Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary |
|
Likely source of the flaky test failure is this upstream PR to TileDB TileDB-Inc/TileDB#5417 which was merged late on Feb 6th (TileDB-Inc/TileDB@3c617e3) |
|
Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary |
|
All linux64 jobs failed last night. I'm still trying to repro. |
|
Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary |
|
Both @kounelisagis and I are unable to reproduce the error locally using # Use the latest nightly-build branch
basename $(pwd)
## tiledb-py-feedstock
git fetch tiledb
git checkout -t tiledb/nightly-build
git branch --show-current
## nightly-build
git log -n 1 --pretty=reference
## 710e7d5 (MNT: Re-rendered with conda-build 24.9.0, conda-smithy 3.45.4, and conda-forge-pinning 2025.02.17.08.22.26, 2025-02-17)
python ./build-locally.py --debug linux_64_numpy2.0python3.9.____cpython
# Setup debugging environment
cd /home/conda/feedstock_root/build_artifacts/debug_1739894496071/work
source /home/conda/feedstock_root/build_artifacts/debug_1739894496071/work/build_env_setup.sh
conda install --yes git
# Install TileDB-Py
bash $RECIPE_DIR/build.sh
## Successfully built tiledb
## Installing collected packages: tiledb
## Successfully installed tiledb-0.33.4.dev14+g3bafb00b
# Setup test environment
# Remove build env to be able to use the host env directly (which is where Python is installed)
conda deactivate
# Install test requirements
conda install --yes hypothesis pyarrow pytest pandas psutil
# To avoid import error due to local directory named tiledb/, move the tests
# to another directory
mkdir ../test
mv tiledb/tests/ ../test/
cd ../test/
# Run tests
pytest -v tests/test_multi_index*
## ============================================ test session starts =============================================
## platform linux -- Python 3.9.21, pytest-8.3.4, pluggy-1.5.0 -- /home/conda/feedstock_root/build_artifacts/debug_1739894496071/_h_env/bin/python3.9
## cachedir: .pytest_cache
## hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase(PosixPath('/home/conda/feedstock_root/build_artifacts/debug_1739894496071/test/.hypothesis/examples'))
## rootdir: /home/conda/feedstock_root/build_artifacts/debug_1739894496071/test
## plugins: hypothesis-6.126.0
## collected 54 items
##
## tests/test_multi_index-hp.py::TestMultiIndexPropertySparse::test_multi_index_two_way_query PASSED [ 1%]
## tests/test_multi_index-hp.py::TestMultiIndexPropertySparse::test_multi_index_inputs PASSED [ 3%]
##
## ============================================= 54 passed in 7.41s ============================================= |
|
The last failing tiledb-py-feedstock build was the night of Monday, Feb 17th. It used TileDB-Inc/TileDB@d0348dc and TileDB-Inc/TileDB-Py@3bafb00. However, it has subsequently passed the previous 3 nights. The first passing tiledb-py-feedstock build on the night of Tuesday, Feb 18th used TileDB-Inc/TileDB@022af98 and TileDB-Inc/TileDB-Py@3bafb00. Thus the only difference was a single commit to TileDB, TileDB-Inc/TileDB@022af98, from TileDB-Inc/TileDB#5461 TileDB-Inc/TileDB@d0348dc...022af98 As this PR was a fix for a problem introduced in TileDB-Inc/TileDB#5417, which was suspected to be the cause of this nightly test failure, this result makes sense. |
Nightly feedstock build failure for TileDB-Py%20Feedstock%20Testing at https://dev.azure.com/TileDB-Inc/CI/_build?definitionId=5&_a=summary
The text was updated successfully, but these errors were encountered: