Entire

I’ll audit the current workflows, test layout, packaging, and native-extension path, then turn that into a prioritized CI roadmap grounded in what the repository already enforces.

The suite is already substantial and the architecture test is a strong foundation. The main gap I see so far is CI design: every pull request runs the entire expensive stack six times, while release/package correctness, coverage trends, and post-merge health are not checked at all. I’m checking test duration/skip behavior and the Gkeyll build path next so the recommendations separate fast gates from genuinely platform-specific verification.

The local run confirms the fast part of the suite is healthy: roughly 1,700 tests reach 93% in about 12 seconds. The final render/export cases dominate the wall time, which supports making render smoke tests a distinct CI tier instead of paying that cost in every Python/OS cell.

Two concrete findings sharpen the priorities:

Excluding the two real Kaleido video exports, the suite passes locally on Python 3.13: 1,682 passed, 6 skipped, at 95% statement coverage in 31.6 seconds.
The first Kaleido GIF end-to-end test hung for over a minute despite the current “Chrome exists” guard. CI needs explicit timeouts and a dedicated renderer-integration lane; binary presence is not a sufficient readiness check.

I also found that the built extension records an absolute runpath to the checkout’s gkeyll/build directory, while libg0core.so is not package data. A clean-wheel install test should be an early priority because it is likely to expose a non-portable artifact.

The repository does not mainly need more test volume—it already has 1,690 tests and about 95.3% statement coverage locally. The next gains are CI reliability, reproducibility, and artifact testing.

Highest priorities

Test the exact commit

The formatting job currently rewrites the code, applies that patch to every test job, then optionally commits it later (test.yml). That means CI tests code that is not actually in the PR commit, and fork PRs can pass while remaining unformatted.

Make formatting a check-only required job:

If automatic formatting is desired, keep it as a separate optional/manual workflow.

Add coverage enforcement

I measured 95.33% statement coverage with the two external Kaleido exports excluded. Add pytest-cov to the test extra and enforce an initial 95% floor in one Linux job. Then introduce branch coverage and changed-line coverage without chasing 100%.

Prevent hangs and silent skips

The Kaleido GIF test (test_render_plotly.py) did not complete within 45 seconds even though Chrome detection passed. Add:

pytest-timeout, with strict timeouts around external renderers.
Job-level timeout-minutes.
Explicit unit, native, render, and external_tool markers.
--strict-markers --strict-config -ra.
A native-job preflight that fails unless gpython.available() is true; native tests must not silently skip because the extension failed to load.

Test distributable artifacts

CI only tests an editable checkout. The extension currently embeds an absolute runpath to the checkout (build_gpython.sh), while libg0core.so is not included as package data (pyproject.toml). A wheel is therefore likely non-portable.

Add a packaging job that:

Builds sdist and wheel.
Runs twine check.
Installs the wheel in a clean temporary environment.
Imports Postgkyl from outside the repository.
Runs pgkyl --version and a native load/interpolate smoke test.
Checks Linux/macOS shared-library dependencies with auditwheel/delocate.

Make builds reproducible

Gkeyll is cloned from a moving branch (build_gkeyll.sh). Pin one exact commit in one authoritative location and key the CI cache from it. Update that commit deliberately through a scheduled dependency-update workflow.

Also reconcile requirements.txt with pyproject.toml; their dependency names and floors currently disagree. Prefer pyproject.toml as the source of truth and generate constraints mechanically.

Recommended CI shape

quality: formatting, Ruff linting, architecture tests, incremental type checking.
unit: pure/core tests over every supported Python version.
coverage: complete Linux suite on one representative Python.
native: targeted Gkeyll/C-extension tests on Linux and macOS.
render: known Chrome/ffmpeg/OpenGL environment, isolated and timed.
package: clean sdist/wheel installation.
sanitizer: scheduled ASan/UBSan native run.

Add push on the default branch, workflow_dispatch, concurrency cancellation, pip/native-build caching, and Python 3.13/3.14 coverage if they are supported. Currently requires-python >=3.10, classifiers, and the 3.10–3.12 matrix disagree.

Improve test quality

The most valuable new tests would be property-based invariants rather than more branch-targeted examples:

nodal/modal/quad round trips;
integration of constants equals physical volume;
native and NumPy linear operations agree after interpolation;
I/O round trips preserve grid, metadata, and value form;
inplace and emitted-result paths are equivalent without aliasing;
malformed states are rejected at construction.

Finally, make warnings errors by default and scope the broad global ignores in pyproject.toml to the tests that intentionally produce them.

I would implement this in three PRs: CI reliability and coverage first, reproducible packaging second, then test tiers/property testing/static analysis.

I’ll implement this as a coordinated CI hardening pass: exact-source checks and tiered jobs, reproducible/native packaging checks, strict pytest behavior with renderer isolation, and higher-value invariant tests. I’ll preserve the existing untracked workspace files and keep each parallel task on disjoint files.

The implementation is underway. I’m also turning linting into a real gate, but conservatively: the existing tree has a small, finite set of unused imports/variables rather than hundreds of style violations, so we can enable Pyflakes-class Ruff rules without a large unrelated rewrite.

Strict warnings immediately found real test-harness leaks rather than product failures: ten failures all come from unclosed files in architecture and I/O tests. I’m fixing those resource leaks instead of weakening the new warnings-as-errors gate.

The hardened non-external suite is now clean: 1,704 passed, 6 expected skips, and warnings-as-errors exposed and eliminated ten unclosed-file leaks. The native extension now loads through $ORIGIN with its bundled library, so I’m doing the final branch-coverage and clean-artifact validation before handing this over.

The pure-Python simulation caught an important classification gap before CI did: many integration tests use interpolation transitively without declaring their own native requirement. I’m replacing the unsafe negative selection with a positive, centrally derived compatibility inventory; with the extension physically hidden, that lane passes 269 tests.

Implemented a comprehensive CI/testing hardening pass.

Key improvements:

Tiered CI across Python 3.10–3.14, Linux/macOS, renderers, coverage, and packaging: test.yml
Strict pytest markers, warnings-as-errors, native capability enforcement, and 93% branch-coverage gate: pyproject.toml
24 new invariant tests for arithmetic, integration, cloning, and representations: test_invariants.py
Reproducible Gkeyll revision and portable native wheels using $ORIGIN/@loader_path: build_gkeyll.sh
Clean, off-checkout wheel verification: smoke_wheel.sh
Ruff and repository sanity hooks added to pre-commit.
Weekly dependency and GitHub Actions updates: dependabot.yml
Removed duplicated requirements.txt; pyproject.toml is now authoritative.

Validation completed:

1,704 passed, 6 expected skips, 4 external tests deselected.
Combined branch coverage: 93.54%.
Native lane: 495 passed.
Pure compatibility lane: 269 passed with the extension physically absent.
Clean native wheel installation, import, load, and interpolation passed.
YAML, TOML, shell syntax, formatting, and diff checks passed.

The macOS loader path and Python 3.14 environment remain platform-CI validations.