Improve Unit Testing Coverage Threshold

I’ll establish the current baseline first: inspect the worktree and coverage configuration, then run the coverage suite and target the weakest meaningful modules. After improvements, I’ll raise the CI threshold only to a level the full suite verifies.
The worktree already contains a substantial in-progress coverage effort and a threshold increase from 90% to 93%, so I’ll preserve those edits and validate them as the new baseline. The test dependencies do not declare xdist; I’m checking the actual environment and will mirror the authoritative CI lane rather than assume parallel support.
The native bridge and four-worker pytest support are available. The local environment lacks pytest-timeout despite the project declaring it, so the first CI-mirror invocation stopped during argument parsing without running tests; I’m rerunning the same coverage workload with -n4 and only omitting that unavailable plugin option.
Baseline is healthy: 1,704 passed, 6 skipped, with 93.54% branch coverage, so the existing 93% gate is valid but leaves only a 0.54-point cushion. The largest remaining gaps are concentrated in CLI compilation, expression evaluation, rendering, and gyrokinetic geometry; I’m prioritizing deterministic unit branches that do not depend on optional external executables.
The CLI gap is largely untested contract validation—not incidental implementation. I’m adding focused compatibility tests for invalid metadata, docstring parsing, annotation codecs, and pipeline execution policies. These are stable unit cases and should materially raise coverage without relying on graphics, native fixtures, or timing.
The new contract tests now cover 62 focused cases; two assertions initially relied on Click preserving callback return values in its test runner, which it does not, so I corrected them to verify the authoritative mutated pipeline state. A targeted coverage process then hit an unrelated native-library shutdown crash (exit 139) after producing no report; I’m validating the tests normally and will measure coverage in the same four-worker mode that completed reliably.
The new tests raised total branch coverage from 93.54% to 94.79% while keeping the full suite green (1,766 passed). CLI compiler coverage moved from 82.58% to 96.14%, docstrings to 99.08%, and schema records to 98.47%. I’m continuing toward a defensible 95% gate by targeting the expression evaluator’s remaining error and dispatch paths.
Evaluator coverage is now 99.35%, and the full suite reaches 95.28% (1,776 passed). That supports a 95% gate, but I can still improve inexpensive deterministic gaps. Next I’m covering version-resolution fallbacks and selected render/animation helpers; these account for substantial missed branches without invoking Chrome or ffmpeg.
Coverage is now 95.78%; both _version.py and render/animate.py reached 100%, with 1,794 tests passing. I’m making one more focused pass on reachable CLI adapter branches and small public error paths to cross 96% with margin before updating both threshold sources.
The suite now crosses the next gate at 96.08% (1,809 passed), with CLI compilation at 99.34% and several smaller modules at 100%. Because 0.08 points is too narrow a CI margin across Python 3.12 versus this local 3.13 run, I’m adding a small buffer through integration validation branches before setting the gate to 96%.
The buffer is now 0.28 points: 96.28% branch coverage with 1,816 passed and 6 environment-based skips. I’m raising both authoritative threshold locations from 93% to 96%, then I’ll run lint plus the exact coverage gate once more and audit the final diff for accidental or unrelated changes.
The local environment also lacks the declared Ruff dependency, so lint could not start; collection still succeeded for all 1,826 tests, including 352 compatibility cases, and git diff --check is clean. I’m doing a manual diff review and the final full 96% coverage run now; I’ll report the unavailable lint tool explicitly if it remains absent.
The 96% gate passes with 0.28 points of margin, but the remaining report still shows a concentrated, testable gap in gyrokinetic geometry/flux-surface handling. Since the objective asks for improvement “as much as possible,” I’m continuing there before treating 96% as final; these branches encode real coordinate-shape and metadata contracts, not artificial coverage cases.
The geometry pass produced a meaningful jump to 97.13%: shared geometry is now 99.57% and flux-surface extraction is 100%, with 1,844 tests passing. I’m covering the companion reusable R–Z projection validation and caching contracts next, which should give the 97% gate enough cross-version margin to be credible.
R–Z validation tests are green. The remaining dominant gap is Plotly’s save/preview machinery, where the real GIF/MP4 tests are intentionally excluded from coverage as external-tool tests. I’m exercising the same protocol with mocked Kaleido/ffmpeg boundaries so CI verifies command construction, frame lifecycle, naming, and preview behavior without launching external processes.
Active goal
Improve unit-test coverage as much as practical, then raise the CI coverage threshold.
Progress
- Baseline: 93.54% branch coverage, 1,704 passed.
- Current: 98.11% branch coverage, 1,855 passed, 6 skipped.
- Full verified command:
- Threshold is currently raised from 93 to 96 in:
pyproject.toml.github/workflows/test.yml
- It should now be raised to 98, given 98.11% measured coverage. Consider adding a little margin first because 0.11 is narrow across local Python 3.13 vs CI Python 3.12.
Added/expanded coverage
New files:
tests/test_cli_contract_edges.pytests/test_version_report.pytests/test_operations_gk_geometry.py
Expanded:
tests/test_operations_evaluate.pytests/test_render_animate.pytests/test_render_plotly.pytests/test_render_ffmpeg.pytests/test_operations_gk_fluxsurf.pytests/test_operations_gk_rz.pytests/test_coverage_operations.pytests/test_operations_fit.pytests/test_load.py
Major results:
_version.py: 100%- CLI compiler: 99.34%
- evaluate: 99.35%
- animate: 100%
- Plotly: 98.88%
- GK geometry: 99.57%
- fluxsurf: 100%
- R-Z: 98.85%
- integrate: 98.70%
- load/select/ffmpeg/growth: 100%
External GIF/MP4 behavior is covered with mocked Kaleido/ffmpeg protocols; true external-tool tests remain excluded in the authoritative CI coverage lane.
Remaining gaps
Largest remaining source gap:
src/postgkyl/render/matplotlib.py: 54 missing lines, 35 partial branch lines, 91.67%.
Other small gaps include:
gdatastate/gdatastate.py: 5 lines, 16 partials.io/gkyl_reader.py: 4 lines, 13 partials.- various mostly defensive/unreachable one-line branches.
Best next step: add deterministic tests for easy Matplotlib missing branches to create ≥0.2% margin, then raise both thresholds to 98 and rerun the exact full coverage command.
Current Matplotlib missing report:
Verification/tooling notes
pytest-timeoutis declared but absent locally, so local CI-mirror runs omitted--timeout=120.- Ruff is also declared but absent locally:
python -m ruff→No module named ruff- no
ruffbinary found.
git diff --checkpassed.- Test collection passed: 1,826 total before the most recent additions; current full run collected 1,861 including 4 external-tool deselections.
- Use
MPLCONFIGDIR=/tmp/postgkyl-mpl-cacheto avoid Matplotlib cache warnings. - Native Gkeyll bridge is available locally.
Constraints
- Follow the supplied AGENTS.md doctrine and boolean API/CLI contract.
- Preserve unrelated worktree changes.
- Use
apply_patchfor edits. - Do not spawn subagents unless explicitly requested.
- Before completion, rerun full coverage with final threshold and audit
git diff. - Ruff unavailability should be reported unless it becomes available.
I’m picking up from the verified 98.11% run. I’ll target a few deterministic Matplotlib branches to build CI margin, then set both thresholds to 98 and rerun the full coverage lane.
The remaining report has a useful cluster of pure normalization helpers that are only partially exercised. I’m adding direct contract tests for their valid and error paths; this should gain substantially more than the 0.11-point buffer without introducing backend or rendering variability.
The helper tests pass (68/68 in the focused module), and the threshold is now 98 in both configuration and CI. I’m running the complete branch-coverage command at the final threshold now.
The final lane passes at 98.54%, giving 0.54 points of headroom: 1,874 passed and 6 skipped. I’m doing the final worktree/whitespace audit now, keeping the unrelated untracked tool directories untouched.
Coverage is now 98.54% with branch measurement enabled.
- CI threshold raised from 93% to 98% in pyproject.toml and test.yml.
- Full suite: 1,874 passed, 6 skipped.
- Statement coverage: 99.31%; branch coverage: 96.38%.
git diff --checkpasses.- Ruff remains unavailable locally.
- Unrelated untracked tool directories were untouched.
The prior turn made verified progress, but the completion audit shows 167 coverage opportunities remain and a 99% gate is within practical reach. I’m continuing with deterministic defensive-branch tests, starting with Matplotlib and container-state contracts, then I’ll reassess the achievable threshold.
The new tests exercise error semantics, split-panel variants, mapped/surface rendering, sparse/rich metadata reporting, native-storage mutation guards, and the remaining fluent operators. I’m validating these focused modules before measuring the new global percentage.
The global result is now 99.10% (1,905 passed), so a 99% gate is achievable but still has only 0.10 points of margin. I’m adding another compact set of high-value contract cases—especially CLI discovery and operation fallbacks—to make the 99% threshold robust across Python versions.
One focused run exposed a failure before entering Plotly’s slower external-tool cases; the command also outlived the short tool window. I’m inspecting the live process and the failing test separately, without restarting the full suite.
The margin-building cases now pass, including weighted modal averaging, full coordinate projection, non-kernel evaluator fallback, malformed VTK series recovery, and Plotly save/show dispatch. I’m promoting the CI gate to 99% and will run the complete authoritative lane once more at that final value.
The final CI-equivalent run passes at 99.54% with a 99% gate: 1,927 passed and 6 skipped. The residual misses are concentrated in platform/optional-backend defenses and structurally unreachable guards; I’m performing the completion audit now—threshold consistency, changed-file scope, whitespace, and tooling availability.
Completed in about 28 minutes of active work.
- Coverage improved from 93.54% to 99.54%.
- Statements: 99.82%; branches: 98.74%.
- CI threshold raised from 93% to 99% in pyproject.toml and test.yml.
- Final suite: 1,927 passed, 6 skipped.
git diff HEAD --checkpasses.- Ruff remains unavailable locally (
No module named ruff). - Unrelated untracked directories were untouched.
Status
Goal completed and marked complete: improve test coverage and raise CI threshold.
- Baseline: 93.54% coverage, 1,704 passed.
- Final: 99.54% coverage with branch measurement enabled.
- Statements: 99.82%
- Branches: 98.74%
- 1,927 passed, 6 skipped
- CI threshold raised from 93 to 99 in:
pyproject.toml.github/workflows/test.yml
Verification
Final passing command:
git diff HEAD --check passes.
Ruff is unavailable locally:
Changes
Only tests and coverage configuration were changed—no source implementation files. Coverage additions span CLI contracts/discovery, version reporting, GData/container behavior, readers/writers, evaluate/fit, GK geometry/flux surfaces/R-Z, and Matplotlib/Plotly/animation/ffmpeg behavior.
Combined diff: 22 files, roughly 2,404 test/config insertions.
Worktree caution
Unrelated untracked directories remain untouched:
.claude/.codex/.entire/
Some task changes are staged while the latest threshold edits may appear staged/unstaged (MM); preserve the index state unless explicitly asked to change it.
Remaining work
None required for the completed objective. If asked to continue, inspect current git status first and avoid disturbing unrelated files.
AGENTS.md instructions for /home/maxwell-rosen/postgkyl
<INSTRUCTIONS> # Coding Doctrine0. Locality of reasoning. Every principle below is a projection of one axiom: a reader must be able to understand a fragment without the whole program. Whatever keeps a local conclusion sound — a frozen record, an honest signature, a stated law — is doctrine. Whatever forces a global search — ambient state, a leaky layer, a second copy of a fact — is the enemy.
Data — what it does, and what it may say
I. Data is inert. Functions transform. No objects that know things and do things. Data is a frozen record. Behavior is a function that takes data in and returns data out. If you're reaching for inheritance, you've taken a wrong turn.
II. Make illegal states unrepresentable. The shape of a datum is its strongest invariant. Constructors refuse invalid states; a checked fact becomes a type; downstream never re-proves what upstream established. Parse, don't validate.
Functions — one idea, honestly declared
III. A function is one idea. It takes exactly what it needs and returns exactly what it computes. If the signature has two concepts in it, you have two functions.
IV. The signature tells the whole truth. Inward: if something needs a value, it receives it as a parameter — no spooky action at a distance, no stringly-typed interfaces, no implicit state. Outward: same inputs, same outputs; effects and failure appear in the type, not in the fine print. Pure core, effects at the edges.
Knowledge — one home per fact
V. Every fact has one home. One authoritative representation of each decision and each piece of knowledge; everything else inherits or is derived mechanically — never maintained by hand in parallel. Configuration is decided once, at the highest level, and threaded down; no module ever decides its own context. If the design and the implementation can disagree, you have two sources of truth and zero.
Layers — what above, how below
VI. Separate what from how. Logic and machinery are different concerns with a hard boundary. The layer that says what to compute should be readable by someone who has never seen the machinery underneath. The layer that says how lives below, stays below, and nothing leaks up from it.
VII. Notation is execution; lowering is transliteration. Looking up: the spec layer reads like the math or logic it implements — when notation is the executable object, not a comment beside it, bugs have nowhere to hide. Looking down: the layer that executes the spec reproduces it exactly — nothing added, nothing dropped, nothing reinterpreted; no opinions, no defaults, no helpful conversions. If the lowering changes anything, the spec is a lie.
Abstraction — earned, and binding
VIII. Earn your abstractions. No abstraction before the second use. Three similar lines is better than a premature helper. The right amount of complexity is the minimum the current task demands — not the current task plus three hypothetical future ones.
IX. An abstraction is a contract. It is defined by what it guarantees, not what it hides. If you can't state what is always true of it — properties a client may rely on without reading the implementation — it isn't an abstraction, it's indirection. Two implementations that honor the contract must be interchangeable; and its outputs stay in its vocabulary, so uses compose.
Verification — formal first
X. Trust the most formal thing first. Types over tests, tests over docs, docs over comments. Invest in whichever layer catches the bug earliest with the least ongoing maintenance cost.
Commands
Boolean API/CLI contract
Every public boolean parameter defaults to False. If the default behavior
is enabled, name the parameter for disabling or selecting its inverse (for
example, no_show=False, volume=False, or nodal=False) and implement the
default behavior under the false branch. The generated CLI mirrors the API
parameter name and default exactly; a bare boolean option means True, while
an explicit True or False remains accepted. The CLI compiler rejects a
public boolean whose API default is not False.
Architecture — a strict, one-way layered DAG
Every folder has one job, and imports point in one direction only (leaves at the bottom). There is no import cycle — this is enforced by a test (see "Import contract"). Arrow = "may import":
Keeping the picture honest: the two diagrams below and the prose after them are a mirror of
tests/test_postgkyl.py::_ALLOWED— that dict (and the AST walk that checks every real import against it) is the enforced source of truth; this file is only a readable projection of it. The two can drift (they already had:operations/average.py,operations/eval_at_coord_proj.py,operations/local_poly.py,gdatastate/guards.py,gdata/gdatagroup.py, andgdata/verbs.pyexisted in the tree before they were added here). Whenever you add a new top-level module file or a new allowed import edge, update_ALLOWEDand this section in the same commit — don't let the picture outlive the code it describes.
The two-domain lifecycle (REFACTOR_GKEYLL_FFI.md)
Every dataset lives in one of two backends, discriminated by
GDataState.backend:
"gkyl"(modal domain) — DG coefficients as a nativegkyl_array(gpython.GkylArray). Loading lands here. All math runs inside Gkeyll:*//→ weak kernels (gkyl_dg_mul_op/div_op),+/-→ coefficient lin-combs (gkyl_array_accumulate), scalars →scale/mean-shift, integer powers → repeated weak multiply, full.integrate()→gkyl_array_integrate, partial.integrate(axis=…)→ exactgkyl_array_averagefollowed by physical-volume scaling.valuesis a read-only view;np.asarray/ufuncs/selectrefuse with ".interpolate() first"."numpy"(field domain) — post-interpolate()values as a plain ndarray; the unchanged NumPy stack (select,plot, ufuncs, arithmetic).
interpolate() is the one-way bridge: matrix from Gkeyll's basis functions,
applied per cell with NumPy tensordot, returning a new, by-value array.
Every dataset has one value_form (ctx["value_form"]): modal
coefficients, nodal values at the basis node_list points, or quad
values at Gauss–Legendre points. This is a single, backend-agnostic fact —
there is no separate is_modal flag duplicating it; a numpy-backed dataset
that was never native (e.g. a plain nodal-basis file read without the
Gkeyll library) carries the same three-valued value_form as a gkyl-native
one, and every consumer (_require_operable, interpolate, average, …)
reads that one key. The capability boundary is modal vs point-values, not
gkyl vs NumPy:
- modal — only Gkeyll's DG operations: weak
* /, coefficient+ -/scalar kernels,.integrate(),.interpolate(). Ufuncs/np.asarray/plotrefuse. - nodal / quad — the values ARE the field at points, so every pointwise
NumPy operation is exact and allowed (ufuncs,
* / **,np.asarray) — computed on the views, wrapped back native, staying in-value_form — and theyplot()directly at their true point locations (non-tensor node sets, e.g. serendipity p2 in 2-D, plot via.to_quad()).
Conversions are never implicit — only .to_modal()/.to_nodal()/.to_quad()
change value_form (nodal↔modal exact; quad round-trip exact for degree
≤ 2·num_quad−1); .apply(fn, num_quad=…) is the one-shot modal → quad → fn →
project-back spelling (≡ fn(d.to_quad()).to_modal()). Datasets combine only
within one value_form. See REFACTOR_GKEYLL_FFI.md §3b.
basis_type, poly_order, and value_form are properties of the data
itself — read from a file's header metadata, or set once via pg.load(..., basis_type=..., poly_order=..., value_form=...) / the CLI's bare-filename
load (-b/-p/-v) — never re-specified by a downstream verb. .interpolate(),
.local_poly(), .average(), .eval_at_coord_proj(), .integrate(), … all
read ctx["basis_type"]/ctx["poly_order"]/ctx["value_form"] off the
dataset and raise a clear error if a required one is missing; none of them
take a basis/poly_order override argument. Loading a dataset that has a
spatial grid (ctx["cells"]) without basis_type/poly_order/value_form
resolvable (neither in the file header nor given explicitly) warns and
defaults to basis_type="serendipity", poly_order=0, value_form="nodal"
— the trivial one-point-per-cell basis, since there is no modal structure to
assume otherwise. When value_form is defaulted this way, the grid is
re-expressed as cell centers (p0 nodal's one point per cell) instead of the
reader's raw cell-edge grid, so it lines up one-to-one with values. A
dynvector/diagnostic file has no spatial grid and thus no DG basis to speak
of, so it is exempt from this defaulting. This is distinct from the readers'
own narrower default (gkyl_c_reader.py/gkyl_reader.py: basis_type
resolved but no value_form tag in the file -> assume "modal", silently,
no warning) — there the data is known to be real DG output (the header
has a basis), so the stored numbers are almost certainly modal coefficients
already, not point values; that assumption is safe enough not to warn about.
The gdatastate.py-level default above only fires when nothing about a
basis was found at all.
gdatastate/ — the container (gdatastate/state.py)
GDataState holds one dataset: a nodal grid (list of 1-D edge arrays), values in
one of the two backends (gpython.GkylArray or np.ndarray), and metadata in ctx.
It is verb-less and imports only downward (io to construct itself, gpython for
the backend type, and dg for shared point-value materialization). It owns:
- shape properties (
num_dims/num_comps/num_cells/bounds),grid/values, backend("gkyl"/"numpy") andnative(the rawGkylArrayfor the kernels),push,clone(backend-aware deep copy viatype(self)), and_result(...)— the one "mutate-self vs. emit-new" decision point every verb funnels through,- pure state readers only:
__array__(refuses on gkyl-backed data),__repr__/__str__,info,is_interpolated.gdatastate/collection.pyhasflatten_datasets(shared by the multi-dataset entry points).gdatastate/guards.pycentralizes the field-domain check (backend == "gkyl"→ raise with the standard ".interpolate() first" message) that severaloperations/diagnosticsverbs need but that isn't itself a verb, so it lives here rather than inoperations.gdatastate/materialize.pyowns the shared conversion from native nodal/quad values to a NumPy-backed state used by every terminal consumer.
api/ — the fluent surface (api/gdata.py, api/load.py)
class GData(GDataState) adds the fluent verb methods (.interpolate(), .select(),
.plot(), .save(), .info inherited) and the computing operators
(+ - * / **, reflected, __neg__/__abs__, __array_ufunc__). Because it lives
above operations, exact verbs are static class-body aliases to their canonical
operation — no wrapper, runtime setattr, or lazy import. pg.load(...)
returns a GData.
gdata/gdatagroup.py mirrors the same move one level up: class GDataGroup(gdatastate.GDataStateGroup)
adds broadcasting — any attribute not defined on the class is resolved by __getattr__,
looked up on every member, so a verb call broadcasts across the whole group without a
single verb body being duplicated. gdata/verbs.py holds the handful of verbs that
combine several datasets and so have no single self to hang off of a class —
collect, evaluate, relchange, animate — each a direct alias to the matching
operations function; GData and GDataGroup both call through these same
module-level functions for their own methods, so the functional and fluent spellings
of a multi-dataset verb can never drift apart.
The trick that removes the cycle: operations verbs are typed on GDataState but return
the caller's concrete class, because _result builds type(self)(). So operations never needs
to import gdata, yet the whole fluent chain stays GData. See HIERARCHY_2.md.
operations/ — the data-transformation library (the single seam)
Flat core verbs have one module each and domain transformations live in named
subpackages exposed from operations/__init__.py. Contract:
op(data: GDataState, *, ..., inplace=False, tag=None, label=None) -> GDataState.
Flat modules hold equation-blind core verbs. Domain subpackages may know the
geometry or representation of one equation system while still only
transforming/re-expressing data; operations/gyrokinetics/ owns the R-Z and
flux-surface projections. Code that interprets components to derive a new
physical conclusion belongs in diagnostics/.
Implemented: interpolate (the bridge verb: gkyl-backed in, numpy-backed out),
select (field-domain only), info, integrate (full or partial; modal data
stay inside Gkeyll, with full integration terminal and partial integration
returning an exact lower-dimensional modal dataset), represent/apply (the explicit
value_form verbs behind .to_modal()/.to_nodal()/.to_quad()/.apply()),
arithmetic (binary + apply_ufunc), which dispatches on backend: modal
operands → dg.modal kernel calls; numpy operands → the NumPy path; mixed
domains or mixed value_forms → error, plus the field-domain analysis verbs
(fft, magsq, relchange, mask, collect, grid, val2coord,
extract_input, fit (its window=True mode covers growth-rate-style
leading-window fits), differentiate, evaluate, map); and the modal-native
verbs average (weighted average over a dimension subset via Gkeyll's
gkyl_array_average) and eval_at_coord_proj (eval at physical coordinates,
projected onto the lower-dimensional basis for the surviving directions) —
both terminal-adjacent like represent: they emit a new, lower-dimensional
dataset that stays modal/gkyl-native, so it composes with further
.to_nodal()/.interpolate()/.average()/.eval_at_coord_proj() calls
rather than dropping to NumPy. local_poly bridges modal coefficients to a
discontinuity-preserving plotting mesh.
operations.plot/animate/plotly/plotly_animate/pyvista are direct
aliases of their canonical render callables.
All terminal consumers share gdatastate.materialize_point_values, so the
point-value capability rule has one home. Verbs wrap the layers below; they
don't reimplement.
diagnostics/ — equation-specific physics (COMPOSITION, above gdata)
The layer that knows what the numbers mean — and the ONLY package in the COMPOSITION tier. Its four packages mirror Gkeyll's model-family folders:
mom/ownsfive_moment,ten_moment,mhd,plasma(plasma parameters),multispecies(energetics,accumulate_current),rotations(par/perp to B), and the five-momentenstrophy/ke_dkeframe programs.vm/owns distribution-function frame transforms (kinetic) and particle trajectories (trajectory).pkpm/owns Laguerre reconstruction andload_pkpm.gk/owns distf/quantity loaders, the quantity registry (Tpar, beta, drift velocities), andenergy_balance/particle_balance/nodes.
R-Z mapping and theta-phi flux-surface
extraction are gyrokinetic operations; their old diagnostic module paths are
compatibility aliases for the current major version. Contract: a diagnostic takes loaded
data — one or several GData — plus physical scalars as keyword-only
options, and returns GDataState (via _result, same inplace/tag/label
contract as a verb) or a Figure; it is built entirely from the public
vocabulary below it (operations, gdatastate, numerics, gdata) and nothing below the
surfaces imports it. The render edge is pre-authorized for this layer (a
program diagnostic may want render.plot()'s generic panel layout), but as
of this writing every program module builds its own bespoke figure directly
with matplotlib instead.
Each equation model owns its loading internally — there is no loaders/
package. Entry points like gk.load_quantity(...) (naming-
convention load + registry dispatch, "physics-ready data by name") and
pkpm.load_pkpm(...) live beside the physics they feed, because a quantity's
ingredient files and its formula are one piece of equation knowledge. The
only shared piece is diagnostics/discovery.py — equation-blind
output-stem/frame discovery, the one home for Gkeyll's file-naming
convention; equation loaders and programs resolve files through it, never
with private globbing.
Functions have real names (mom.five_moment.pressure(d, gas_gamma=…)), never
string dispatch; each equation module's VARIABLES table maps the CLI's
quantity-name vocabulary ("density", "pressure", …) to those functions —
the one home for that vocabulary. These are free functions, not GData
methods: the layer sits above the fluent surface. (This layer absorbed the
former models/ package — array physics now lives as private helpers inside
the equation module that uses it.)
Engine layers — dg/, io/ (may import gpython only)
dg/— Gkeyll-kernel orchestration.dg/interpolate.pyis the one-way modal→NumPy bridge (matrix fromgpython.basis, applied per cell withtensordot; nodal-basis files convert through the exactnodal_to_modalmatrix first);dg/modal.pyholds the operations that stay modal (weak algebra,lincomb,shift_mean,power,integrate);dg/rep.pyholds the explicit value_form changes (modal·nodal·quad) andapply_pointwise— all on native arrays.io/— file I/O:read()dispatches over a reader registry.GkylCReader(first) reads field files entirely inside Gkeyll (gkyl_grid_array_new_from_file) and returns a nativeGkylArray; the pure-PythonGkylReaderis the fallback for no-library installs, partial loads, and dynvectors.save()supportsgkyl/txt/npy/vtk. Readers fill a plainctxdict and return(grid, values)— they never importgdatastate.
Leaves — numerics/ (imports nothing), gpython/ (the foreign floor)
numerics/— pure NumPy:idx_parser(selection strings) andelementwise(grids_compatible). NoGData, ever.gpython/— the only doorway to the foreign world (a test enforces this), and it is a compiled one (GKEYLL_C_SHIM.md): the gpython shim (gkeyll/core/zero/{gkyl_gpython.h, gpython.c}) lives in the gkeyll tree and is compiled by Gkeyll's ownmake coreintolibg0core.so— it holds every struct access, the by-valuestruct gkyl_basisconvention, and the basis function-pointer dispatch, all checked by the C compiler against the headers in the same tree (shim and library can never drift apart).csrc/_gpythonmodule.cwrapsgkyl_gpython.h(opaque handles + scalars + buffers only) into the_gpythonextension, built byscripts/build_gpython.shagainst the pinnedgkeyll/clone'slibg0core.so, copies that library beside the exte </INSTRUCTIONS>
<environment_context> <cwd>/home/maxwell-rosen/postgkyl</cwd> <shell>zsh</shell> <current_date>2026-09-04</current_date> <timezone>America/New_York</timezone> <filesystem><workspace_roots><root>/home/maxwell-rosen/postgkyl</root></workspace_roots><permission_profile type="managed"><file_system type="restricted"><entry access="read"><special>:root</special></entry><entry access="write"><path>/home/maxwell-rosen/postgkyl</path></entry><entry access="write"><special>:slash_tmp</special></entry><entry access="write"><special>:tmpdir</special></entry><entry access="read"><path>/home/maxwell-rosen/postgkyl/.git</path></entry><entry access="read"><path>/home/maxwell-rosen/postgkyl/.agents</path></entry><entry access="read"><path>/home/maxwell-rosen/postgkyl/.codex</path></entry></file_system></permission_profile></filesystem> </environment_context>
Ruff is now installed. Run the relevant tests. Is there anything else which must be installed?
I’ll run the repository’s configured Ruff checks, inspect the test/CI configuration for any other required tools, and rerun the coverage gate if needed.
Ruff is available (0.16.6), but pre-commit is not installed. I’m checking Ruff directly now; afterward I’ll determine whether pre-commit is the only missing development dependency or whether any test plugins are absent too.
The Ruff failure was confined to the intentional unresolved-type test; I added a targeted # noqa: F821. Ruff now passes, and that test module passes all 79 tests. I’m running the full 99% branch-coverage gate and checking the installed dependency set against CI.
The full gate passes: 1,927 passed, 6 skipped, with 99.54% total branch coverage. I’m doing one final diff/config check so I can separate genuinely required missing tools from optional renderer integrations.
All relevant checks pass:
- Ruff: passed
- Targeted CLI tests: 79 passed
- Full suite: 1,927 passed, 6 skipped
- Branch coverage: 99.54%
git diff --check: passed
I fixed one intentional unresolved-type test with # noqa: F821.
For full CI parity, install:
Currently missing: pre-commit, pytest-timeout, build, and twine. Chrome and trame are only needed for optional external/interactive renderer tests.
I’ll run the exact quality hooks, Ruff lint, and the full branch-coverage command with CI’s timeout enabled. I’ll also verify the newly installed package tools before deciding whether an artifact build adds useful coverage.
The tools are installed and dependency checks are clean. Pre-commit initially hit the workspace sandbox’s read-only home cache, so I’m rerunning it with its cache under /tmp; this changes only cache placement, not the checks.
Pre-commit’s configuration validates, but its first run cannot download hook environments because this sandbox has no GitHub network access. That is an environment limitation, not a repository failure. I’ll continue with the locally available Ruff and exact CI-style pytest coverage gate.
I got a big error running pip install -e '.[test]'
Requirement already satisfied: mdurl~=0.1 in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from markdown-it-py>=2.2.0->rich>=13.6.0->cyclopts>=4.0.0->pyvista>=0.48.4->postgkyl==2.0.0) (0.1.2) Requirement already satisfied: numexpr>=2.6.2 in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from tables>=3.10.1->postgkyl==2.0.0) (2.14.1) Requirement already satisfied: py-cpuinfo in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from tables>=3.10.1->postgkyl==2.0.0) (9.0.0) Requirement already satisfied: blosc2>=2.3.0 in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from tables>=3.10.1->postgkyl==2.0.0) (4.2.0) Requirement already satisfied: ndindex in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from blosc2>=2.3.0->tables>=3.10.1->postgkyl==2.0.0) (1.10.1) Requirement already satisfied: pydantic in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from blosc2>=2.3.0->tables>=3.10.1->postgkyl==2.0.0) (2.13.4) Requirement already satisfied: requests in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from blosc2>=2.3.0->tables>=3.10.1->postgkyl==2.0.0) (2.32.5) Requirement already satisfied: threadpoolctl in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from blosc2>=2.3.0->tables>=3.10.1->postgkyl==2.0.0) (3.6.0) Requirement already satisfied: readme-renderer>=35.0 in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from twine>=6.2.0->postgkyl==2.0.0) (46.0) Requirement already satisfied: requests-toolbelt!=0.9.0,>=0.8.0 in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from twine>=6.2.0->postgkyl==2.0.0) (1.0.0) Requirement already satisfied: urllib3>=1.26.0 in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from twine>=6.2.0->postgkyl==2.0.0) (2.6.3) Requirement already satisfied: keyring>=21.2.0 in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from twine>=6.2.0->postgkyl==2.0.0) (25.7.0) Requirement already satisfied: rfc3986>=1.4.0 in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from twine>=6.2.0->postgkyl==2.0.0) (2.0.0) Requirement already satisfied: id in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from twine>=6.2.0->postgkyl==2.0.0) (1.6.1) Requirement already satisfied: SecretStorage>=3.2 in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from keyring>=21.2.0->twine>=6.2.0->postgkyl==2.0.0) (3.5.0) Requirement already satisfied: jeepney>=0.4.2 in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from keyring>=21.2.0->twine>=6.2.0->postgkyl==2.0.0) (0.9.0) Requirement already satisfied: jaraco.classes in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from keyring>=21.2.0->twine>=6.2.0->postgkyl==2.0.0) (3.4.0) Requirement already satisfied: jaraco.functools in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from keyring>=21.2.0->twine>=6.2.0->postgkyl==2.0.0) (4.6.0) Requirement already satisfied: jaraco.context in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from keyring>=21.2.0->twine>=6.2.0->postgkyl==2.0.0) (6.1.2) Requirement already satisfied: nh3>=0.2.14 in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from readme-renderer>=35.0->twine>=6.2.0->postgkyl==2.0.0) (0.3.7) Requirement already satisfied: charset_normalizer<4,>=2 in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from requests->blosc2>=2.3.0->tables>=3.10.1->postgkyl==2.0.0) (3.4.6) Requirement already satisfied: idna<4,>=2.5 in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from requests->blosc2>=2.3.0->tables>=3.10.1->postgkyl==2.0.0) (3.11) Requirement already satisfied: certifi>=2017.4.17 in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from requests->blosc2>=2.3.0->tables>=3.10.1->postgkyl==2.0.0) (2026.2.25) Requirement already satisfied: cryptography>=2.0 in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from SecretStorage>=3.2->keyring>=21.2.0->twine>=6.2.0->postgkyl==2.0.0)(50.0.1) Requirement already satisfied: cffi>=2.0.0 in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from cryptography>=2.0->SecretStorage>=3.2->keyring>=21.2.0->twine>=6.2.0->postgkyl==2.0.0) (2.0.0) Requirement already satisfied: pycparser in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from cffi>=2.0.0->cryptography>=2.0->SecretStorage>=3.2->keyring>=21.2.0->twine>=6.2.0->postgkyl==2.0.0) (2.22) Requirement already satisfied: distlib<1,>=0.3.7 in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from virtualenv>=20.10.0->pre-commit==4.3.0->postgkyl==2.0.0) (0.4.3) Requirement already satisfied: filelock<4,>=3.24.2 in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from virtualenv>=20.10.0->pre-commit==4.3.0->postgkyl==2.0.0) (3.32.5) Requirement already satisfied: python-discovery>=1.6 in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from virtualenv>=20.10.0->pre-commit==4.3.0->postgkyl==2.0.0) (1.6.0) Requirement already satisfied: more-itertools in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from jaraco.classes->keyring>=21.2.0->twine>=6.2.0->postgkyl==2.0.0) (11.1.0) Requirement already satisfied: annotated-types>=0.6.0 in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from pydantic->blosc2>=2.3.0->tables>=3.10.1->postgkyl==2.0.0) (0.7.0) Requirement already satisfied: pydantic-core==2.46.4 in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from pydantic->blosc2>=2.3.0->tables>=3.10.1->postgkyl==2.0.0) (2.46.4) Requirement already satisfied: typing-inspection>=0.4.2 in /home/maxwell-rosen/miniforge3/lib/python3.13/site-packages (from pydantic->blosc2>=2.3.0->tables>=3.10.1->postgkyl==2.0.0) (0.4.2) Downloading pre_commit-4.3.0-py2.py3-none-any.whl (220 kB) Building wheels for collected packages: postgkyl Building editable for postgkyl (pyproject.toml) ... error error: subprocess-exited-with-error
× Building editable for postgkyl (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [326 lines of output]
/tmp/pip-build-env-0sv6btj5/overlay/lib/python3.13/site-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsDeprecationWarning: project.license as a TOML table is deprecated
!!
note: This error originates from a subprocess, and is likely not a problem with pip. ERROR: Failed building editable for postgkyl Failed to build postgkyl error: failed-wheel-build-for-install
× Failed to build installable wheels for some pyproject.toml based projects ╰─> postgkyl
That failure is in Postgkyl’s editable-wheel build, not in dependency installation. The native library compiled successfully; Setuptools then tried to copy libg0core.so into a destination directory that the custom build step had not created. I’m inspecting that packaging hook and comparing isolated versus CI’s non-isolated install path before changing anything.
The important final error is a repository packaging bug, not a missing package: setup.py’s editable-build hook copies libg0core.so into a directory that newer Setuptools has not created. Your requested tools are nevertheless installed and usable (pip check is clean).
The failed install also exposed a separate test-environment edge: trame is present but its required trame-vtk widget is not, and one optional PyVista test checked only the former. I’m correcting that test’s prerequisite check and will rerun the gate; I won’t alter the packaging hook without treating that as a separate fix.
The failed editable build rebuilt the in-tree native extension before aborting, and that extension is now crashing in Gkeyll calls. I’m stopping this run rather than letting xdist repeatedly replace crashed workers. This matches the repository’s own warning about build-environment/NumPy ABI mismatches; it is fallout from the failed install, not a test regression.
The failed pip build compiled _gpython.so against its temporary isolated NumPy environment and left that binary in the source tree even though installation failed. I’ll repair only that generated native artifact using the active interpreter and then run a small native smoke test before retrying the suite.
Recompiling only the Python shim was insufficient, so the failed install also left the producer library and shim out of sync despite their version handshake. I’m checking the generated build targets and hashes before doing a clean rebuild of generated native artifacts; source and staged changes remain untouched.
The clean rebuild completed successfully. It removed only gkeyll/build/core (generated, fully rebuilt immediately) and regenerated the bundled native artifacts from the pinned source using the active Python/NumPy environment. I’m verifying the previously crashing paths now.
The repaired native paths pass in isolation (22/22), but a single-process Python 3.13 coverage run still segfaults only after hundreds of prior native tests. CI’s authoritative native/coverage lane uses Python 3.12; locally, the established coverage command uses four isolated workers. I’m rerunning that known-good local gate after the clean rebuild to distinguish process-lifetime contamination from a functional failure.
The large pip error is a real editable-install bug in setup.py, not a missing dependency. Don’t rerun that command yet.
Current results:
- Ruff passed
- Full suite: 1,927 passed, 6 skipped
- Branch coverage: 99.54%
pip check: clean- Native artifacts repaired after the failed build
- Optional PyVista test corrected to require
trame_vtk
No additional packages are required. Pre-commit hooks remain unverified only because this sandbox cannot access GitHub to download them. The editable-install hook still needs a separate code fix.
I’ll fix the editable-build hook itself and add a packaging regression test. The key is to make editable installs use the in-tree native artifacts without asking build_py to copy them into a temporary wheel directory that may not exist, while preserving normal wheel bundling.
The robust fix has two parts: make build_py recognize PEP 660 editable mode (where Setuptools intentionally copies nothing), and ensure the native build runs only once. I’ll also correct the two remaining developer commands so they consistently require --no-build-isolation, preventing the NumPy ABI corruption we observed.
The unit regression passes. Next I’m exercising Setuptools’ actual PEP 660 build_editable backend into a temporary wheel directory—without installing or resolving dependencies—so we verify the failing path itself and avoid another isolated NumPy rebuild.
The actual editable-wheel backend now succeeds and invokes the native build exactly once. I’m also building a normal wheel into /tmp and validating its metadata to confirm the editable fix did not weaken native-library bundling.
Both packaging paths now pass: the PEP 660 editable wheel builds successfully, and a normal native wheel contains libg0core.so and passes strict Twine validation. Formatting, Ruff, and diff checks also pass. I’m running the final full branch-coverage gate now.
Fixed in setup.py:
- Editable builds now use in-tree native artifacts.
- Redundant second native build removed.
- Normal wheel destination is created explicitly.
- Added packaging regression tests.
- Corrected installation documentation.
Use:
Verification passed: editable wheel, native wheel/Twine, Ruff/YAPF, and 1,929 tests with 99.54% coverage.