# Postgkyl documentation integration plan

Keep Postgkyl documentation in this repository and publish it as a section of the existing Gkeyll Sphinx website. The website selects a Postgkyl revision; that revision supplies the prose, API and CLI reference, executable examples, input data, and figure generation. This is a proposed implementation plan; the website integration has not yet been implemented.

Reviewed against Postgkyl 9355e5614914d0255b97c9729588cab2f500b8e4 and gkyl-doc 3c339bb46df318025a141d88fcf86fcad9133677.

Existing foundations

• gkyl-doc builds source/ with Sphinx and Furo. Its active toctree has no Postgkyl entry; the previous documentation lives in source_archive/postgkyl/. source/conf.py still references postgkyl/_static.
• The website's .readthedocs.yaml selects Python 3.8, while Postgkyl declares Python >=3.10. Use Python 3.12 for the combined build, matching Postgkyl's principal CI environment, and validate the resolved dependencies together.
• Public function signatures and Google-style docstrings already supply both Python help and the generated CLI. tests/test_documentation.py checks this contract. Extend that machinery rather than writing another command registry.
• examples/scripts/ and examples/cli_tutorial.md are already executable documentation, exercised by tests/test_examples.py.
• tests/test_data/ contains real gyrokinetic output. Synthetic fixtures are produced by tests/generate_test_data.py into the gitignored tests/test_data/generated/ directory.

Ownership and build contract

Material	Authoritative home	Website treatment
Concepts and task guides	Postgkyl docs/	Stage during build
Parameters, defaults, return values	Implementing function signatures and docstrings	Generate reference
CLI names and options	Existing compiled CLI and command metadata	Generate reference
Tutorial code	examples/scripts/ and existing CLI tutorial	Include the tested source
Synthetic data	tests/generate_test_data.py	Generate before running examples
Simulation fixtures	tests/test_data/	Read from selected revision
Figures	Outputs of example scripts	Generate into ignored build directories
Theme, navigation, hosting	gkyl-doc	Add the section and build integration
Published Postgkyl revision	gkyl-doc Git submodule entry	Use for code, docs, examples, and data together

Add Postgkyl as external/postgkyl in gkyl-doc, pinned by the Git submodule commit. Do not maintain a second revision file or install an unrelated PyPI release. Normal website builds use that pin; local integration checks may explicitly supply a different checkout. Validate that the imported package comes from the selected checkout.

Add one build entry point in Postgkyl, proposed as scripts/build_docs.py, with explicit source and output paths. It prepares a self-contained Sphinx source subtree: guides, generated reference pages, literal example sources, figures, and downloadable inputs. gkyl-doc invokes it to populate the ignored source/postgkyl/ directory before Sphinx reads the toctree. Clean only this owned output directory so removed pages cannot survive a rebuild. No generated pages or figures are committed to gkyl-doc.

The same preparation step supports a standalone Postgkyl preview. A small Postgkyl-owned Sphinx extension supplies any shared directives and setup to both builds; the host retains its own theme. Keep documentation machinery outside src/postgkyl/ so the library's import DAG does not change.

Build sequence:

1. Check out the selected revision, including its test fixtures.
2. Install NumPy and build tools, then install that checkout with --no-build-isolation and a new docs extra. Declare Postgkyl documentation dependencies once in pyproject.toml. Build the native bridge through the existing installation flow and require gpython.available().
3. Generate fixtures explicitly; a documentation build must not rely on pytest having run first. Execute the gallery headlessly with MPLBACKEND=Agg and PGKYL_EXAMPLE_OUTPUT pointing into the build directory.
4. Generate reference pages and stage sources/assets with relative links that work both standalone and under postgkyl/.
5. Build the combined website with its existing Sphinx invocation.

Use Read the Docs submodule configuration and build hooks for these steps. Measure a cold native build on the hosted service during the integration milestone; local success alone does not establish its time or memory budget. Do not silently skip examples or substitute mock imports when the bridge fails. Caching can be added if needed, keyed by the source revision, Gkeyll revision, Python, NumPy ABI, and platform.

Reader-facing section

Add postgkyl/index to source/index.rst, with these destinations:

• Start here: installation, verifying the bridge, obtaining example data, and one complete load → inspect → interpolate → lineout → plot → save task.
• Working with data: grids and components; modal, nodal, and quadrature representations; explicit conversions; DG versus pointwise arithmetic; integration and averaging; groups and CLI pipelines.
• Tutorials: scientific questions answered with executable examples and the corresponding CLI spelling where available.
• Reference: loading, fluent API, operations, diagnostics grouped by model family, rendering, and the complete generated command inventory.
• Troubleshooting and contributing: missing metadata, bridge builds, representation errors, headless rendering, and building these docs locally.

Include existing installation instructions from the README using a Markdown include mechanism, or move them once and replace the README section with a link. Do the same for the CLI tutorial; do not hand-copy either. Use Sphinx autodoc plus Napoleon for Google-style docstrings. Use literal includes of named script regions so displayed code is the code that runs.

Generate CLI pages from the actual Click commands produced by postgkyl.cli.app, preserving underscore names, defaults, boolean behavior, and command groups. Derive API inventory from public exports and public class members, including Python-only members that CLI discovery intentionally hides. Document canonical callables once and link their fluent/functional aliases. Derive diagnostic quantity listings from their existing registries.

Describe capability rules using value_form, not the oversimplification that all native data reject NumPy or plotting. Explain that basis metadata is fixed at loading and that modal arithmetic has different semantics from arithmetic after interpolation. Fix inaccurate source docstrings at their implementation.

Initial tutorial gallery

Reader's task	Existing source/data	Figure and teaching outcome
Inspect a field and extract a lineout	01_quickstart.py; generated 2d_c2p_rot45_ms_p1.gkyl	Field map plus lineout; identify the input as an analytic coordinate map, explain axes and components, and demonstrate save/reload
Compare two profile calculations	mirror_comparison.py; generated mirror_comparison_*_1d_ms_p1.gkyl	Four labeled panels with linear/log scales; explain that these are synthetic profiles and that the script rescales temperatures
Recover fluid primitives	03_diagnostics_five_moment.py	Density and pressure across a shock-tube initial discontinuity; show conserved-to-primitive conversion
Inspect gyrokinetic density and velocity-space structure	04_gyrokinetics.py; rt_gk_tcv_iwl* moment, distribution, and geometry files	Density profile and fixed-mu distribution slice; explain Jacobian correction and velocity coordinates
View density in physical R–Z coordinates	05_gk_rz.py; rt_gk_tcv_nt_iwl_3x2v_p1-elc_M0_5.gkyl and matching map	R–Z density map; show which geometry files are needed and when projection reuse helps
Measure a growth interval	Existing CLI tutorial and generated energy_dynvec.gkyl	Energy history with fitted interval and residuals; distinguish early growth from saturation
Choose DG or pointwise arithmetic	02_arithmetic_and_numpy.py	Computed comparison with an analytic check; explain why interpolating first changes the question

Each page states the scientific question, input provenance, prerequisites, copyable execution command, expected interpretation, and a small numerical check. Include the complete script and the minimum required input bundle for download, preserving relative filenames and geometry companions. Provide clear axis labels, units only when verified, captions, and alt text. Generate PNG or SVG for browser display; keep PDF downloads where useful.

Before publishing, audit the existing tutorials' scientific explanations: the shock example is an initial condition, not an evolved shock solution; the mirror profiles are not a simulation convergence study; the dynvector uses logistic curves rather than measured instability data. Check the M1 description and its illustrative mass=2.0 against the quantity implementation and available provenance before presenting physical units or conclusions.

No new simulation output is needed for the first gallery. Move the shock-tube initial-condition construction into the existing test-data generator so the tutorial loads a reproducible fixture. Add deterministic analytic fixtures there only if a tutorial needs a known growth rate or a clearer field example. Store generated files under tests/test_data/generated/, with correct basis, value-form, component, and time metadata. Test numerical invariants rather than pixel equality. Do not add random coefficient fields as physical examples.

Implementation milestones and acceptance

1. Documentation foundation in Postgkyl. Add docs/, the docs extra, reference generation, and the standalone preparation/build command. Acceptance: a clean checkout builds navigable reference pages with actual signatures and links; changing a source default changes the reference.
2. Executable gallery. Reuse and improve existing examples, add the shock fixture, and generate figures and downloads. Extend existing example tests for scientific invariants and required outputs. Acceptance: each displayed command runs, inputs are downloadable, and every figure is inspected for readable labels, scaling, clipping, and truthful captions.
3. Website integration. Add the pinned submodule, navigation entry, ignored staging directory, shared extension setup, and RTD hooks. Update Python and remove or reconcile obsolete static-path configuration. Acceptance: the complete website builds from a clean clone and the new section appears in navigation, search, and a hosted preview.
4. Continuous checks and maintenance. Add a Postgkyl docs CI lane and a gkyl-doc integration lane. Require the bridge and run the existing documentation/example tests; build Postgkyl docs with warnings as errors. Check the host's existing warning baseline separately and fix integration issues without blanket warning suppression. Check internal links and downloads on every build; external link checks can run separately. Acceptance: a broken example, missing figure, invalid reference, or wrong imported revision fails the build.
5. Publication and revision updates. Merge the Postgkyl implementation, then select its tested commit in gkyl-doc. A subsequent code release updates that submodule pin and rebuilds the site; it does not require prose edits in gkyl-doc. Include the Postgkyl version/commit and source-edit links on its pages, pointing to Postgkyl rather than the generated host files. Retain archived material as historical and map old entry URLs to the new landing page where hosting supports redirects. Updating the pin is the initial maintenance workflow; cross-repository update automation can follow if needed.

The current local check passed: POSTGKYL_REQUIRE_GKEYLL=1 MPLBACKEND=Agg python -m pytest tests/test_examples.py tests/test_documentation.py -q — 30 tests passed. This verifies the reusable baseline, not the proposed website build or the scientific accuracy of every existing caption.

Integration references

• Sphinx autodoc and Napoleon provide signature/docstring extraction and Google-style parsing.
• Sphinx literal includes allow pages to display the executable source.
• Read the Docs configuration supports selecting Python and checking out submodules.
• Read the Docs build hooks provide the preparation steps before the Sphinx build.

Implemented in both repositories:

• Generated time-series data, collect examples, and two animations.

• Plotly 2D surfaces/3D isosurfaces and PyVista examples.

• Direct grid/value access and manual NumPy manipulation.

• Paired Python scripts and CLI commands for all 19 published figures, with automated equivalence checks.

• 171 individual Python API reference pages linked to CLI commands.

• Python 3.12 documentation CI, with website builds sourcing Postgkyl main.

Validation: 41 documentation/example tests and 3 website integration tests passed. The complete website builds with warnings treated as errors.

Open the local preview (/home/maxwell-rosen/Documents/gkyl- doc/build/html/postgkyl/index.html). Changes are local; nothing has been published.