Skip to content

Add interactive pagination for JSON list output#5016

Open
simonfaltum wants to merge 3 commits intomainfrom
simonfaltum/list-json-pager
Open

Add interactive pagination for JSON list output#5016
simonfaltum wants to merge 3 commits intomainfrom
simonfaltum/list-json-pager

Conversation

@simonfaltum
Copy link
Copy Markdown
Member

Why

List commands without a row template fall through to the JSON renderer, which today dumps the entire array in one go. For workspaces with hundreds of apps, jobs, pipelines, or files, the output scrolls past before you can read any of it. An interactive terminal should get a chance to step through the output.

This PR adds that pager for the JSON case. A follow-up (#5015) adds the same interaction for commands that do have a row template, reusing the shared infrastructure introduced here.

Changes

Before: RenderIterator always emitted the entire JSON array at once.

Now: when stdin, stdout, and stderr are all TTYs and the command has no row template, databricks <resource> list streams 50 JSON items at a time and prompts on stderr:

[space] more  [enter] all  [q|esc] quit

SPACE fetches and renders the next page. ENTER drains the remaining iterator (still interruptible by q/esc/Ctrl+C between pages). q/esc/Ctrl+C stop immediately. The emitted JSON is always a syntactically valid array — even on early quit — so anything that redirects stdout still gets parseable output.

Piped output and redirected stdout keep the existing non-paged behavior: the capability check requires all three streams to be TTYs.

New files under libs/cmdio/:

  • capabilities.goSupportsPager() (stdin + stdout + stderr all TTYs, not Git Bash).
  • pager.go — shared plumbing: raw-mode stdin setup with a key-reader goroutine, pagerNextKey / pagerShouldQuit, a crlfWriter to compensate for the terminal's cleared OPOST flag while raw mode is active, and the prompt/key constants.
  • paged_json.go — the JSON pager itself. Defers writing the opening [ until the first item is encoded, so empty iterators and iterators that error before yielding produce valid [].
  • render.goRenderIterator routes to the JSON pager when the capability check passes and no row template is set.

No cmd/ changes. No new public API beyond Capabilities.SupportsPager.

Test plan

  • go test ./libs/cmdio/... (all passing, new coverage includes crlfWriter, the key helpers, and every pager control path for JSON output).
  • make checks passes.
  • make lintfull passes (0 issues).
  • Manual smoke in a TTY against a command that renders as JSON — space fetches pages, enter drains, q/esc/Ctrl+C quit, the on-screen JSON remains valid after any of these.
  • Manual smoke with piped stdout (| jq) — output matches main.

When stdin, stdout, and stderr are all TTYs and a list command has no
row template (so the CLI falls back to the JSON renderer), dumping
hundreds of items at once scrolls everything past the user before
they can read any of it. Introduce a simple interactive pager that
streams 50 items at a time and asks the user what to do next on
stderr:

    [space] more  [enter] all  [q|esc] quit

Piped output and redirected stdout keep the existing non-paged
behavior — the capability check requires all three streams to be
TTYs. The accumulated output is always a syntactically valid JSON
array, even if the user quits early, so readers that capture stdout
still get parseable JSON.

New in libs/cmdio:

- capabilities.go: `SupportsPager()` — stdin + stdout + stderr TTYs,
  not Git Bash.
- pager.go: shared plumbing for any interactive pager we add later —
  raw-mode stdin setup with a key-reader goroutine, `pagerNextKey` /
  `pagerShouldQuit`, `crlfWriter` to compensate for the terminal's
  cleared OPOST flag while raw mode is active, and the shared
  prompt/key constants.
- paged_json.go: the JSON pager. Defers writing the opening bracket
  until the first item is encoded, so empty iterators and iterators
  that error before yielding produce a valid `[]` instead of a
  half-open array.
- render.go: `RenderIterator` routes to the JSON pager when the
  capability check passes and no row template is registered.

Test coverage:

- crlfWriter newline translation (6 cases).
- `pagerShouldQuit` / `pagerNextKey` behavior on quit keys,
  non-quit keys, and closed channels.
- JSON pager: fits in one page, SPACE one/two more pages, ENTER
  drains, q/esc/Ctrl+C quit, Ctrl+C interrupts a drain, empty
  iterator, `--limit` respected, fetch errors preserve valid JSON,
  prompt goes to the prompts stream only.

Co-authored-by: Isaac
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 17, 2026

Approval status: pending

/libs/cmdio/ - needs approval

6 files changed
Suggested: @mihaimitrea-db
Also eligible: @tanmay-db, @renaudhartert-db, @hectorcast-db, @parthban-db, @Divyansh-db, @tejaskochar-db, @chrisst, @rauchy

General files (require maintainer)

Files: NEXT_CHANGELOG.md, NOTICE, go.mod
Based on git history:

  • @pietern -- recent work in libs/cmdio/, ./

Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @shreyas-goenka, @renaudhartert-db) can approve all areas.
See OWNERS for ownership rules.

simonfaltum added a commit that referenced this pull request Apr 17, 2026
Depends on #5016.

Extends the interactive pager introduced in #5016 to commands that
register a row template (jobs, clusters, apps, pipelines, etc.).
Reuses the shared plumbing from that PR — raw-mode key reader,
crlfWriter, prompt helpers, SupportsPager capability — and adds only
the template-specific rendering on top.

Shape of the new code:

- paged_template.go: the template pager. Executes the header + row
  templates into an intermediate buffer per batch, splits by tab,
  locks visual column widths from the first batch, and pads every
  subsequent batch to those widths. The output matches the
  non-paged tabwriter path byte-for-byte for single-page results
  and stays aligned across pages for longer ones.
- render.go: `RenderIterator` now routes to the template pager
  when a row template is set, and to the JSON pager otherwise.

Covers the subtle rendering bugs that come up when you drop into
raw mode and page output:

- `term.MakeRaw` clears OPOST, disabling '\n'→'\r\n' translation; the
  already-shared crlfWriter fixes the staircase effect.
- Header and row templates must parse into independent
  *template.Template instances so the second Parse doesn't overwrite
  the first (otherwise every row flush re-emits the header text).
- An empty iterator still flushes the header.
- Column widths are locked from the first batch so a short final
  batch doesn't visibly compress vs the wider batches above it.

Co-authored-by: Isaac
go mod tidy promoted golang.org/x/term from an indirect dependency to
a direct one (the JSON pager uses it for raw-mode stdin). The repo's
TestRequireSPDXLicenseComment in internal/build rejects direct
dependencies that don't carry an SPDX identifier in a comment, which
tripped `make test` on every platform.

Move the dependency into the main require block alongside the other
golang.org/x packages and add the `// BSD-3-Clause` comment that
matches its upstream license.

Co-authored-by: Isaac
simonfaltum added a commit that referenced this pull request Apr 17, 2026
Mirrors the fix in the base PR (#5016). go mod tidy promoted
golang.org/x/term from indirect to a direct dependency, and the
repo's TestRequireSPDXLicenseComment in internal/build rejects
direct dependencies without an SPDX identifier comment — failing
`make test` on every platform.

Move the dependency into the main require block with the correct
`// BSD-3-Clause` comment. This commit is independent from #5016
so 5015 can land on top of main; once the base PR merges, git will
resolve this trivially on rebase.

Co-authored-by: Isaac
TestNoticeFileCompleteness cross-checks the BSD-3-Clause section of
NOTICE against the go.mod require block. Adding golang.org/x/term
as a direct dependency (for raw-mode stdin) also requires adding
its attribution to NOTICE. Mirror the existing entries for
golang.org/x/sys and golang.org/x/text.

Co-authored-by: Isaac
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant