[FEATURE] Keep lexical continuations in caller-provided storage #25
Labels
No labels
alpha-1
bug
ci
docs
duplicate
enhancement
help wanted
invalid
performance
phase-6
question
refactor
security
wontfix
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
erix/exsh#25
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Problem and motivation
Mixed quotes and expansions pass through mutually calling lexical scanners. Move their continuation state into caller-provided storage so nesting does not grow the Rust call chain.
Proposed behavior and scope
Use one iterative scanner with explicit continuation frames. Migrate lexer, parser, here-document and runtime callers together, and retire the recursive storage-free entry points. Keep token ranges, quoting, line joins, errors and grammar-produced substitution boundaries unchanged.
Authority, security and reliability
Use separate scratch from live grammar-discovery frames. Add no allocation, capability or fixed nesting limit. Report the exact required storage separately from lexical errors; clear scratch and restore the original lexer state on shortage, so a retry with more storage has the same result. Work frames cannot authorize a substitution boundary.
Acceptance criteria
Alternatives and tradeoffs
A fixed local stack adds a nesting policy; hidden allocation breaks caller ownership. A constant-space replay algorithm would need a separate correctness and performance argument. Explicit caller storage fits the existing parser model and makes exhaustion visible.
Tracking and rollout
Continue the frame work in #4 and PR #3. Coordinate the API migration and catalog adoption before removing old entry points. Phase completion remains in Integration #65.
The iterative lexer and production callers are published in 411998e6. Integration a28bf98a adopts the version-23 workspace producers, including explicit lexical capacity. The first four criteria are checked. The generated API/manual and remaining native source, route and frame work stay open, with the original 60-second gate unchanged.
The public Rustdoc JSON is now generated from the published bc058d90 source, using its native library metadata and exact pinned dependencies. This supplies the input for the maintained manual converter. Manual regeneration and the remaining native frame and deployment requirements are still open.
The storage-layout and manual criterion is complete: Docs f3f144b publishes the updated chapter and generated reference, and Integration 61b0d41 selects it. The remaining criterion includes the frame checker at its existing deadline; #4 tracks that work.