Add caller-owned string tokenization #47

Manually merged
Owner

Split mutable byte strings with an explicit continuation pointer and unsigned separator membership. Supply the native alias through the original instruction companion without selecting a heap, thread runtime or shared parser state.

Depends on #46. Refs #17. Tracks erix/integration#65.

  • Include the scoped source and contract changes.

  • Use main-tracking dependency declarations, frozen explicit sources and complete the original runtime checks.

Split mutable byte strings with an explicit continuation pointer and unsigned separator membership. Supply the native alias through the original instruction companion without selecting a heap, thread runtime or shared parser state. Depends on #46. Refs #17. Tracks erix/integration#65. - [x] Include the scoped source and contract changes. - [x] Use main-tracking dependency declarations, frozen explicit sources and complete the original runtime checks.
erikinkinen added this to the Alpha 1 milestone 2026-10-30 19:18:51 +01:00
feat: Add caller-owned string tokenization
Some checks failed
CI / memory (pull_request) Has been skipped
CI / thread (pull_request) Has been skipped
CI / markdown (pull_request) Has been skipped
CI / markdown (push) Failing after 14s
CI / memory (push) Failing after 1m11s
CI / thread (push) Successful in 1m26s
465b091fe6
Split mutable byte strings with an explicit continuation pointer and unsigned separator membership. Supply the native alias through the original instruction companion without selecting a heap, thread runtime or shared parser state.
docs: Repair Markdown paragraph wrapping
Some checks failed
CI / memory (pull_request) Has been skipped
CI / thread (pull_request) Has been skipped
CI / markdown (pull_request) Has been skipped
CI / markdown (push) Successful in 8s
CI / memory (push) Failing after 51s
CI / thread (push) Successful in 1m4s
10cdb9ca9f
Keep the original feature prose and all source inputs while correcting
Markdown spacing and line wrapping.
ci: Feed frozen development sources to the native Cargo owner
Some checks failed
CI / memory (pull_request) Has been skipped
CI / thread (pull_request) Has been skipped
CI / markdown (pull_request) Has been skipped
CI / markdown (push) Successful in 9s
CI / memory (push) Failing after 56s
CI / thread (push) Successful in 1m9s
f258ba6c22
Resolve shared companion branches before native acceptance and retain the selected snapshot with its helper and query evidence. Keep the published process, instruction and original-source owners unchanged; the existing pipeline acquires exact selected revisions without another branch query.
erikinkinen changed title from WIP: Add caller-owned string tokenization to Add caller-owned string tokenization 2026-11-15 01:26:37 +01:00
Author
Owner

The prepared branch passes the complete original tokenizer pipeline. The freestanding debug probe now exercises its two original cases without pulling in compiler panic code; the token, continuation, guard and callback checks are unchanged.

Ready for human review at 26200b2a84601f751975c76dc934d76d7d125e18; approval of that exact head is pending. The original source branch remains intact. Guest and release-image acceptance remain separate.

The [prepared branch](https://git.erikinkinen.fi/erix/lib-cstd/src/branch/feature/caller-owned-string-tokenization-merge-ready-26200b2a) passes the [complete original tokenizer pipeline](https://git.erikinkinen.fi/erix/lib-cstd/actions/runs/808). The freestanding debug probe now exercises its two original cases without pulling in compiler panic code; the token, continuation, guard and callback checks are unchanged. Ready for human review at `26200b2a84601f751975c76dc934d76d7d125e18`; approval of that exact head is pending. The original source branch remains intact. Guest and release-image acceptance remain separate.
erikinkinen changed target branch from feature/heap-owned-string-duplication to feature/native-producer-cleanup-receipts 2026-11-15 02:03:26 +01:00
erikinkinen manually merged commit 2ec70eb4f8 into feature/native-producer-cleanup-receipts 2026-11-15 02:03:26 +01:00
Sign in to join this conversation.
No description provided.