[FEATURE] Provide caller-owned multibyte conversion #17

Open
opened 2026-10-28 04:45:24 +01:00 by erikinkinen · 1 comment
Owner

Problem and motivation

The fixed ASCII character component does not provide multibyte conversion or encoding state. The C runtime needs deterministic conversion with explicit memory ownership before toolchain adoption.

Proposed behavior and scope

Add a no_std Rust conversion core and a separately selected Cargo runtime, with a partial erix/cstd/multibyte.h header. Provide erix_c_ calls and matching hidden versioned entries for mbsinit, mbrlen, mbrtowc, wcrtomb, mbsrtowcs, wcsrtombs, mbsnrtowcs, wcsnrtombs, btowc, wctob, mblen, mbtowc, wctomb, mbstowcs, wcstombs, mbrtoc16, c16rtomb, mbrtoc32 and c32rtomb, plus mbstate_init, mbstate_reset and mb_cur_max.

A caller-owned state explicitly selects ASCII C encoding or strict UTF-8. C accepts only bytes/scalars 0..127; UTF-8 never becomes C implicitly. Null state does not select shared fallback state. Legacy prefixed forms have explicit state and documented complete-input/reset behavior. The header owns fixed-width wide/scalar types rather than borrowing host wchar_t or locale objects.

Append local ENOTTY=41 and EILSEQ=42 coherently to the scalar error catalog and header. Existing compatibility, birth and lifecycle packet vocabularies remain unchanged. Conventional aliases, current-locale selection and upstream workload completeness need their own mapped contracts.

Authority, security and reliability

Conversion acquires no capability, heap, environment, locale file, service or host fallback. Callers retain initialized state and live disjoint input/output through each synchronous call. Count and pointer geometry precede access; incomplete input retains only owned scalar state. Never emit a partial encoded character or consume beyond an explicit input bound. Specify prefix/source updates, query/reset paths and sticky invalid state; malformed input is EILSEQ and structural misuse is EINVAL. Success preserves original thread errno; error writes require its admitted thread image.

Acceptance criteria

  • Implement the complete declared roster through one core, with useful Rustdoc, FFI safety, state-layout and return contracts.
  • Cover independent states, ASCII/C high-byte refusal, UTF-8 scalar edges, every split point, overlong/surrogate/out-of-range encodings, UTF-16 pending units, NUL/reset, query, capacity, source-position and unchanged-tail/error behavior.
  • Preserve existing errno and packet oracles; positively check both new scalar assignments and reject them in unchanged packets.
  • Add exact C/C++ signatures and conflicting-signature/missing/duplicate-link controls, a real freestanding C probe, selected native runtime/source graph and manual/API contracts.
  • Exercise the C workload through its original native process/guest path and document remaining current-locale and upstream API gaps.

Alternatives and tradeoffs

Unconditional UTF-8 mislabeled as C and hidden process-global state are rejected. Explicit encodings make streaming state testable without locale discovery; future locale adapters can select them under a separate lifetime contract. A narrow decoder would be smaller but leave the declared families incomplete.

Tracking and rollout

Extend the C runtime design and coordinate the pure compatibility ABI and toolchain integration. Coordinate ENOTTY with restricted terminal controls. Review the state/encoding and full interface sheet before implementation, then publish coherent library, runtime, source selection, C consumer and manual changes. No conventional libc or complete guest-build claim follows from namespaced source implementation.

## Problem and motivation The fixed ASCII character component does not provide multibyte conversion or encoding state. The C runtime needs deterministic conversion with explicit memory ownership before toolchain adoption. ## Proposed behavior and scope Add a no_std Rust conversion core and a separately selected Cargo runtime, with a partial erix/cstd/multibyte.h header. Provide erix_c_ calls and matching hidden versioned entries for mbsinit, mbrlen, mbrtowc, wcrtomb, mbsrtowcs, wcsrtombs, mbsnrtowcs, wcsnrtombs, btowc, wctob, mblen, mbtowc, wctomb, mbstowcs, wcstombs, mbrtoc16, c16rtomb, mbrtoc32 and c32rtomb, plus mbstate_init, mbstate_reset and mb_cur_max. A caller-owned state explicitly selects ASCII C encoding or strict UTF-8. C accepts only bytes/scalars 0..127; UTF-8 never becomes C implicitly. Null state does not select shared fallback state. Legacy prefixed forms have explicit state and documented complete-input/reset behavior. The header owns fixed-width wide/scalar types rather than borrowing host wchar_t or locale objects. Append local ENOTTY=41 and EILSEQ=42 coherently to the scalar error catalog and header. Existing compatibility, birth and lifecycle packet vocabularies remain unchanged. Conventional aliases, current-locale selection and upstream workload completeness need their own mapped contracts. ## Authority, security and reliability Conversion acquires no capability, heap, environment, locale file, service or host fallback. Callers retain initialized state and live disjoint input/output through each synchronous call. Count and pointer geometry precede access; incomplete input retains only owned scalar state. Never emit a partial encoded character or consume beyond an explicit input bound. Specify prefix/source updates, query/reset paths and sticky invalid state; malformed input is EILSEQ and structural misuse is EINVAL. Success preserves original thread errno; error writes require its admitted thread image. ## Acceptance criteria - [x] Implement the complete declared roster through one core, with useful Rustdoc, FFI safety, state-layout and return contracts. - [x] Cover independent states, ASCII/C high-byte refusal, UTF-8 scalar edges, every split point, overlong/surrogate/out-of-range encodings, UTF-16 pending units, NUL/reset, query, capacity, source-position and unchanged-tail/error behavior. - [x] Preserve existing errno and packet oracles; positively check both new scalar assignments and reject them in unchanged packets. - [ ] Add exact C/C++ signatures and conflicting-signature/missing/duplicate-link controls, a real freestanding C probe, selected native runtime/source graph and manual/API contracts. - [ ] Exercise the C workload through its original native process/guest path and document remaining current-locale and upstream API gaps. ## Alternatives and tradeoffs Unconditional UTF-8 mislabeled as C and hidden process-global state are rejected. Explicit encodings make streaming state testable without locale discovery; future locale adapters can select them under a separate lifetime contract. A narrow decoder would be smaller but leave the declared families incomplete. ## Tracking and rollout Extend the [C runtime design](https://git.erikinkinen.fi/erix/lib-cstd/issues/1) and coordinate the [pure compatibility ABI](https://git.erikinkinen.fi/erix/lib-posixabi/issues/1) and [toolchain integration](https://git.erikinkinen.fi/erix/integration/issues/7). Coordinate ENOTTY with [restricted terminal controls](https://git.erikinkinen.fi/erix/ttyd/issues/5). Review the state/encoding and full interface sheet before implementation, then publish coherent library, runtime, source selection, C consumer and manual changes. No conventional libc or complete guest-build claim follows from namespaced source implementation.
Author
Owner

The 22 declared conversion/state calls are implemented in one caller-owned ASCII/UTF-8 core, with the native runtime, C header and independent conversion/error controls. The matching Posixd consumer, Integration producer and manual are pushed. The integrated native/VM run and new runtime API reference remain open, along with conventional/current-locale adoption.

The 22 declared conversion/state calls are implemented in one caller-owned ASCII/UTF-8 core, with the native runtime, C header and independent conversion/error controls. The matching [Posixd consumer](https://git.erikinkinen.fi/erix/posixd/commit/fac72a2145735a6ae702b3774d1cf1dc4f611a50), [Integration producer](https://git.erikinkinen.fi/erix/integration/commit/3477eb9ca611a0f0423207dfb21510a69dade8d8) and [manual](https://git.erikinkinen.fi/erix/docs/commit/c2632fc4b33dbe900aaee088fc577c14fe442eef) are pushed. The integrated native/VM run and new runtime API reference remain open, along with conventional/current-locale adoption.
erikinkinen added this to the Alpha 1 milestone 2026-10-30 09:34:41 +01:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
erix/lib-cstd#17
No description provided.