alpha-2027 · draft-alpha-2027 · tests/language/text/alpha-testsView source

Feature contract: byte literals#

1. Lexical and grammar rules, including precedence and termination#

A byte-string-literal is b"…", with no whitespace in the prefix. Raw ASCII, escaped quote/backslash, \n, \t, \r, and exactly two hex digits after \x are accepted. Raw non-ASCII and CR/LF are rejected. Unicode scalar escapes belong to Text. Spaces, parentheses and -- within the token are content; b"" is empty.

2. Scope and binding rules, including generated names#

No bindings are introduced. Literal contents never participate in identifier or comment recognition. Identifiers remain ASCII.

3. Type and quantity requirements#

The result is existing core Bytes. It carries no UTF-8 validity claim; b"\xff" is one byte,255. Converting to Text requires checked UTF-8 decoding.

4. Effects, totality, and evaluation-order rules#

Literal construction is pure. Exact bytes have no locale or encoding conversion. Byte equality is exact; path and OS validation are separate boundaries.

5. Elaboration into existing core or a specifically versioned new core extension#

b"AB\x00" becomes (bytes 65 66 0) through the existing BytesLiteral node. No new core constructor, proof obligation, or serialization tag.

6. Representation and target support, where relevant#

Uses the existing Bytes representation. Alpha payload scanner reuses the integer literal hex-digit decoder; independent exact-byte proofs cover00,7f,80,ff. Native output is verified on the evaluator-carrying executable lane. Direct file-writing lowering is currently refused by the effect-sequencing gate.

7. Error codes and primary/secondary origins#

ALPHA-LITERAL-BYTES-NON-ASCII, -BYTE-ESCAPE, -UNKNOWN-ESCAPE, -INCOMPLETE-ESCAPE, -UNTERMINATED, and -NEWLINE distinguish failures. Whole-source malformed UTF-8 uses ALPHA-SOURCE-INVALID-UTF8 first. Bootstrap escape spans and Alpha payload failures retain half-open byte ranges. Alpha parser/module/linter results preserve these origins into StdDiagnostic; rich batch wire serialization and direct-lowering qualification remain open.

8. Formatting and migration rules#

Preserve original spelling. Registered ascii-bytes-to-byte-literal rewrites only printable ASCII decimal byte arrays, escaping quote/backslash. Arrays containing comments are preserved. debug/bytes-migration-check.py --alpha COMPILER --evidence DIR verifies exact rewrites, dry-run safety, core identity, sensitivity to changed bytes, and idempotence through the actual CLI. Full L14 qualification remains pending.

9. Positive, negative, execution, and edge-case tests#

tests/language/text/alpha-tests checks UTF-8 laws, rejected byte sources, Alpha parser preservation/editions, payload decoding, encoder boundaries and AST elaboration. The language-server replay suites (lsp-replays-utf16-*, lsp-replays-utf8-*; tools/registry/build_lsp_replays.py) check protocol encoding/ranges after a multibyte literal. NativeLiterals.alpha and NativeRuntime.alpha provide emitted-executable probes; 62 exact-output cases passed on the native evaluator lane. Direct file effects and dependent-family projection remain open.

10. Compatibility with the old edition, serialized forms, and cache keys#

Draft alpha-2027 only; alpha-2026 rejects the new token syntax with ALPHA-EDITION-UNSUPPORTED-FORM. Existing decimal byte arrays remain accepted. No new core constructor or serialized core tag. Edition is already a cache-key component. Full old-edition/cache identity and native diagnostic-origin parity still require the L14 qualification gate; focused checks do not prove that gate.