Adopt canonical call syntax and wire in orphaned tokens

- Function calls are f(a, b) via a postfix argument list; juxtaposition
  application removed; nullary calls f() supported
- Wire DEFINITION (x ≜ e), ALLOC/RELEASE postfix (r ⊕ / r ⊖), channel
  operations (⇀_ch e / ↽_ch e), and qualified module access (M‧f)
- Delete tokens with no rule and no example: QUERY (?), EXISTS, IMPORT,
  ARROW, DOT; delete the unary ? prefix operator
- Canonical handler clause: ↯pattern ⟹ expr (pattern is an identifier or
  a string); clauses separated by semicolons
- SEMICOLON has one role: sequence separator with optional trailing use
- Disambiguate braces structurally: record ({a: e}), set ({a, b}), block
- IDENTIFIER no longer allows a leading underscore, so subscripts such as
  ⌉_db_lock and ↽_socket lex as UNDERSCORE + IDENTIFIER
- Add '/' as ASCII alias of ÷; add unary minus; pathLiteral accepts
  🖫identifier as well as 🖫"…"
- Exactly one ASCII escape per glyph (drop \lam, \gets, \ne, \le, \ge,
  \vee, \wedge, \cup alias forms, \N, \Z, \Q, \R, \C, \B)
- Replace deprecated ANTLRInputStream with CharStreams in the test
  harness: it fed UTF-16 units and could never tokenize the
  supplementary-plane glyphs 𝔹, 𝓜 and 🖫
- Update LexerTest/ParserTest to canonical syntax; add coverage for
  subscript lexing, channels, resources, module access, unary minus,
  nullary calls, and negative tests for juxtaposition and the ternary
This commit is contained in:
developtheweb 2026-07-09 02:51:12 -04:00
parent 5230273ab1
commit f076a1dade
6 changed files with 253 additions and 111 deletions

View file

@ -11,3 +11,19 @@ against the existing examples.
- **The Java package for generated code comes from `-package` in build.gradle only**; rejected: `@header` package declaration in the grammar (combined with `-package` it generates a duplicate `package` statement that does not compile).
- **ANTLR warnings fail the build** (`-Werror` in build.gradle); rejected: warnings as advisory output (they hid the shadowed-token bugs).
- **The choice-type rule is `⟨ expr ⟩`** with the interior `|` consumed by `condExpr`; rejected: explicit `⟨ expr | expr ⟩` (the interior expr already consumes the bar, making the explicit BAR unreachable).
- **Function calls are `f(a, b)` — a postfix argument list, nullary `f()` included**; rejected: Haskell juxtaposition `f x` (programmer convention, fails the Fatima test — children learn `f(x)` in school).
- **SEMICOLON has one role: sequence separator (trailing `;` permitted)**; the program, blocks, `(...)`, `⌈...⌉`, `...`, `⌜...⌝`, `⌞...⌟` and `⟳(...)` all contain one `seqExpr`; rejected: a separate statement-terminator rule duplicating the same token (two roles for one symbol).
- **Braces disambiguate structurally**: `IDENTIFIER :` → record, two-plus comma-separated exprs → set, everything else (incl. `{}` and `{x}`) → block; a singleton set literal cannot be written (deferred to M1); rejected: parser-order coin flips left undocumented.
- **`≜` defines, `←` assigns; both wired into the chain** with `≜` binding looser than `←`, both right-associative; rejected: `≜` tokenized but unreachable.
- **`⊕`/`⊖` are postfix resource operators** (`database ⊕`, `conn ⊖`), matching every example; rejected: prefix form `⊕open(path)` that appeared only in the whitepaper.
- **Channel operations are subscripted prefix operators** `⇀_ch expr` / `↽_ch expr`; rejected: leaving SEND/RECEIVE orphaned.
- **`‧` is qualified module access** (`Mathematics‧sin(angle)`), a postfix `‧IDENTIFIER`; rejected: leaving MIDDOT orphaned, or `.` (removed — one access syntax).
- **Handler clauses are `↯pattern ⟹ expr`, semicolon-separated**, where pattern is an identifier (binds the exception) or a string (matches a message); rejected: `↯e ⇒ expr` (⇒ is EXPORT; the clause arrow should mirror the guarded-alternative arrow ⟹).
- **IDENTIFIER may not start with `_`**, so subscripts (`⌉_db_lock`, `↽_socket`) lex as UNDERSCORE + IDENTIFIER; rejected: identifiers with a leading underscore (made every subscript lex as one identifier token).
- **`/` is an ASCII alias of ÷ (DIV)** so `π/4` parses; rejected: ÷-only division (unreachable on most keyboards).
- **Unary minus exists** (`-x`), sharing the MINUS token at prefix level; rejected: binary-only minus (cannot write negative numbers); unary plus was NOT added (`x ++ y` stays invalid).
- **`pathLiteral` accepts `🖫"…"` and `🖫identifier`**, as required by `readFile(🖫path)` in example 03.
- **Deleted tokens: `?` (QUERY), `∃` (EXISTS), `⇐` (IMPORT), `→` (ARROW), `.` (DOT)** — defined but used by no parser rule and no example; dead operators are debt; each returns in M1 only with a documented semantic. Rejected: keeping them tokenized-but-unreachable.
- **Exactly one ASCII escape per glyph** (`\lambda` not `\lam`, `\leftarrow` not `\gets`, `\neq` not `\ne`, `\nat` not `\N`, …); rejected: alias sets (two ways to write the same token).
- **`%` (modulo), `∑`, `√`, `²`, `|x|`, ranges `[a..b]`, indexing/slicing, `where`, and record field access are NOT in M0** — every document says so instead of using them; deferred to M1 with semantics, not smuggled in via prose.
- **Lambda parameters are a bare comma-separated pattern list** (`λa, b: body`); rejected: parenthesized parameter lists `λ(a, b):` (two ways to write parameters).