attention and trust
as first-class data types
anrl adds three structural operators to any document. not metadata, not comments. ast fields that the compiler, the parser, and the llm all read natively.
!weight — saliency
every anrl statement is prefixed with an attention weight. the formatter quantizes continuous values to five discrete tiers, empirically sufficient for llms to distinguish priorities without float-noise confusion.
| weight | tier | meaning |
|---|---|---|
| !1.0 | critical | query target — mandatory attention |
| !0.8 | high | important supporting context |
| !0.5 | normal | standard content (default, elided) |
| !0.2 | weak | distractor / background context |
| !0.0 | suppress | ignored by the model |
# saliency in practice !1.0 @Incident_Node :: Alert # model MUST attend here !0.8 *severity: critical !0.5 *region: us-east-1 !0.2 *created_at: 2024-08-12T08:00:00Z # background context
?confidence — epistemic clarity
a suffix on primitives and edges encoding the certainty of a statement. omit it for verified facts (?1.0 is the default and is always suppressed to save tokens). the behavioral effect depends entirely on the system prompt.
three sources assert contradictory temperatures: 60°c (?0.3), 40°c (?0.9), 100°c (?0.1). with the conflict-resolution prompt, the model resolves to 40°c without explicit chain-of-thought.
+60% accuracy over plain json in conflict resolution tasks
a summarization task. without any prompt, the llm reads ?0.1 and naturally injects hedging language: “according to some sources, it is estimated that the temperature may be 60°c.” the confidence operator is read, not filtered.
+33% rouge-1 over confidence-stripped anrl in hedging tasks
the same ?confidenceoperator requires different system prompts for different tasks. mode a needs “prefer highest confidence.” mode b needs no prompt — adding one produces incorrect behavior. workaround: use ^query directives at document top. see the spec
--rel-→ — relational anchoring
typed, directed edges between named entities form an explicit graph embedded in text. llm attention heads traverse named entity references in edges naturally. this is what json $ref and string matching cannot do.
--contains-->--depends_on-->--imports-->--implies-->=>*owner: "John_Doe"multi-hop reasoning collapses — string matching is not a graph
@Service --owned_by--> @John_Doetraversable, typed, bidirectional in the compiled graph
benchmark results
16-file corpus spanning 462 bytes to 288 KB. token overhead is shape-dependent: tabular, wide-array, and null-heavy payloads achieve net compression. structured graphs and deep nesting incur overhead from entity cost (~3 tokens/entity).
| fixture | size | source tokens | anrl tokens | savings | compile ms |
|---|---|---|---|---|---|
| empty_values.json | 462 B | 148 | 69 | +53.4% | 0.03 |
| wide_array.json | 2.7 KB | 1,020 | 527 | +48.3% | 0.15 |
| large_flat.json | 4.6 KB | 1,952 | 1,540 | +21.1% | 0.08 |
| deep_nested.json | 2.4 KB | 644 | 818 | −27.0% | 0.21 |
| openapi.json | 4.9 KB | 1,274 | 1,809 | −42.0% | 0.32 |
| mixed.md | 2.9 KB | 725 | 823 | −13.5% | 0.05 |
| large_openapi.json | 288 KB | 53,247 | 67,453 | −26.7% | 13.22 |
| large_event_log.json | 127 KB | 46,462 | 57,433 | −23.6% | 4.38 |
the six-stage compiler pipeline
every file passes through the same six stages, regardless of input format. the output is always a Vec<AnrlNode> ir.
parses source format (json/yaml → json_yaml.rs, markdown → pulldown-cmark, code → tree-sitter cst, anrl text → recursive descent parser, binary anrl → rmp_serde decode)
runs fastembed AllMiniLM-L6-v2 (~22MB) if auto heuristics trigger: island detection (link density < 0.5) AND prose entropy (>25% unstructured text). capped at top-3 implies edges per source.
contrastive weight optimization (BFS from query anchors), weight quantization to 5 tiers, confidence calibration, flatness warning.
renders ast to text or messagepack binary. enforces syntactic elision (!0.5 and ?1.0 suppressed). deterministic ordering: entities → links → primitives.
recovers ast from text for round-trip. injects default values for elided fields (weight 0.5, confidence 1.0) so equality holds.
for directory input: resolves cross-file imports to typed edges (--imports--> / --references-->). unresolved external deps become ephemeral stub entities.