XRAI
Voice your world into being. See through. See over. See across.
Open spatial graph format — authored by any LLM, rendered by any engine, queryable by any agent. Shipping today in Portals on iPhone: 60 FPS, 360+ VFX, persistent stateful worlds (CVPR 2026). Free forever. No CLA. No gatekeeper. No take-backs.
Join jARvis. Jam with friends. Harness holographic intelligence.
Inspire infinite imagination. See more. Be more.
Open the doors of perception. The singularity is near.
A far-out future is here — open, not captured.
The problem XRAI answers
Social media brainwashes and isolates. GUIs cripple bodies and minds. Profit and algorithms drive conflict and instant gratification. People feel small and powerless. Civilization is at risk.
XRAI is the substrate for a different medium — one where intent becomes space, memory persists, minds bridge, and the agent's view of the world is legible to you, not hidden behind a prompt.
Infinite zoom
A graph that scales in four directions. Every XRAI scene can be navigated along each axis.
Proof it ships
Portals — the reference implementation — shipped on iOS and at CVPR 2026. The numbers below are from the peer-reviewed camera-ready paper.
60 FPS on iPhone 14 Pro — 2.7–4.1× speedup over prior art. LOD-adaptive Gaussian splatting (SPAG) + shared spatial-media compute substrate fusing depth, stencil, audio, and ML-pose channels. Drives 360+ source-agnostic VFX.
Stateful worlds, not stateless scenes. Persistent geospatial scene-state with layered world metadata, reloadable payloads, anchor-guided re-alignment across sessions, devices, users.
Voice-driven semantic authoring. On-device intent parsing + cloud fallback for ambiguous utterances. No-code composition pipeline bridging reconstruction and generation.
Cross-platform today. iOS + web viewers (Apple Vision Pro compatible) for reconstructed environments, volumetric humans, holographic spatial media.
Paper: Portals: Persistent, Editable 4D Spatial World Models on Edge Devices — Tunick, Brant, Pennock, Kasowski — H3M Inc. + IMC Lab — CVPR 2026 Workshop on 4D World Models, submitted 2026-04-10. Prior clinical deployment of volumetric AR at Memorial Sloan Kettering established the real-time rendering primitives.
What XRAI is
A typed hypergraph of entities, relationships, events, and intentions — anchored in space and time, renderable in any medium, authored by any LLM, queryable by any agent.
One format. Scene graphs for AR / VR / XR today. Episodic memory for voice assistants tomorrow. Living knowledge graphs the year after. Language of thought eventually.
What XRAI is not
- Not a company's proprietary format
- Not a vendor-controlled standard
- Not a committee-designed artifact
- Not paywalled, tracked, or rent-extractive
- Not a replacement for HTML / glTF / USD — XRAI composes with them all
What you can say — 20 use cases
Every example maps one voice command → XRAI payload → existing Portals spec → rendered on any runtime. Not reinvented — curated from shipping work. See full details: USE_CASES.md.
- 🎮 Mini game — "make a game where cubes fall and I catch them" → 0 new code, already works via existing bridge
- 🌍 Dynamic world — "create a forest world around me" → sparse generator rules, expanded at runtime (spec 023)
- 🎨 Audio-reactive artwork — "paint a glowing spiral that reacts to my voice" (spec 009 Paint-AR + 5 audio shaders)
- 🌐 Spatial lens on a webpage — "show me this Wikipedia article as a memory palace" (spec 006 KB visualizer + hypergraph arch)
- 🧠 Agentic HUD — "show me what's interesting in this room" (spec 021 PTHI — fighter pilot HUD meets Jarvis)
- 🎭 Neural camera lens — "make everything look like Ghibli" (spec 024 neural filters via Unity Sentis)
- 🧩 Shared thinking space — "invite Alice into this scene" (spec 010 multiplayer + federated graph)
- 🔧 CLI / codebase as a 3D city — spec 006 + 017 MegaBrain orchestration
- 🎬 Memex / time travel — "show me what I was working on last Tuesday" (constitution XRAI Philosophy)
- 🏗 Architecture walkthrough — same XRAI file plays in Portals AR / Three.js web / Unreal / visionOS tabletop
- ✈️ Flugtag — "launch me off a pier in a flying bathtub with wings" — hand-tilt steers contraption, leaderboard by distance (specs 002 + 019 + 023)
- 🏁 Racing — "build a rainbow-road track around my living room" — track wraps real furniture, share XRAI = friends race same track (specs 023 + 019 + 011)
- 🪂 Sky diving — "drop me from 10,000 feet over my couch" — body pose controls drag, gesture-deploy parachute, land on couch for score (specs 021 + 009 + 003)
- 🔑 Where did I leave my keys? — passive camera logs object sightings → jARvis answers "last seen 2h ago on kitchen counter." The mundane memex. (specs 021 + 011 + 004 + constitution)
- 🌱 Sample anything as a seed — point camera at a tree / chair / room / song → jARvis extracts features → generator emits 20 variants. "XRAI = DNA, not organism" — kills 99% of asset marketplaces. (specs 022 + 023 + 024)
- 🌐 Make my website 3D — browser extension +
<meta name="xrai">→ any site becomes a walkable spatial version on any XRAI runtime. Adoption propagates like RSS. (spec 006 + constitution "We Are Blind") - 🦍 King Kong in your backyard — ARWorldMap anchor + body tracking + federated view; ships today at room/neighborhood scale. Landmark-scale (Empire State) ships post-MVP via Cesium for Unity (Apache 2.0, already researched + verified compatible — `bleeding-edge-jt` kb-followup 2026-02-21). (specs 003 + 010 + 011 + Cesium)
- 🕺 10 mini best-friends doing jumping jacks on my desk — live avatar stream + 10 clones + random phase + ObjectAnimator. The sticker economy of spatial. (specs 002 + 003 + 010)
- 🎨 Your space as an art gallery — anchor paintings to living room / office / local park via ARWorldMap; friends with same scan see it. City-square variant (Times Square) deferred until open geospatial VPS. (specs 009 + 022 + public namespace)
- 🪧 Community protest sign garden — aggregate signs by topic at local civic spaces (city hall, quad). Platform-unownable civic speech, room/neighborhood scale today. Landmark scale (White House) deferred. (specs 009 + 022 + crowd-aggregator)
First jARvis demos
Short videos that sell the thesis — spatial AI that gives users superpowers. See the full demo plan in demos/jarvis_demos.md.
Say a sentence, see a world. Spatial meditation garden materializes from voice. XRAI graph visible. Saved scene → portable to any device via QR code.
"Show me the last 5 conversations I had about Portals." Spatial timeline of past sessions. Tap one. Replay the workspace as it was.
Look at a plant. Ask "what's wrong?" Glasses overlay diagnosis grounded in XRAI knowledge graph.
Design principles
- Radical simplicity — grokkable in 1 hour
- LLM-authorable — any model can emit valid XRAI from natural input
- Permissionless adoption — no gatekeeper, no tracking
- Forgiving parsing — partial inputs still work (Postel's law)
- Extensible without breaking — v1 stays valid as v2 ships
- Spatial + temporal + modal — scenes, episodes, alternatives
- Typed n-ary relations — hyperedges, not just pairs (v1.1+)
- Personal + federated — your graph is yours; publish slices
- MCP-native — any agent reads/writes via standard tools
- Self-referential — the schema describes itself
The lineage
Every generation tried. Each missed one piece that now exists.
- Bush 1945 — memex (associative personal trails). Tech wasn't ready.
- Nelson 1965 — Xanadu (bidirectional + versioning). Over-designed, never shipped.
- Berners-Lee 1989 — WWW (one-way simple). Won on simplicity.
- Page/Brin 1998 — PageRank (importance from topology).
- Berners-Lee 2007 — Giant Global Graph. Stalled on authoring burden.
- Hofstadter (ongoing) — analogy as cognition.
LLMs solve the authoring burden. Cameras ground the symbols. MCP federates the agents. The window just opened.
Get started
Start here — the vision: VISION.md 🌟 — problem + Sight Triad + jARvis goal + CVPR proof. Load-bearing source for everything else.
Read the manifesto: MANIFESTO.md
Read the spec: SPEC.md v1.0 draft
Demos: demos/jarvis_demos.md
Spatial MCP server: mcp-server/ — 11 tools (9 xrai.* + 2 paint.*). Any MCP-compatible agent (Claude Desktop, Cursor, Cline, OpenAI, Gemini-with-shim) can author XRAI scenes + procedural paint strokes. Replaces Open Brush's port-40074 HTTP API with typed, stateless, multi-agent MCP.
Engine runtimes: runtimes/ — Three.js · PlayCanvas · Unity · Unreal · WebXR · visionOS RealityKit. One XRAI doc, every engine.
Repo: github.com/imclab/xra1 (launching 2026-04-29)
Contact: hello@xrai.dev
Governance (Year 1)
BDFL: @jamestunick (IMC Lab + H3M Inc.). Public RFC process. Weekly updates. No corporate sponsor controlling direction.
Year 2+: transfer to Apache Software Foundation or W3C Community Group once 1000+ external adopters validated. Never to a single vendor's foundation.
Commercial relationship
The spec is free. Forever. Period. Portals (H3M Inc.) builds commercial products on top: hosted XRAI cloud, best-in-class renderer, priority-quality LLM authoring, enterprise spatial intelligence API, vertical apps. The format stays free; services built on top are paid.
Git → GitHub pattern. HTML → Chrome. Markdown → Notion. The community can never be held hostage.