Building CyberHUD, Month One: From a Clock in My Glasses to an iPhone Desktop

At the beginning of July, I had a small and slightly unreasonable complaint: I had AR Glasses that could show my iPhone over USB-C, but I could not find the iOS experience I wanted to run on them.

I was not looking for a full spatial-computing platform. I wanted something closer to the fictional heads-up displays I had enjoyed in games: a clock, a little system status, perhaps a few useful pieces of information at the edges, and enough black space that the display still felt like a HUD rather than a television strapped to my face. Cyberpunk 2077 was one early art-direction reference, although inspiration never meant copying its assets, layouts, logos, or product identity.

It sounded like a fun personal project, so I decided to make it.

The first version was an ordinary iOS project. By the middle of August, CyberHUD had become an iPhone-hosted Desktop shell with native and bundled applets, optional side-by-side 3D presentation, games, a bounded coding agent named Netics, deterministic browser previews, and a development system built around specifications, skills, scripts, tests, and concurrent agent worktrees.

That is a much larger result than the original clock. It is still a pre-release result. This is the story of how the scope expanded, why several exciting directions were deliberately narrowed, and how I kept one human decision-maker in control while multiple AI agents worked concurrently.

The cover is an AI-generated concept visual. It is not a product screenshot, through-glasses photograph, physical-display proof, or compatibility claim.

The constraint that defined the product

The useful starting point was to be honest about the hardware. The glasses I was targeting could accept video from an iPhone as an external display. CyberHUD did not need to pretend that this made them a full AR headset.

The product has three named surfaces. The iPhone is the Controller. It owns touch input, settings, permissions, text entry, and escape paths. The glasses presentation is the Monitor. Between them is a fixed logical Desktop, rendered at 1920 x 1080 and fitted into the connected display without stretching the layout.

The Desktop can keep up to four glanceable Docked widgets in its corners and open one supported applet as the Active widget in the centre. Dark pixels remain negative space. I avoided a full-screen wallpaper, permanent grid, or arbitrary overlapping windows simply because those elements looked futuristic. The composition needed to remain readable on transparent display glasses and understandable from the phone.

This boundary also simplified the core interaction model. The main controls do not rely on glasses cameras, motion sensors, head tracking, eye tracking, spatial anchors, or a high refresh rate. The phone supplies a large trackpad, familiar gestures, the system keyboard, settings, and a reliable way to leave or restore the HUD.

Physical testing repeatedly changed the design. Sleep and Wake are a good example. Several reasonable software approaches failed to produce the behavior I wanted on the actual display route. The working direction was to let iOS resume ordinary phone mirroring while CyberHUD slept, then restore the Desktop on Wake. That result became a pattern for the project: a plausible implementation remained a hypothesis until the real route proved it.

One useful widget led to another

Clock and Status were the obvious beginning because they worked as glanceable information. They also exposed an early distinction. Public battery, charging, and network information belonged to the host. Detailed cursor, gesture, and runtime diagnostics were useful during development, but they did not belong in the public product.

Map changed the shape of the project more substantially. It introduced location, heading, camera state, and more complex gestures. I kept it native because those capabilities depend on iOS permissions and host-owned services. The applet is called Map in the interface; the underlying provider remains an implementation and attribution concern rather than its product identity.

Weather took a less direct route. I explored a native implementation early, removed it when the product was not ready for it, and continued with a bundled experiment. Near release preparation, the balance changed. Weather moved back into the native host so permission handling, attribution, and location minimization could be treated as first-class product responsibilities.

Text input exposed another boundary. A custom keypad was controllable and unpleasant to use. CyberHUD moved to the native iOS keyboard while keeping the active field visible on the Monitor. That sounds like ordinary application plumbing, which is precisely why it mattered: the project was becoming something I could use rather than a collection of visual prototypes.

The catalogue continued to grow. Clock, Map, Music, Agenda, RSS, Camera, Architect, Weather, Checklist, Lost Signal, Brick, Wireframe, Galaxy, Profile, Vector Roll, and Basketball each tested a different combination of input, storage, permissions, rendering, and presentation. The point was not to collect features. Each applet had to fit the same focused Desktop model.

Applets turned the HUD into a platform

The bundled applets began as small web experiences inside fixed widgets. As their behavior became richer, the host needed a clear lifecycle and a clear authority model. An applet had to understand when it was ready, active, compact, expanded, sleeping, returning, or leaving. Optional side-by-side presentation also had to remain one logical experience rather than two independent copies drifting apart.

This led to a bounded applet platform shared by utilities, visual experiments, and games. The native host owns geometry, input, storage boundaries, permissions, and presentation. Bundled applets receive a deliberately limited interface to those capabilities. The browser layer is useful because it makes visual and interactive ideas easier to develop, but it does not become an unrestricted doorway into iOS.

One applet, two eye views

The interaction design starts with one logical applet session. Switching between Docked and Active presentation changes its context rather than launching a second app. The iPhone turns gestures into semantic input, the host decides which actions belong to the Desktop, and only supported applet input crosses the boundary.

Mono presentation uses one renderer. SBS uses two renderers, one for each physical eye, attached to the same logical session. One renderer is the authority: it receives interaction, advances simulations, and requests bounded effects. The other is a replica that renders the latest accepted state. The user's dominant-eye choice can move authority, but it never swaps the physical left and right images.

Only one presentation path is active at a time. The diagram keeps the essential transitions: Controller interaction enters one logical session, that session drives the active presentation mode, and host composition produces the Monitor output. In SBS, the authority publishes accepted state for the replica instead of creating a second independent session. For applets using the synchronized runtime, this prevents the two eye views from becoming two copies of the game, two authority-owned capability calls, or two competing text fields. It also lets a renderer be replaced or an authority handoff occur without changing the identity of the applet itself.

Two coordinated layers of depth

CyberHUD combines depth at two levels. The host places the whole widget surface, its chrome, and the cursor on one semantic HUD plane. When the widget moves deeper or closer, those elements move together, while its logical frame and hit testing stay unchanged.

A 3D applet then adds depth inside that surface. It begins from one canonical Mono camera pose and uses a shared stereo profile to produce parallel, off-axis eye views around a chosen convergence plane. Geometry on that focal plane adds no extra applet-local disparity, so it visually meets the host widget plane. Geometry in front of or behind it gains relative parallax.

The same user depth calibration influences both layers without making them the same transform. Galaxy can converge around the current celestial focus, Wireframe around the viewed model, and Vector Roll around the course ahead. Each of those applets still chooses a meaningful scene scale, while the common eye signs, camera model, and depth response keep the overall parallax experience consistent.

This remains software-generated binocular depth on an external display. The calibration is not a measurement of physical eye distance or real-world metres, and it still requires human comfort review on the glasses.

Games were especially useful architecture probes. Lost Signal tested deterministic state in a compact space. Galaxy and Wireframe tested 3D presentation. Vector Roll tested motion input and stereo consistency. Basketball and Brick added richer interaction and exposed pressure points that simpler widgets never reached.

Those pressure points were valuable. When a game uncovered a synchronization or input problem, I tried to separate the immediate applet fix from the platform-level lesson. Some issues belonged to one game; repeated issues belonged lower in the shared runtime. That distinction kept the project moving without pretending every local problem justified an immediate rewrite.

I also explored whether arbitrary remote web services could behave like trusted applets. The answer was narrower than the prototype. A mutable remote website should not receive the same capabilities as reviewed bundled code, and two unrelated pages should not be described as synchronized stereo views. The experiment stayed isolated and internal rather than becoming a general remote-app store.

That rejection matters. CyberHUD's platform boundary is stronger because it does not claim to host everything.

"Fun, not danger"

The idea that became Netics began with four fixed template slots. My note to the agent was simple: I wanted the feature to be "fun" but not "danger."

Those slots became Custom Applet 1 through 4. They can hold bounded HTML, CSS, and JavaScript source stored on the iPhone. The advanced editor supports conversation, preview, validation, revision, and explicit application. Architect provides a smaller intent-first surface: describe the capability, choose a target slot, and explicitly choose Generate.

Netics is the native service behind those flows. It can use an available on-device model, a supported cloud provider, or a compatible service configured by the user. The resulting Custom Applet source is stored and run locally. If a remote provider is selected, the app must disclose what relevant request and context leave the device. Credentials remain outside the generated applet and its ordinary data.

Generated source is still treated as untrusted input. It has to fit the Custom Applet boundary and pass CyberHUD's validation before it can be used. The four active slots remain a deliberate exception, not a downloadable app store or a promise that generated code can access arbitrary phone capabilities.

How Netics builds an applet

The advanced editor, Architect on the HUD, and confirmed Siri/Shortcuts requests do not own three separate generation systems. They share one per-slot request draft and delegate to the same Netics service. Text entered from the Controller editor or Architect therefore represents the same current request for that Custom Applet slot, while Siri can populate the same path after confirmation. The shared runtime also owns admission, cancellation, and host-observed progress, so a generation can continue when the initiating view disappears and another product surface can report its current phase without exposing model reasoning.

Netics does not send one giant undifferentiated prompt. Trusted system instructions are rendered from a versioned coding contract that describes the Custom Applet environment, product constraints, available capabilities, and required output shape. The user's request, current source, local summary, and recent conversation are packaged separately as bounded, untrusted project context.

When a model route supports tools, Netics can progressively load focused development skills for ordinary widgets, text input, games, or 3D work. Its tools are deliberately read-only and scoped to the admitted request: they can inspect bounded parts of the current source, read declared guidance, find suitable system icons, and perform static checks. They cannot open a shell, browse the device, read credentials, inspect another applet, or grant themselves a new capability.

Apple Intelligence is one possible route. On supported devices and locales, Netics can use Apple's on-device Foundation Models without a provider key or remote tool loop. The complete admitted source stays in the local request, and the result still passes the same CyberHUD validator. A selected remote model can instead receive the disclosed bounded context and use the progressive skills and tools. Choosing Generate is the explicit boundary for either route.

The diagram focuses on those public-facing paths: Apple Intelligence on device and a configured remote provider. Separately gated Apple cloud experimentation is outside this story and is not a release or availability claim.

Each slot keeps a bounded local conversation rather than sending its complete history forever. A generation receives a compact local summary and the most recent turns. When I apply a result, CyberHUD stores the displayed source and its matching bounded conversation together as a recoverable revision checkpoint. It behaves like a small source-and-context commit for Undo and Redo, but it is not a Git commit, an immutable audit ledger, or unlimited chat storage.

My Netics tests explored a range of small tools rather than variations of one demo. A few of the more interesting results were:

• a configurable currency converter that kept five useful rates visible as a Compact widget, then added amount entry, currency-pair selection, and pair swapping when Expanded;
• a two-sided chess game that moved pieces in turn and rejected illegal moves while keeping both stereo views synchronized;
• a BPM coach with selectable tempos, audio or visual ticks, and a Compact presentation that kept the beat running while showing only the pulse and current tempo; and
• a small 3D strategy prototype where I could pan across a simple map, select units, mine resources, and construct a limited set of buildings and soldiers.

These were calibration and development examples, not bundled applets or launch promises. They were useful because each one stressed a different part of the contract: readable Compact layout, text and choices, rule-driven state, continued behavior across presentation changes, stereo consistency, and bounded 3D interaction.

Netics also changed how I responded to generated-code failures. Patching one generated result might fix a demo while hiding a more useful platform problem. When the same class of error appeared repeatedly, I asked whether the applet interface, higher-level SDK, system guidance, or development skills made the correct solution unnecessarily difficult.

The goal became to make the safe path the simple path. Better shared helpers reduced how much fragile lifecycle, text, state, and stereo code a generated applet needed to invent. Evaluation still mattered, but a better score could not overrule product constraints, leak benchmark language into the shipped system, or replace human review.

Coordinating concurrent agents

By the second half of July, I was normally running two or three coding agents concurrently. That created an immediate coordination problem. One shared workspace was not enough, and a private task list was not a durable handoff system.

I gave each task an isolated Git worktree and a written brief. One workspace remained the integration point, while each coding agent worked inside its assigned task boundary. Parallel work used separate areas where possible, and shared project state or device testing stayed serialized.

This process was learned through mistakes. Agents occasionally crossed task boundaries, touched shared code at the wrong time, or saw a general refactor opportunity inside a narrowly scoped problem. The resulting rules became stricter: preserve dirty work, ask before changing another worktree, continue without repeatedly asking inside the assigned one, and keep authorization for edits, commits, hardware, paid calls, deployment, and publication as separate human decisions.

The task record became the durable whiteboard. A handoff records what changed, what evidence exists, what failed, what still needs human verification, and the next bounded action. The next agent is assumed to be capable and to know nothing about the previous chat. That assumption produces better handoffs than relying on a long transcript remaining available and correctly interpreted.

I remain the human integrator. I own the product direction, integration decisions, physical tests, provider spending, rights review, and publication. Agents can investigate, implement, test, review, and simplify concurrently, but I do not treat them as named employees or turn output volume into a claim that a particular model replaced a team.

What the workflow does show is breadth. A conventional software organization might distribute this work among native iOS, web runtime, 3D and game, AI-agent, test and tooling, release operations, technical writing, product governance, and asset-provenance disciplines. CyberHUD keeps those responsibilities coordinated under one human integrator.

Documents became part of the system

The project became difficult for agents because its operational context was growing faster than any single source directory. An applet change could affect native behavior, a browser runtime, data shape, storage, tooling, tests, assets, release gates, and public wording. Asking every agent to read everything was slow and eventually counterproductive.

The documentation model started with a simple separation. Implemented behavior belonged in design or specification documents. Useful ideas that did not exist yet belonged in a roadmap. Durable constraints belonged in principles. Procedures belonged in runbooks. Active work belonged in task records.

This became CyberHUD's version of Document Driven Development. It does not declare that every Markdown file outranks code. Different artifacts own different kinds of facts:

• human authorization controls side effects;
• machine-readable contracts own exact shapes;
• principles own durable constraints;
• focused specifications own implemented behavior;
• skills route agents through the right context and workflow;
• scripts create or validate reproducible artifacts;
• tests establish a specific class of evidence; and
• public claims remain downstream of product behavior and review.

I think of it as a chain of custody for a decision. A requirement begins in a bounded work item, moves into the document that owns the behavior, becomes an implementation, receives automated and manual evidence, and finally reaches public copy only if that evidence supports the wording.

The development loop can be summarized without exposing the implementation details:

Netics provides one example: its documented boundary keeps the product and its development tools aligned. The applet release process makes browser previews reproducible and reviewable. The design-material process keeps provenance, rights state, and human approval attached to visual and audio work. The product-site review connects public wording to the evidence and qualifications behind it.

Automation can establish reproducibility without granting approval. A package can be technically valid and still lack publication permission. An image can be processed deterministically and still fail rights review. A website can build correctly and still lack enough evidence for a product claim. Keeping those states separate prevents a passing script from silently promoting its own output.

CyberHUD also moved away from permanent broad architecture, decision, and task snapshots that repeated current behavior. Focused specifications own the current system, the roadmap owns future outcomes, and task records own active work. Temporary plans should disappear after their durable facts reach the correct owner.

The agent context eventually needed a compaction pass of its own. The root instructions had grown into a second product manual, and individual skills had accumulated copies of facts that changed elsewhere. I simplified both layers so the root became a router and skills loaded focused documents only when a task needed them.

This approach still depends on judgment. Structural checks can validate links, metadata, and file shapes; they cannot prove that prose matches product behavior. Some rationale remains historical rather than permanent. Document Driven Development reduces drift only when the human review keeps closing the loop.

How large did it become?

The scale investigation began with frustration. At one point the agents started to feel slower, even on work that should have been straightforward. I was annoyed enough to start cursing at them, which was neither a useful debugging method nor a good way to prepare for a future in which Terminators might have access to my old chat logs.

So I stopped treating "the agents are slow" as a diagnosis. I wanted to know whether model behavior had changed, whether my workflow was creating friction, or whether the project had simply grown beyond the context and coordination model that worked at the beginning.

I measured the project on 15 August because I wanted a more honest answer than counting every dependency and generated file. The raw tree contained 2,199 tracked files and 251,114 physical text lines, but that included vendor libraries, generated indexes and catalogues, imported general-purpose skills, lockfiles, minified dependencies, and nonshipping design material.

After excluding those categories, along with media, fonts, and large icon collections, the adjusted first-party snapshot contained 439 text files and 156,267 physical lines. Grouping the same snapshot into four broader buckets makes its shape easier to see:

Category	Includes	Files	Approx. physical lines
Product implementation	Native app, first-party applets, and product site	221	78,600
Tests and development tooling	Test suites, validators, and project tooling	119	49,400
Specifications and agent guidance	Product specifications and project-agent instructions	69	10,300
Resources and project support	Resources, localization, licences, build files, and project configuration	30	18,000
Measured total		439	156,267

The Swift and Python suites contained at least 733 statically named test functions. The main development history contained 119 commits and 113 merged changes across 31 active commit days in the research window. Those counts describe scope and cadence. They do not tell me how difficult each change was, how much review it required, or how close the product is to release.

A conventional-team estimate

I did not track actual hours, so there is no honest way to say that CyberHUD "contains" a precise amount of human labor. I can still build a planning estimate for what a conventional, experienced team might budget to reach a comparable pre-release snapshot without concurrent coding agents.

For that calculation, I used the approximately 128,000 physical lines of first-party implementation, tooling, product-site code, and tests. I excluded documentation, localization, licences, project configuration, vendor code, generated indexes, and media. If I assume a broad range of 1,500 to 3,000 net maintained physical lines per engineer-month, including implementation, review, refactoring, and tests, the engineering work lands around 43 to 85 person-months.

That line-based range still misses product integration, interaction and visual review, external-display testing, release preparation, privacy and rights work, documentation, and coordination across disciplines. Adding a 20 to 35 percent allowance produces a planning range of roughly 52 to 115 person-months. At 160 hours per person-month, that is approximately 8,000 to 18,500 person-hours.

Another way to picture the same range is approximately six experienced contributors for nine to nineteen months, or eight contributors for seven to fourteen months. The roles would likely span native iOS, web runtime, 3D and games, AI-agent integration, test and developer tooling, product and interaction design, release operations, and technical documentation or compliance. A smaller team could combine several roles but would usually need more calendar time.

This is a replacement-planning estimate, not a measurement of the hours I spent, the hours the agents ran, or labor that was literally "saved." It also stops at a pre-release engineering snapshot. Recruiting, onboarding, external review, App Store decisions, hardware compatibility work, and launch operations could move a conventional schedule substantially in either direction.

The useful conclusion is narrower: CyberHUD had become a small product platform spanning multiple engineering disciplines, not a typical single-screen indie app. Concurrent agents helped me work across those disciplines, while the document and worktree structure helped keep their output reviewable.

Two websites with different jobs

cyberhud.net is the factual product surface. It explains the iPhone Controller, Monitor composition, applets, Custom Applet boundaries, themes, external-display requirements, privacy routes, support limitations, and current availability. This article is not a download, TestFlight invitation, waitlist, or release-date announcement.

This blog is the founder journal. It can show the decision history and host separately reviewed deterministic browser previews. If you would like to try some of the games, selected applets have been ported to the browser preview page, where they run as static packages with explicit controls and disclosures.

Those previews are not the native iPhone app or a through-glasses demonstration. They cannot load executable code into CyberHUD, and the app does not discover or install packages from either website. Keeping that boundary explicit lets a browser demo be inspectable without turning the product site or journal into a remote-code source.

The same caution applies to captures. Simulator output can prove that a specific layout rendered and help diagnose composition. A physical-iPhone capture can prove what that phone rendered. A website composite can present a product state. None of those alone proves physical glasses comfort, broad compatibility, or what a user sees through a lens.

What month one produced

CyberHUD started because I wanted a clock and a cool status display in glasses that otherwise behaved like an external screen. The project expanded whenever a useful feature exposed a missing system boundary: Map clarified native ownership, Weather clarified provider responsibility, games stressed deterministic state and stereo, Netics forced a safer creation model, concurrent agents forced worktree discipline, and public preparation forced evidence and claims to become first-class artifacts.

The result is more ambitious than my original personal project. At the time of writing, it was also deliberately unfinished. Physical display validation, generated-code App Review, rights review, purchase validation, signed archives, final privacy and support publication, and release approval remained separate gates rather than claims I could collapse into a passing build.

That is the process I want to preserve. Agents can move quickly and work in parallel. Documents, contracts, tests, and scripts can make their work reproducible. The evidence still decides what I am willing to claim, and the final decision remains mine.