# Refracta: real-time refractive vision simulator (Astigmastism and Myopia sim)

Try here: https://sane24.github.io/Refractive-Vision-Simulator/

**What the world looks like through *your* prescription**
Refracta renders any scene, live webcam, uploaded photo, or one of ten bundled
environments, the way it lands on the retina of an eye with myopia, hyperopia,
astigmatism, or presbyopia. The blur is not a Photoshop gaussian: it is the
geometric point-spread function of a sphero-cylindrical eye, oriented by the
prescription's AXIS, scaled by pupil diameter, and driven per-pixel by scene
depth and the eye's accommodation state. A Python/OpenCV reference
implementation is the ground truth; the GLSL shader that runs at 60 fps is
validated against it to **0.006/255 RMS** on this page: open the demo with
`?validate=1` and watch it re-prove itself in your browser.

![wipe comparison: corrected vs uncorrected night driving with astigmatism](assets/hero.gif)

*Split-wipe on the night-drive scene, then an AXIS sweep: `−1.00 −2.75 × θ`,
6 mm pupil. Every light source smears along the axis+90 meridian, and rotating
AXIS visibly rotates the smear, which is the visual signature no isotropic
blur can fake.*


- **10 switchable environments** with exact, procedurally generated depth
  maps: night drive 🌃, traffic lights 🚦, Snellen chart 👁️, restaurant menu 🍝,
  highway billboard 🛣️, classroom 🧑‍🏫, phone at 30 cm 📱, starry night ✨,
  grocery shelf 🛒, sunny park 🌳 (+ an engineering test card 🎯).
- **Live webcam** (graceful fallback if denied) and **photo upload**, with
  optional depth: upload a depth map, run `tools/depth.py` offline, or click
  "Estimate depth with AI" to run Depth-Anything-V2 in the browser.
- **Preset prescriptions** (mild/high myopia, hyperopia, with-the-rule and
  oblique astigmatism, presbyopia, dilated night driver) plus full sliders:
  SPH, CYL, AXIS, ADD, pupil, focus target, FOV, chromatic aberration.
- **Split-wipe comparison** with a draggable divider, live clinical notation
  (`−2.00 −1.50 × 090`), an estimated uncorrected acuity readout, and a live
  drawing of the current PSF ellipse.

## Optics physics

**1. Defocus.** A refractive error of `ΔD` diopters through a pupil of
diameter `p` meters spreads a point over a blur disk of angular diameter

```
β ≈ p · ΔD                      [radians]
blur_px = β / rad_per_pixel,    rad_per_pixel = 2·tan(FOV/2) / width_px
```

Intuition: the pupil is the aperture of an out-of-focus camera; more diopters
or a bigger pupil both widen the cone of confusion. This is why the same
−2.00 D feels far worse at night: the dark-adapted pupil doubles `p`.

**2. Astigmatism: the oriented PSF.** A sphero-cylindrical eye has two focal
powers on perpendicular meridians: `SPH` along AXIS and `SPH + CYL` along
AXIS+90. A point source therefore images as an **ellipse** (a cross-section of
Sturm's conoid): the pupil disk scaled per-meridian by each meridian's
residual defocus,

```
r_axis  ∝ |V + SPH − A|         (semi-axis along AXIS)
r_cross ∝ |V + SPH + CYL − A|   (semi-axis along AXIS+90)
```

so a streak, not a halo. Sanity check that the code enforces: a pure cylinder
with a horizontal axis has all its error in the vertical meridian, so points
smear **vertically**.

**3. Depth and accommodation.** Objects live at vergence `V = 1/distance`.
The simulated eye accommodates `A` diopters (clamped to the reserve implied by
the ADD: `A_max = 2.5 − ADD`) to put the circle of least confusion on the
retina for your chosen focus target. Per pixel, the depth texture supplies
`V`, and `ΔD = V + SPH − A` per meridian: the focal plane snaps sharp while
everything nearer and farther blurs by its true dioptric distance. Depth maps
store **normalized disparity** (linear in `1/d`), so defocus is linear in the
stored value and 8 bits are plenty.

**4. Emissive HDR expansion.** 8-bit sources clip a headlight at 1.0, and
blurring clipped pixels yields dim grey smudges. Near-white pixels are
expanded back toward plausible radiance before convolution (inverse
tonemapping), then the result is re-clamped: streaks stay vivid, exactly like
real night glare. With the boost at 0 the pipeline is exactly
linearize → convolve → encode, which is what validation uses.

**5. Chromatic aberration (optional).** The eye carries ~1.2 D of
longitudinal chromatic aberration across the visible band; blue focuses in
front of the retina. Modeled as per-channel defocus offsets
(R +0.35 / G 0 / B −0.85 D), it fringes night lights in a very recognizable
way.

## Architecture

```mermaid
flowchart LR
    subgraph "Python ground truth"
        P[reference/psf.py<br>PSF + vergence math<br>12 unit tests] --> S[reference/simulate.py<br>OpenCV convolution]
        P --> A[reference/validate_acuity.py<br>tumbling-E ideal observer]
    end
    subgraph "content pipeline"
        G[tools/make_scenes.py<br>10 environments +<br>exact disparity maps] --> W
        D[tools/depth.py<br>Depth-Anything-V2 / MiDaS<br>for user photos] --> W
    end
    subgraph "web app (static)"
        W[js/optics.js<br>same formulas → uniforms] --> F[shaders/vision.frag<br>oriented variable-radius<br>gather at 60 fps]
        F --> UI[split wipe · presets ·<br>webcam · PSF inset]
    end
    S -->|reference PNGs| V[?validate=1<br>in-browser diff:<br>RMS / PSNR table]
    F --> V
```

The same formulas exist exactly three times, on purpose: NumPy (tested,
readable), JS (bakes uniforms), GLSL (per-pixel). The validation page is the
proof they never drift.

Inside `shaders/vision.frag` the transport is split, and the comments walk
through it line by line: **pass A** gathers over the *center pixel's* ellipse
(a golden-angle spiral through the ellipse matrix, mip-biased so large kernels
stay noise-free), which is the classic depth-of-field gather and correct for
surfaces. **Pass B** handles emissive energy (anything HDR-expanded above 1.0)
with scatter-as-gather: each tap asks "does *your* PSF, computed from *your*
depth, reach me?", Monte-Carlo normalized, so a headlight's streak correctly
paints over sharp neighbors, something a center-kernel gather cannot do. An
**exact mode** (dense `texelFetch` loop, border-replicate) reproduces the
Python convolution for validation and stills.

## Validation

**Shader vs reference.** Open [`?validate=1`](./?validate=1): each case
renders the shader in exact mode on the test card and diffs it against the
committed OpenCV output. Current results (Apple M-series, Chrome):

| case | prescription | RMS /255 | PSNR |
|---|---|---|---|
| identity | plano (must be lossless) | **0.000** | 168.1 dB |
| myopia | −4.00 sph, pupil 4 mm | **0.003** | 97.3 dB |
| astig-30 | −1.00 −2.50 × 030, pupil 5 mm | **0.002** | 101.6 dB |
| astig-117 | plano −3.00 × 117, pupil 6 mm | **0.001** | 111.6 dB |
| presby-near | +1.50 ADD 2.50 at 50 cm | **0.006** | 92.4 dB |

**Diopters → acuity.** `reference/validate_acuity.py` builds tumbling-E
optotypes at exact Sloan proportions, blurs them with the reference PSF
(4 mm pupil), and asks an ideal observer to tell the four orientations apart
from the blurred templates. The discrimination threshold is calibrated once to
the classic clinical anchor *2.00 D ≈ 20/200*; every other row is then a
prediction:

| defocus | blur disk | simulated acuity | clinical rule of thumb |
|---|---|---|---|
| 0.00 D | 0.0′ | 20/15 | 20/20 (display-limited) |
| 0.25 D | 3.4′ | 20/20 | 20/20 – 20/25 |
| 0.50 D | 6.9′ | 20/40 | 20/40 |
| 0.75 D | 10.3′ | 20/70 | 20/50 – 20/70 |
| 1.00 D | 13.8′ | 20/100 | 20/70 – 20/150 |
| 1.50 D | 20.6′ | 20/200 | ~20/200 |
| 2.00 D | 27.5′ | 20/200 | 20/200 *(anchor)* |
| 3.00 D | 41.2′ | 20/300 | 20/300+ |

The resolvable denominator grows linearly with defocus (R² = 0.95), which is
the physical claim under test: MAR ∝ blur-disk diameter ∝ `p · ΔD`.

**Depth estimation.** Bundled scenes carry *exact* procedural depth. For user
photos, `tools/depth.py` runs Depth-Anything-V2 (or MiDaS via torch.hub); on
the synthetic park scene the estimate correlates with ground truth at
Pearson r = 0.61, a fair illustration of why relative monocular depth needs
the app's near/far calibration sliders:

![ground-truth vs estimated depth](assets/depth_compare.png)

## Gallery (raw shader output)

| | |
|---|---|
| ![vertical streaks](assets/shots/night_axis000.jpg) *cyl axis 0: vertical streaks* | ![horizontal streaks](assets/shots/night_axis090.jpg) *same cyl at axis 90* |
| ![starfield with astigmatism + LCA](assets/shots/stars_astig.jpg) *stars render the PSF itself (LCA on)* | ![presbyopia at the menu](assets/shots/menu_presbyopia.jpg) *presbyopia: menu at 42 cm unreadable, candles bokeh* |
| ![focus on whiteboard](assets/shots/class_focus_board.jpg) *accommodating at the board: notes blur* | ![focus on notes](assets/shots/class_focus_notes.jpg) *accommodating at the notes: board blurs* |

## Repo layout

```
index.html  js/  css/  shaders/     the app (static, no build step)
shaders/vision.frag                 the heavily-commented core
reference/  psf.py  simulate.py     Python ground truth + unit tests
            validate_acuity.py      diopter→acuity experiment
tools/      make_scenes.py          procedural environments + exact depth
            depth.py  serve.py  make_validation_refs.py  make_gif.py
assets/scenes/                      10 environments (+ test card) with depth
validation/ cases.json  refs/       inputs for the in-browser validation
```


## Current limitations

- **Geometric PSF only.** The kernel is the uniform pupil-projection ellipse;
  diffraction, Stiles-Crawford apodization, higher-order aberrations (coma,
  spherical) and intraocular scatter are not modeled, so tiny blurs (< ~0.25 D)
  are slightly too clean and real night halos have more texture.
- **Relative depth for photos.** Monocular networks return ordering, not
  meters; the near/far sliders are an explicit, honest calibration step.
  Bundled scenes sidestep this with exact synthetic depth.
- **Monocular, static accommodation.** One eye, no binocular summation or
  rivalry; accommodation is a least-confusion policy with an age-derived
  amplitude (`2.5 − ADD`), not a dynamic control loop, and night myopia /
  accommodative lead-lag are ignored.
- **Scatter approximations in fast mode.** Pass A uses the center pixel's
  kernel (mild halos at hard depth edges); pass B is Monte-Carlo with a
  clamped estimator (very sharp emitters route through A). Exact mode exists
  precisely so these shortcuts are measurable: they never touch validation.
- **Display ≠ retina.** The screen has its own MTF, and 20/15 is the finest
  row the bundled chart's pixel grid can encode at its 15° FOV.

## Stack

| piece | role |
|---|---|
| Python + NumPy + OpenCV | reference optics, unit tests, scene/asset pipeline |
| Depth-Anything-V2 / MiDaS | monocular depth for user photos (offline + in-browser) |
| Three.js (vendored) + WebGL2 | texture plumbing, render loop, webcam |
| GLSL ES 3.0 | `vision.frag`: the oriented variable-radius PSF, exact + fast paths |
| Vanilla JS/HTML/CSS | app shell; ships as a static site |

*Refracta is an educational visualization, not a medical device; it will not
refract your eyes.*