React records GelSight-Mini images and OptiTrack poses — but, like all UMI-style data, no applied force: demonstrated pose equals achieved pose, so replaying the actions presses with zero intended force. Two methods put it back: tactile-estimated normal force, and DexForce-style force-informed action targets.
The plan matched FARM (arXiv:2510.13324): run FEATS (U-net, GelSight-Mini images → FEA-supervised force distributions, MAE < 1 N) offline over the recordings. It does not transfer: FEATS is trained on marker-dot gel, and the React sensors use markerless gel — different texture, different illumination profile.
Left: FEATS training sensor (63 marker dots). Right: React sensor, markerless. On this input the network returns its no-contact output for every frame — the strongest-contact frame of pushT/episode_000 predicts −0.040 N total, identical to a free frame (−0.042 N). Dot-based per-sensor calibration is impossible for the same reason.
The gel pad is modelled as a bed of independent springs (thin-elastic-layer approximation); the constant c is fitted on FEATS ground truth rather than assumed (see the external-validation section). Per-sensor calibration uses the 15 lowest-intensity frames of each episode: median depth becomes the zero map and the MAD of the plane-removed residuals sets a 5σ contact threshold — a fixed threshold cannot work because reconstruction noise differs per sensor (measured ~10 µm after plane removal). Each frame additionally gets a robust background-plane fit, which removes the frame-to-frame illumination tilt that otherwise dominates integrated depth.
Strongest contact of motherboard/episode_000 (left sensor): raw frame | difference vs reference | reconstructed depth. Indentation localises exactly where the image difference shows contact.
E1.1 specificity — no-contact rows must read ≈0 N; SNR = median force on high-intensity rows / p95 on reference rows. E1.2 correlation — Spearman ρ against contact intensity, an independent statistic of the same images. E1.3 spikes — single-frame excursions are solver noise; a median-3 over fresh frames must remove them. E1.4 calibration stability across episodes.
| episode | side | fresh rows | max F [N] | SNR | ρ vs intensity | spike raw→filt [%] | threshold [µm] |
|---|---|---|---|---|---|---|---|
| 2026-05-10/episode_000 | left | 2010 | 7.63 | 2732732.8 | 0.91 | 1.4 → 0.0 | 10 |
| 2026-05-10/episode_000 | right | 2007 | 6.39 | 52601.4 | 0.83 | 1.1 → 0.0 | 10 |
| 2026-05-10/episode_001 | left | 2157 | 9.46 | 625.6 | 0.83 | 1.4 → 0.0 | 10 |
| 2026-05-10/episode_001 | right | 2155 | 8.19 | 1295176.4 | 0.75 | 0.9 → 0.0 | 10 |
| 2026-05-10/episode_002 | left | 3037 | 9.92 | 750.6 | 0.77 | 1.0 → 0.0 | 10 |
| 2026-05-10/episode_002 | right | 3025 | 14.55 | 7284.3 | 0.79 | 1.4 → 0.0 | 10 |
| 2026-05-10/episode_003 | left | 2629 | 5.22 | 868.3 | 0.81 | 0.1 → 0.0 | 10 |
| 2026-05-10/episode_003 | right | 2629 | 3.20 | 2.0 | 0.82 | 0.5 → 0.0 | 10 |
| 2026-05-10/episode_004 | left | 690 | 3.68 | 0.6 | 0.68 | 0.3 → 0.0 | 10 |
| 2026-05-10/episode_004 | right | 692 | 4.22 | 2.1 | 0.89 | 0.3 → 0.0 | 10 |
| 2026-05-10/episode_005 | left | 220 | 2.85 | 1.4 | 0.80 | 0.5 → 0.0 | 10 |
| 2026-05-10/episode_005 | right | 221 | 3.47 | 17.3 | 0.75 | 0.0 → 0.0 | 10 |
| 2026-05-10/episode_006 | left | 1153 | 2.59 | 63549.3 | 0.87 | 0.1 → 0.0 | 10 |
| 2026-05-10/episode_006 | right | 1152 | 4.37 | 1608350.8 | 0.94 | 0.0 → 0.0 | 10 |
| 2026-05-10/episode_007 | left | 556 | 2.25 | 0.8 | 0.68 | 0.2 → 0.0 | 10 |
| 2026-05-10/episode_007 | right | 556 | 3.05 | 1334479.9 | 0.90 | 0.0 → 0.0 | 10 |
| 2026-05-10/episode_008 | left | 376 | 2.94 | 235.5 | 0.92 | 0.0 → 0.0 | 10 |
| 2026-05-10/episode_008 | right | 377 | 2.31 | 3.0 | 0.83 | 0.3 → 0.0 | 10 |
| 2026-05-10/episode_009 | left | 523 | 4.30 | 108.2 | 0.88 | 0.8 → 0.0 | 10 |
| 2026-05-10/episode_009 | right | 526 | 6.29 | 3279941.6 | 0.92 | 1.1 → 0.0 | 10 |
| 2026-05-10/episode_010 | left | 300 | 4.13 | 5.2 | 0.90 | 0.7 → 0.0 | 10 |
| 2026-05-10/episode_010 | right | 299 | 2.55 | 3.1 | 0.73 | 2.0 → 0.0 | 10 |
| 2026-05-10/episode_011 | left | 635 | 9.08 | 67.1 | 0.88 | 0.0 → 0.0 | 10 |
| 2026-05-10/episode_011 | right | 636 | 2.21 | 4213.5 | 0.87 | 0.2 → 0.0 | 10 |
| 2026-05-11/episode_003 | left | 2796 | 9.89 | 39.6 | 0.81 | 1.8 → 0.0 | 10 |
| 2026-05-11/episode_003 | right | 2804 | 14.90 | 2818249.9 | 0.79 | 2.0 → 0.0 | 10 |
| 2026-05-11/episode_004 | left | 735 | 11.18 | 766.6 | 0.76 | 1.5 → 0.0 | 10 |
| 2026-05-11/episode_004 | right | 736 | 10.16 | 2624701.5 | 0.78 | 1.1 → 0.0 | 10 |
| 2026-05-11/episode_005 | left | 4038 | 10.27 | 3623773.1 | 0.87 | 1.1 → 0.0 | 10 |
| 2026-05-11/episode_005 | right | 4036 | 10.44 | 2970277.3 | 0.83 | 1.1 → 0.0 | 10 |
| 2026-05-11/episode_006 | left | 3361 | 11.17 | 17.8 | 0.86 | 0.7 → 0.0 | 10 |
| 2026-05-11/episode_006 | right | 3364 | 12.86 | 2791.2 | 0.82 | 1.0 → 0.0 | 10 |
| 2026-05-11/episode_007 | left | 1923 | 9.29 | 8665.4 | 0.82 | 0.4 → 0.0 | 10 |
| 2026-05-11/episode_007 | right | 1923 | 8.26 | 12200.4 | 0.84 | 1.4 → 0.1 | 10 |
| 2026-05-11/episode_008 | left | 3827 | 13.67 | 36170.0 | 0.79 | 1.1 → 0.1 | 10 |
| 2026-05-11/episode_008 | right | 3824 | 13.06 | 3973363.4 | 0.85 | 1.4 → 0.0 | 10 |
| 2026-05-11/episode_009 | left | 1213 | 8.48 | 1427.0 | 0.84 | 0.8 → 0.0 | 10 |
| 2026-05-11/episode_009 | right | 1212 | 12.98 | 81585.4 | 0.91 | 1.6 → 0.0 | 10 |
| 2026-05-11/episode_010 | left | 1903 | 4.66 | 1923.1 | 0.76 | 0.8 → 0.0 | 10 |
| 2026-05-11/episode_010 | right | 1896 | 17.10 | 4043420.8 | 0.87 | 1.8 → 0.1 | 10 |
| 2026-05-11/episode_011 | left | 2133 | 8.51 | 17.8 | 0.83 | 0.8 → 0.0 | 10 |
| 2026-05-11/episode_011 | right | 2127 | 10.01 | 3374461.2 | 0.86 | 1.9 → 0.0 | 10 |
| 2026-05-11/episode_012 | left | 4131 | 11.21 | 3452.4 | 0.88 | 1.6 → 0.0 | 10 |
| 2026-05-11/episode_012 | right | 4130 | 11.36 | 3865536.7 | 0.87 | 1.6 → 0.0 | 10 |
| 2026-05-11/episode_013 | left | 3499 | 1.84 | 2.1 | 0.71 | 0.2 → 0.0 | 10 |
| 2026-05-11/episode_013 | right | 3508 | 2.65 | 691976.0 | 0.85 | 0.8 → 0.0 | 10 |
| 2026-05-11/episode_014 | left | 675 | 1.57 | 394078.5 | 0.87 | 0.3 → 0.0 | 10 |
| 2026-05-11/episode_014 | right | 679 | 2.53 | 224.0 | 0.86 | 0.6 → 0.0 | 10 |
| 2026-05-11/episode_015 | left | 1650 | 1.80 | 468833.1 | 0.85 | 0.3 → 0.0 | 10 |
| 2026-05-11/episode_015 | right | 1647 | 5.03 | 1496452.7 | 0.88 | 3.0 → 0.0 | 10 |
| 2026-05-11/episode_016 | left | 3269 | 2.22 | 70.2 | 0.83 | 0.9 → 0.0 | 10 |
| 2026-05-11/episode_016 | right | 3252 | 4.41 | 1317693.8 | 0.84 | 1.9 → 0.0 | 10 |
| 2026-05-11/episode_017 | left | 3980 | 21.55 | 2568.4 | 0.81 | 1.5 → 0.1 | 10 |
| 2026-05-11/episode_017 | right | 3954 | 22.76 | 88.9 | 0.88 | 1.4 → 0.0 | 12 |
| 2026-05-19/episode_000 | left | 518 | 7.57 | 9.9 | 0.94 | 3.1 → 0.2 | 10 |
| 2026-05-19/episode_000 | right | 506 | 11.46 | 6680776.6 | 0.92 | 4.0 → 0.2 | 10 |
| 2026-05-19/episode_001 | left | 251 | 7.86 | 1.8 | 0.91 | 4.0 → 0.0 | 26 |
| 2026-05-19/episode_001 | right | 246 | 10.06 | 1.5 | 0.92 | 2.0 → 0.0 | 10 |
| 2026-05-19/episode_002 | left | 590 | 7.09 | 1385.5 | 0.94 | 5.3 → 0.0 | 10 |
| 2026-05-19/episode_002 | right | 594 | 9.80 | 20.6 | 0.91 | 4.6 → 0.2 | 10 |
| 2026-05-19/episode_003 | left | 29 | 4.56 | 1.1 | 0.18 | 0.0 → 0.0 | 27 |
| 2026-05-19/episode_003 | right | 31 | 4.92 | 4724913.1 | 0.92 | 10.3 → 3.4 | 70 |
| 2026-05-19/episode_004 | left | 35 | 4.00 | 1.2 | 0.79 | 3.0 → 0.0 | 26 |
| 2026-05-19/episode_004 | right | 41 | 5.62 | 1.7 | 0.94 | 2.6 → 0.0 | 10 |
| 2026-06-18/episode_000 | left | 689 | 1.84 | 152355.9 | 0.75 | 0.4 → 0.1 | 10 |
| 2026-06-18/episode_000 | right | 693 | 3.72 | 772015.0 | 0.86 | 1.4 → 0.0 | 10 |
| 2026-06-18/episode_001 | left | 4099 | 7.10 | 304962.0 | 0.80 | 0.5 → 0.0 | 10 |
| 2026-06-18/episode_001 | right | 4153 | 5.81 | 556445.1 | 0.78 | 0.7 → 0.0 | 10 |
| 2026-06-18/episode_002 | left | 3956 | 4.20 | 315086.3 | 0.79 | 0.5 → 0.0 | 10 |
| 2026-06-18/episode_002 | right | 3994 | 5.58 | 731457.7 | 0.83 | 1.0 → 0.0 | 10 |
| 2026-06-18/episode_003 | left | 4021 | 3.79 | 191250.3 | 0.76 | 0.2 → 0.0 | 10 |
| 2026-06-18/episode_003 | right | 4065 | 5.12 | 583764.9 | 0.83 | 0.7 → 0.0 | 10 |
12 s around each episode's force peak: live force bar under the tactile stream (median-3 filtered). Left: pushT (light, intermittent contact). Right: motherboard (sustained presses).
React has no force ground truth, but FEATS (arXiv:2411.03315) does: a GelSight Mini pressed by a CNC machine against 24 indenters, with FEA-derived force labels. Running our estimator on its val split is a transfer test — marker-dot gel, different pad, single global no-contact reference — i.e. strictly harder than the React setting, where reference frames come from the same episode seconds away.
| metric | value | scope |
|---|---|---|
| Spearman ρ (pooled) | 0.74 | normal loading, ≤30 N, n=150 |
| Pearson r (pooled) | 0.65 | |
| MAE after scale calibration | 4.2 N | |
| ρ within one indenter family | 0.66–0.94 | zelda 0.94, cylinder 0.75, triangle 0.70 |
| ρ on unseen indenter shapes | 0.85 | test_unknown_indenters split, n=100 — shape generalization holds |
| ρ vs press depth on FoTa (T3) | 0.43 / 0.24 | 61 captures, third-party Panda rig, household objects; markerless / marker gel medians, 84% positive. FoTa has no force GT, so pose press-depth is the (attenuated) monotone proxy |
| ρ vs F/T ground truth — FoTa cnc_Mini | 0.43 / 0.65 | 400 frames, CNC gantry + F/T sensor, markerless Mini (pooled / edge-filtered, after flat-field optimization). FeelAnyForce on identical frames: 0.83 / 0.92; FEATS U-net: 0.07 |
| ρ vs F/T ground truth — GlowTact | 0.63 | 600 of 14,715 cleaned markerless-Mini presses, 0–20 N; per-family up to 0.93. FeelAnyForce: 0.90. Third independent scale estimate (2.61 N/mm³) inside the 2–4× band |
| scale stability across splits | 2–4× | fitted N/mm³ varies 1.9–7.2 between sessions/sensors: within-episode relative force is reliable, cross-sensor absolute scale is not |
| ρ on shear-loaded captures | −0.15 | excluded & documented: shear churns the image without adding indentation volume |
| fitted scale | 2.01 N/mm³ | 43× the assumed Winkler constant — the fit absorbs the depth-unit and gel-constant assumptions; React forces are reported in FEATS-calibrated newtons |
What the validation bought. Chasing the initial ρ = 0.42 exposed two pipeline defects that also affected React inference: a global illumination tilt dominating the integrated depth (fixed by a robust background-plane fit per frame — React contact thresholds tightened from 50 µm to 10 µm) and marker-dot imprints from the SDK's mask-and-interpolate path (fixed by inpainting the dots before the network). A quadratic background model was also tried and rejected — it absorbed real contact (ρ 0.61→0.50). Sanity check on scale: the strongest motherboard press reads ~0.2 N under the theoretical constant but ~7 N calibrated — the latter is what pressing a connector home actually takes.
Rather than adding force as a new action dimension, the estimated normal force becomes a virtual position target past the contact surface (DexForce, arXiv:2501.10356):
ptarget = pobserved + (F̂n / k) · n̂, k = 1500 N/m (deployment impedance)
The action stays a pose — it composes with the existing 30 Hz pose actions,
needs only an impedance controller at deployment, and in free space reduces
exactly to the observed pose. The pressing direction n̂ is not a guessed
axis: it is the rig's dual-ball calibrated gel_axis_in_rigid
(pose-to-pose consistency ≈1°), rotated into world frame per row. The naive
[0,0,1] guess produced negative approach alignment at force onsets; the
calibrated axis makes it positive.
E2.1 free-space invariance — zero force must leave the action exactly the observed pose. E2.2 boundedness — F/k must stay millimetre-scale to be a safe action. E2.3 roundtrip — k·‖target − pose‖ must reproduce the input force (pure algebra). E2.4 geometry — motion should align more with n̂ while pressing than in free motion.
| episode | side | free-space max offset [m] | penetration p50 / max [mm] | roundtrip err [N] | v·n̂ pressing / free |
|---|---|---|---|---|---|
| 2026-05-10/episode_000 | left | 0.0e+00 | 1.01 / 5.09 | 6.3e-14 | +0.127 / -0.004 |
| 2026-05-10/episode_000 | right | 0.0e+00 | 0.93 / 4.26 | 5.5e-14 | +0.038 / +0.025 |
| 2026-05-10/episode_001 | left | 0.0e+00 | 1.24 / 6.31 | 6.9e-14 | +0.147 / +0.079 |
| 2026-05-10/episode_001 | right | 0.0e+00 | 1.10 / 5.46 | 5.9e-14 | +0.220 / +0.082 |
| 2026-05-10/episode_002 | left | 0.0e+00 | 0.91 / 6.62 | 4.6e-14 | +0.162 / +0.076 |
| 2026-05-10/episode_002 | right | 0.0e+00 | 1.11 / 9.70 | 6.0e-14 | +0.074 / +0.082 |
| 2026-05-10/episode_003 | left | 0.0e+00 | 0.57 / 3.48 | 4.0e-14 | -0.025 / +0.073 |
| 2026-05-10/episode_003 | right | 0.0e+00 | 0.38 / 2.13 | 6.3e-14 | +0.028 / -0.001 |
| 2026-05-10/episode_004 | left | 0.0e+00 | 0.66 / 2.45 | 2.1e-14 | -0.017 / +0.040 |
| 2026-05-10/episode_004 | right | 0.0e+00 | 0.63 / 2.82 | 4.3e-14 | -0.028 / -0.178 |
| 2026-05-10/episode_005 | left | 0.0e+00 | 0.86 / 1.90 | 2.1e-14 | -0.063 / -0.015 |
| 2026-05-10/episode_005 | right | 0.0e+00 | 0.67 / 2.31 | 3.3e-14 | -0.033 / +0.007 |
| 2026-05-10/episode_006 | left | 0.0e+00 | 0.68 / 1.73 | 2.9e-14 | -0.024 / +0.093 |
| 2026-05-10/episode_006 | right | 0.0e+00 | 0.52 / 2.91 | 3.3e-14 | +0.034 / -0.009 |
| 2026-05-10/episode_007 | left | 0.0e+00 | 0.53 / 1.50 | 3.4e-14 | -0.011 / +0.189 |
| 2026-05-10/episode_007 | right | 0.0e+00 | 0.66 / 2.03 | 2.9e-14 | -0.006 / -0.052 |
| 2026-05-10/episode_008 | left | 0.0e+00 | 0.33 / 1.96 | 2.9e-14 | -0.026 / +0.202 |
| 2026-05-10/episode_008 | right | 0.0e+00 | 0.51 / 1.54 | 2.7e-14 | +0.047 / -0.053 |
| 2026-05-10/episode_009 | left | 0.0e+00 | 0.78 / 2.87 | 4.2e-14 | -0.066 / +0.106 |
| 2026-05-10/episode_009 | right | 0.0e+00 | 1.33 / 4.19 | 4.7e-14 | +0.012 / +0.065 |
| 2026-05-10/episode_010 | left | 0.0e+00 | 0.65 / 2.75 | 2.7e-14 | -0.031 / +0.117 |
| 2026-05-10/episode_010 | right | 0.0e+00 | 0.74 / 1.70 | 2.5e-14 | +0.059 / +0.083 |
| 2026-05-10/episode_011 | left | 0.0e+00 | 0.45 / 6.05 | 3.0e-14 | -0.014 / +0.077 |
| 2026-05-10/episode_011 | right | 0.0e+00 | 0.54 / 1.47 | 4.6e-14 | +0.002 / +0.027 |
| 2026-05-11/episode_003 | left | 0.0e+00 | 1.31 / 6.59 | 4.3e-14 | +0.115 / +0.103 |
| 2026-05-11/episode_003 | right | 0.0e+00 | 1.67 / 9.93 | 6.7e-14 | +0.080 / +0.073 |
| 2026-05-11/episode_004 | left | 0.0e+00 | 2.30 / 7.45 | 5.0e-14 | +0.146 / +0.127 |
| 2026-05-11/episode_004 | right | 0.0e+00 | 1.82 / 6.78 | 5.6e-14 | +0.076 / +0.106 |
| 2026-05-11/episode_005 | left | 0.0e+00 | 1.64 / 6.85 | 6.0e-14 | +0.075 / +0.021 |
| 2026-05-11/episode_005 | right | 0.0e+00 | 1.55 / 6.96 | 8.4e-14 | +0.036 / -0.011 |
| 2026-05-11/episode_006 | left | 0.0e+00 | 1.89 / 7.45 | 4.5e-14 | +0.084 / +0.109 |
| 2026-05-11/episode_006 | right | 0.0e+00 | 2.03 / 8.58 | 8.5e-14 | +0.066 / +0.118 |
| 2026-05-11/episode_007 | left | 0.0e+00 | 1.55 / 6.19 | 3.1e-14 | +0.008 / +0.120 |
| 2026-05-11/episode_007 | right | 0.0e+00 | 1.08 / 5.50 | 7.2e-14 | +0.040 / +0.096 |
| 2026-05-11/episode_008 | left | 0.0e+00 | 1.96 / 9.11 | 3.5e-14 | +0.051 / +0.123 |
| 2026-05-11/episode_008 | right | 0.0e+00 | 1.99 / 8.71 | 6.8e-14 | +0.051 / +0.084 |
| 2026-05-11/episode_009 | left | 0.0e+00 | 0.77 / 5.65 | 2.9e-14 | +0.049 / +0.065 |
| 2026-05-11/episode_009 | right | 0.0e+00 | 1.76 / 8.65 | 3.6e-14 | -0.015 / +0.072 |
| 2026-05-11/episode_010 | left | 0.0e+00 | 0.72 / 3.11 | 3.9e-14 | +0.141 / +0.068 |
| 2026-05-11/episode_010 | right | 0.0e+00 | 1.88 / 11.40 | 4.8e-14 | +0.074 / +0.010 |
| 2026-05-11/episode_011 | left | 0.0e+00 | 1.05 / 5.67 | 3.8e-14 | +0.144 / +0.095 |
| 2026-05-11/episode_011 | right | 0.0e+00 | 1.81 / 6.67 | 4.2e-14 | +0.061 / +0.008 |
| 2026-05-11/episode_012 | left | 0.0e+00 | 1.04 / 7.48 | 3.6e-14 | +0.089 / +0.090 |
| 2026-05-11/episode_012 | right | 0.0e+00 | 1.72 / 7.57 | 5.9e-14 | +0.013 / +0.037 |
| 2026-05-11/episode_013 | left | 0.0e+00 | 0.25 / 1.23 | 4.2e-14 | +0.005 / +0.057 |
| 2026-05-11/episode_013 | right | 0.0e+00 | 0.39 / 1.77 | 7.3e-14 | +0.061 / +0.079 |
| 2026-05-11/episode_014 | left | 0.0e+00 | 0.22 / 1.05 | 3.1e-14 | -0.011 / +0.043 |
| 2026-05-11/episode_014 | right | 0.0e+00 | 0.29 / 1.69 | 8.1e-14 | +0.069 / +0.069 |
| 2026-05-11/episode_015 | left | 0.0e+00 | 0.27 / 1.20 | 3.5e-14 | -0.014 / +0.039 |
| 2026-05-11/episode_015 | right | 0.0e+00 | 0.70 / 3.35 | 3.7e-14 | +0.065 / +0.112 |
| 2026-05-11/episode_016 | left | 0.0e+00 | 0.30 / 1.48 | 3.9e-14 | +0.017 / +0.073 |
| 2026-05-11/episode_016 | right | 0.0e+00 | 0.61 / 2.94 | 4.8e-14 | +0.010 / +0.125 |
| 2026-05-11/episode_017 | left | 0.0e+00 | 3.95 / 14.37 | 5.0e-14 | +0.044 / +0.154 |
| 2026-05-11/episode_017 | right | 0.0e+00 | 4.07 / 15.17 | 7.8e-14 | -0.006 / +0.123 |
| 2026-05-19/episode_000 | left | 0.0e+00 | 1.56 / 5.04 | 3.0e-14 | +0.019 / -0.004 |
| 2026-05-19/episode_000 | right | 0.0e+00 | 3.27 / 7.64 | 3.4e-14 | +0.036 / -0.018 |
| 2026-05-19/episode_001 | left | 0.0e+00 | 1.70 / 5.24 | 3.2e-14 | -0.009 / +0.132 |
| 2026-05-19/episode_001 | right | 0.0e+00 | 2.97 / 6.71 | 2.9e-14 | +0.050 / +0.007 |
| 2026-05-19/episode_002 | left | 0.0e+00 | 1.67 / 4.73 | 3.1e-14 | +0.048 / -0.077 |
| 2026-05-19/episode_002 | right | 0.0e+00 | 1.77 / 6.54 | 4.4e-14 | +0.054 / +0.008 |
| 2026-05-19/episode_003 | left | 0.0e+00 | 1.48 / 3.04 | 1.6e-14 | -0.087 / +0.151 |
| 2026-05-19/episode_003 | right | 0.0e+00 | 2.15 / 3.28 | 2.4e-14 | +0.023 / +0.106 |
| 2026-05-19/episode_004 | left | 0.0e+00 | 1.88 / 2.67 | 1.9e-14 | +0.004 / +0.397 |
| 2026-05-19/episode_004 | right | 0.0e+00 | 2.78 / 3.74 | 2.6e-14 | -0.213 / +0.278 |
| 2026-06-18/episode_000 | left | 0.0e+00 | 0.31 / 1.22 | 4.3e-14 | -0.048 / +0.067 |
| 2026-06-18/episode_000 | right | 0.0e+00 | 0.38 / 2.48 | 6.5e-14 | +0.079 / -0.001 |
| 2026-06-18/episode_001 | left | 0.0e+00 | 0.30 / 4.74 | 4.7e-14 | +0.050 / +0.087 |
| 2026-06-18/episode_001 | right | 0.0e+00 | 0.43 / 3.87 | 8.4e-14 | +0.067 / +0.051 |
| 2026-06-18/episode_002 | left | 0.0e+00 | 0.30 / 2.80 | 4.8e-14 | +0.078 / +0.065 |
| 2026-06-18/episode_002 | right | 0.0e+00 | 0.46 / 3.72 | 9.0e-14 | +0.114 / +0.017 |
| 2026-06-18/episode_003 | left | 0.0e+00 | 0.28 / 2.53 | 5.3e-14 | +0.106 / +0.101 |
| 2026-06-18/episode_003 | right | 0.0e+00 | 0.40 / 3.42 | 7.2e-14 | +0.081 / +0.040 |
Across all episodes: free-space invariance 0.0e+00 m, roundtrip error 9.0e-14 N (machine precision), max penetration 15.2 mm.
| found | fix |
|---|---|
| FEATS returns its no-contact output on every markerless frame | switched force estimation to photometric-stereo depth (negative result kept above) |
| fixed dot-threshold 55 detects 0 dots (our dots bottom out at gray 56) | percentile-based threshold — then made moot by the markerless finding |
| 1.1 mm phantom depth at the image border (Poisson/Neumann edge artifact) | exclude a 12/16-px margin from force integration |
| contact threshold exploded 30× when a reference frame was lightly touching | median zero-map + MAD threshold instead of mean + std |
| 4% single-frame force spikes (bad Poisson solves) | median-3 over fresh frames only — row-wise filtering would see each duplicated value 3× and keep it |
| assumed gel normal [0,0,1] gave negative approach alignment at onsets | use the rig's dual-ball calibrated gel_axis_in_rigid; sign verified
against approach kinematics |
| fed the full camera view to a depth MLP trained on the SDK's 15%-cropped view | same 1/7 border crop for React frames and FEATS images — geometry now identical between inference and validation |
| FEATS transfer started at ρ = 0.42: illumination tilt dwarfed real indentation, dot imprints leaked into depth | per-frame robust background-plane removal + marker inpainting → ρ = 0.70; quadratic background tried and rejected (absorbed contact) |
| shear-loaded captures anti-correlate (ρ = −0.15) | scoped the estimator to normal loading; documented as a limitation (React pushing involves shear — treat high-shear force estimates with caution) |
| theoretical Winkler scale off by ~40× (depth-unit + gel-constant assumptions) | absolute scale fitted on FEATS ground truth; React forces in FEATS-calibrated newtons, uncertainty stated |
Data:
yxma/React · code ships with the dataset (preprocess/,
toolbox/) · references: FARM (2510.13324), DexForce (2501.10356),
FEATS (2411.03315), ACP (2410.09309), gsrobotics SDK.