Samsung Galaxy XR platform limitations affecting clinical-grade neurologic exam capture
2026-05-26
Sevaro built SENSE / SDNE — a Unity-based Standardized Digital
Neurologic Exam — on Samsung Galaxy XR (Android XR). After ten rounds of
on-device validation between February and May 2026, ten distinct
platform-level issues have been identified that materially limit
clinical-grade exam capture. Five of these resolve via existing
partner-pathway agreements with Samsung Korea R&D and Google Android
XR engineering (raw inward-camera access under research NDA, post-hoc
Watch ↔︎ XR time synchronization, pass-through boundary exception,
world-facing camera FOV characterization, and eye / face video segment
export). The remaining five require new firmware, runtime-service, or
sensor-platform work (compositor frame-pacing stall, undocumented
eye-tracking precision, face-tracking service unreachable in Google’s
own first-party app, no real lower-body tracking, SLAM translation drift
in passthrough). This document consolidates the on-device evidence
behind each issue and maps each to a specific ask, drawn from
docs/SAMSUNG_XR_PLATFORM_ASKS.md and the Cheri Malo
correspondence thread from 2026-05-04 onward. Findings here are reported
evidence-first and without advocacy; clinical context is provided per
issue. The intended use of this document is to anchor the 2026-05-27
Samsung + Google working session in Plano, TX, and to serve as a
reference artifact for downstream partner conversations.
What this is. A consolidated record of the
platform-level issues Sevaro has encountered while developing the SDNE
neurologic-exam app on Samsung Galaxy XR. Each issue is grounded in
measured on-device evidence (session JSON, logcat, profiling) rather
than inference. Each is mapped to a specific ask — either a Samsung
Platform Ask (numbered 1 through 7 in
docs/SAMSUNG_XR_PLATFORM_ASKS.md) or one of the three
research-partner pathway asks from the 2026-05-04 Cheri Malo thread
(Cheri Asks #1 through #3, with #4 as a narrower fallback to #1).
Who it’s for. External technical partners.
Specifically: Samsung Korea R&D (the May 27 Plano workshop
audience), Google Android XR engineering (issue #21 / #22 / #23
maintainers on android/android-xr-unity-package),
Dr. Jennifer Reneker / UMMC Center of Excellence (research-pathway
collaborator), and future investors and clinical partners reviewing
platform readiness.
How to read it. Section 3 sets up the platform stack diagram. Section 4 is the core inventory, organized by neurologic domain (Eye, Face, Motor, Gait, Audio). Section 5 covers the three research-partner pathway proposals. Section 6 is the issue × ask cross-reference table. Section 7 is methodology. Appendices A, B, and D follow. (Appendix C, Glossary, is deferred to v1.1.)
Version 1.0, 2026-05-26. Status fields in Section 4 will be updated post-workshop.
┌─────────────────────────────────────────────────────────────────────────┐
│ APPLICATION LAYER Sevaro owns │
│ ───────────────────────────────────────────────────────────────────────│
│ SDNE Unity application (com.sevaro.sdne / com.sevaro.sense) │
│ - 25 exam tasks across 7 neurologic phases │
│ - Per-task scoring, threshold logic, session JSON emission │
│ - AWS Lambda upload pipeline, dashboard at sense.neuroplans.app │
└─────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ UNITY / OPENXR LAYER Unity / Google own │
│ ───────────────────────────────────────────────────────────────────────│
│ Unity 6 (6000.3.6f1) + Universal Render Pipeline │
│ com.unity.xr.openxr 1.16.1 / 1.17.0-pre.2 │
│ com.unity.xr.androidxr-openxr 1.3.0-pre.1 / 1.3.1 │
│ com.google.xr.extensions 1.3.1 (added 2026-05-25) │
│ - XR_ANDROID_eye_tracking, XR_ANDROID_face_tracking │
│ - XR_ANDROID_unbounded_reference_space (newly available) │
│ - XRFineEyeFeature, XRBodyTrackingFeature, XRSceneMeshingFeature │
└─────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ ANDROID XR SERVICES LAYER Google / Samsung own │
│ ───────────────────────────────────────────────────────────────────────│
│ com.google.xr.perception.inputserver (holds inward cameras) │
│ Face-tracking service (returns SERVICE_NOT_READY) │
│ Eye-tracking service │
│ Hand-tracking service │
│ SLAM / spatial-mapping service │
│ OpenXR compositor (~118 ms stalls measured) │
│ Consumer safety boundary policy (~2 m halt) │
└─────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ HARDWARE / FIRMWARE LAYER Samsung owns │
│ ───────────────────────────────────────────────────────────────────────│
│ Galaxy XR (SM_I610) - inward cameras, eye trackers, world cameras, │
│ IMU, SoC + thermal envelope, firmware │
└─────────────────────────────────────────────────────────────────────────┘
The application layer is fully under Sevaro’s control. Issues at this layer are not subjects of this report. The Unity / OpenXR layer is partially under Sevaro’s control (we choose package versions and select features) but the underlying extension definitions and runtime are Google’s. The Android XR services layer is Google’s and Samsung’s joint responsibility, and is where the majority of issues in this report surface. The hardware / firmware layer is Samsung’s. Each issue in Section 4 is labeled with the layer at which the block originates.
For each issue: a one-sentence headline with the headline measurement; the evidence trail (file path or session ID); the root cause as measured (not guessed); the ask that resolves it; current status; and the clinical impact in one or two lines.
SDNE_Core15_77e2af69-2524-4178-8626-9f93f58d2385_20260526_024516.json
(2026-05-25 healthy run). Task T04 IntrusiveSaccadeCount = 147 over 12
trials; task T05 SaccadicIntrusionCount = 81 over 30 s pursuit.
Per-subsystem profiling: eye / face / hand / UI / render / task
subsystems all measured at ≈0.1 ms/frame. The 118 ms stall sits inside
the OpenXR runtime frame-submission path, not in app code. The OpenXR
runtime reports Confidence: 0.95 and
IsLikelyHeadPoseFallback: false throughout the corrupted
stream — neither runtime signal exposes the failure.android/android-xr-unity-package) since 2026-04-16, no
response.XR_ANDROID_eye_tracking
publishes no sample-rate / latency / accuracy specs; FINE and COARSE
modes have no documented numerical differences.XR_ANDROID_eye_tracking; developer.android.com/develop/xr
documentation; verified absent from
com.unity.xr.androidxr-openxr 1.3.1 source.Covered in Section 5.1 (Cheri Ask #1) — research-partner pathway.
XR_ERROR_SERVICE_NOT_READY_ANDROIDvalid_frame_ratio = 0 across a full T07 task; same
error reproduces inside Google’s own first-party calibration app.docs/UNITY_BUG_face_tracking_service_not_ready.md.
Confirmed 2026-05-25 PM. Permissions verified granted via
adb shell cmd appops (FACE_TRACKING, EYE_TRACKING_COARSE,
EYE_TRACKING_FINE all allow). System face calibration
completed via both Settings > Avatars > Calibrate and
com.google.xr.facetracking.calibration/.FaceTrackingCalibrationActivity.
Headset worn (awake) throughout. Logcat shows
E .../FaceProvider.cpp:351 Face state data was unavailable. Could not retrieve region confidence data.
and
XR_ERROR_SERVICE_NOT_READY_ANDROID in xrGetFaceCalibrationStateANDROID
every session. Critical: running the first-party Google
calibration app itself, its own logcat (under its own PID)
spams the same XR_ERROR_SERVICE_NOT_READY_ANDROID hundreds
of times.androidxr-openxr 1.3.1 cleared the application-side error
path (more graceful TryGetIsFaceTrackingCalibrated) but
blendshape data remained all-zero. The failure reproduces in Google’s
own first-party app — conclusively a Galaxy XR runtime / service-level
bug, not specific to Sevaro’s app or to Unity.android/android-xr-unity-package) since at least
2026-04-09, no response.com.google.xr.perception.inputserver holds the inward
camera exclusively while the OpenXR session is active;
Camera.open() returns a handle but
MediaRecorder.start() throws
RuntimeException: start failed. Resulting MP4 is 0
bytes.docs/T07_VIDEO_FALLBACK_BLOCKED.md. Session JSON
2026-05-06:
T07_FacialActivation: QC: T07 video fallback could not start: java.lang.RuntimeException: start failed. Live blendshape pipeline still active.;
diag.video_fallback_attempted: true;
diag.video_fallback_failed: true; output file 0 bytes.
Camera2 has the same XR-perception-runtime exclusivity. MediaProjection
captures the rendered framebuffer (patient’s POV / passthrough world),
not the front camera.tools/post_hoc_face_mesh.py). Cannot consume
Galaxy-XR-captured frames today. Phone-camera capture works but adds a
second hardware piece to the protocol.XRBodyTrackingFeature), not measured
from leg-mounted sensors or cameras.SAMSUNG_XR_PLATFORM_ASKS.md Ask 5. Confirmed in the
com.google.xr.extensions package documentation (Body
Tracking marked “Experimental*“).camera_id=0 world-camera access with Camera2 + permission,
but it is labeled “preview”; downward FOV (lap / feet coverage when
seated) is not specified.XR_ANDROID_unbounded_reference_space extension
enabled.com.google.xr.extensions package
providing XRUnboundedRefSpaceFeature was added to SDNE
2026-05-25 and built into SENSE Fine v3; first worn-headset overground
test pending.android/android-xr-unity-package) open since at least
2026-04-16.docs/ONDEVICE_RUN_FINDINGS_2026-05-25.md §1 row P1.Note on Audio / Speech specifically. The microphone
capture path on Galaxy XR functions correctly. The 2026-05-25 T13
Semantic Fluency “0 words” failure observed in session 77e2af69 was
caused by a Sevaro-side audio-upload pipeline gap (the WAV captured
locally to
/storage/emulated/0/Android/data/com.sevaro.sense/files/sessions/audio/
but did not reach S3 for downstream Bedrock scoring). This was a
client-side bug, now resolved in SENSE Fine v2 via an orphan-WAV scan in
AudioUploader.RetryPendingCoroutine. No Samsung-side
platform ask under Audio at this time.
Three of the issues in Section 4 surface through consumer-policy boundaries that Samsung has confirmed will not move in the consumer build. For these, the proposed resolution is a research-pathway agreement: a partner NDA framework that grants Sevaro (and clinical collaborators including UMMC CoE) access to data streams that consumer apps cannot reach. These three asks were originated in the 2026-05-04 Cheri Malo correspondence thread and refined through the 2026-05-08 must-haves draft.
sense.neuroplans.app infrastructure).A narrower fallback to Cheri Ask #1: rather than streaming raw eye-camera frames in real time, allow Sevaro to receive short post-hoc video segments (e.g., per-task 30-second clips) for clinician review and off-device CV processing. Used as evidence for Issue 4 (face-tracking SERVICE_NOT_READY) by enabling MediaPipe Face Mesh post-hoc analysis on captured video. Less ambitious than #1 but easier to scope for an initial research agreement.
| Issue | Domain | Samsung Platform Ask | Cheri Research Pathway Ask | Status |
|---|---|---|---|---|
| 1 Compositor stutter | Eye | Ask 1 (TOP priority) | Partial via #1 | Open |
| 2 Eye-tracking specs | Eye | Ask 2 | Partial via #1 | Open |
| 3 Raw eye-camera | Eye | Ask 3 | #1 (primary), #4 (fallback) | Acknowledged consumer “No”; research-path pending |
| 4 Face-tracking SERVICE_NOT_READY | Face | Ask 7 | Partial via #4 | Open; reproduces in Google’s first-party app |
| 5 Inward camera held by inputserver | Face | Ask 3 | #1 (primary), #4 (fallback) | Acknowledged consumer “No”; research-path pending |
| 6 No real lower-body tracking | Motor | Ask 5 | Partial via #2 (Watch-on-ankle) | Open |
| 7 World-camera availability + FOV | Motor | Ask 4 | — | Open |
| 8 ~2 m safety boundary | Gait | Ask 6 (first) | #3 (Knox exception) | Acknowledged consumer “No”; research-path pending |
| 9 SLAM passthrough drift | Gait | Ask 6 (second) | — | Open |
| 10 Thermal envelope | Audio / system | (open question, no formal ask) | — | Open question |
Each row connects a measured platform issue to (a) the Samsung
platform-side ask from docs/SAMSUNG_XR_PLATFORM_ASKS.md and
(b) where applicable the research-partner pathway from the Cheri thread.
Bold entries mark the primary ask for that issue.
The findings in this report are drawn from ten rounds of on-device
validation on Samsung Galaxy XR loaner unit R32Y8003FKF
(SM_I610, Android XR), between 2026-02 and 2026-05. Each round captured
a full Core-15 protocol session, the resulting session JSON, and
synchronized logcat. The companion XR Test Companion tool captured
per-frame perf telemetry, including thermal events, frame-time
distribution, GC pressure, and memory peak. Where the underlying root
cause sits inside the OpenXR runtime or below, profiling was extended
via Unity Profiler (per-subsystem) and adb logcat filters
scoped to OpenXR, FaceProvider,
EyeTrackingProvider, and com.sevaro.sdne.
This work is positioned as V3+ analytical validation, in
collaboration with Dr. Jennifer Reneker at the University of Mississippi
Medical Center Center of Excellence. Platform-level rigor is established
here before clinical validation begins. Healthy-baseline runs (Steve
Arbogast, MD, clinically healthy) anchor the false-positive analysis:
any RED / INVALID flag on a healthy patient is reported either as a
platform issue (this document) or as a Sevaro-side scoring or capture
bug (tracked separately in repository commit history). The 2026-05-25
healthy session
(sessionId 77e2af69-2524-4178-8626-9f93f58d2385) is the
most recent and most thoroughly characterized data point and is
referenced throughout Section 4.
No clinical claims are made in this document. The asymmetry indices, drift measurements, and intrusion counts are reported as platform characterization data — they describe what the platform delivers (and fails to deliver) on a healthy subject, not what a clinical population would produce.
| Artifact | Location | Use |
|---|---|---|
| 2026-05-25 healthy session JSON | /storage/emulated/0/Android/data/com.sevaro.sense/files/sessions/SDNE_Core15_77e2af69-2524-4178-8626-9f93f58d2385_20260526_024516.json |
Issues 1, 4, 5, 6, 7, 8, 9 evidence |
| Local copy of same session JSON | /tmp/sense_fine_session.json |
Quick analyst inspection |
| T07 video-fallback failure JSON | Session JSON 2026-05-06;
videos/T07_<sessionId>_<ts>.mp4 (0 bytes) |
Issue 5 evidence |
| SAMSUNG_XR_PLATFORM_ASKS.md | docs/SAMSUNG_XR_PLATFORM_ASKS.md |
Asks 1–7 primary source |
| UNITY_BUG_face_tracking_service_not_ready.md | docs/UNITY_BUG_face_tracking_service_not_ready.md |
Issue 4 primary evidence |
| T07_VIDEO_FALLBACK_BLOCKED.md | docs/T07_VIDEO_FALLBACK_BLOCKED.md |
Issue 5 primary evidence |
| ONDEVICE_RUN_FINDINGS_2026-05-25.md | docs/ONDEVICE_RUN_FINDINGS_2026-05-25.md |
Synthesized 2026-05-25 evidence (Issues 1, 4, 10) |
| PLAN_2026-05-11_may27_demo.md | docs/PLAN_2026-05-11_may27_demo.md |
Cheri Asks #1–#4 primary source |
ADB commands to reproduce key evidence on a fresh loaner:
# Confirm device + serial
adb devices -l
# Confirm Galaxy XR permissions granted
adb shell cmd appops query-op --user 0 FACE_TRACKING
adb shell cmd appops query-op --user 0 EYE_TRACKING_FINE
# Reproduce Issue 4 (face SERVICE_NOT_READY) in Google's first-party app
adb shell am start com.google.xr.facetracking.calibration/.FaceTrackingCalibrationActivity
adb logcat -d | grep "xrGetFaceCalibrationStateANDROID"
# Pull the most recent SDNE session JSON
adb pull /storage/emulated/0/Android/data/com.sevaro.sense/files/sessions/ ./sessions_pulled/
# Companion perf telemetry (XR Test Companion)
# Capture via the iPhone companion app while a session runs;
# export the JSON sidecar from the companion's session view.Logcat lines observed on every session, both on the Sevaro app
(com.sevaro.sdne / com.sevaro.sense) and
inside Google’s own first-party
com.google.xr.facetracking.calibration app:
E .../FaceProvider.cpp:351 [FaceProvider] Face state data was unavailable.
Could not retrieve region confidence data.
E .../FaceProvider.cpp:330 [FaceProvider] Face state data was unavailable.
Could not retrieve blend shape data.
E com.sevaro.sense XR_ERROR_SERVICE_NOT_READY_ANDROID in
xrGetFaceCalibrationStateANDROID: Failed to get face calibration state.
E .../XrFuncTable.h:119 xrGetFaceCalibrationStateANDROID failed with result Unknown.
W .../FaceProvider.cpp:225 [FaceProvider] Could not determine face calibration
state. Face state data may not yet be available.
The first-party reproduction (running Google’s
com.google.xr.facetracking.calibration app, observing
its PID in logcat) shows the same
XR_ERROR_SERVICE_NOT_READY_ANDROID repeated hundreds of
times. This conclusively places the failure inside the Galaxy XR runtime
/ service layer rather than inside Sevaro’s app or Unity’s package
layer.
| Version | Date | Notes |
|---|---|---|
| 1.0 | 2026-05-26 | Initial release. Drafted ahead of the 2026-05-27 Plano workshop. Status fields reflect pre-workshop state. |
Planned updates post-workshop:
6df9fd8 and
prior).