Learned Optical Sensing
Machine learning — Nov 2026 – Present
What if a camera only measured what the task needs? A simulated single-pixel camera whose light masks are learned to predict motion.
Independent research, simulation
What I did
- Simulated a learned single-pixel (DMD-style) camera: 16 trained light masks feed a sequence model that predicts a bouncing ball’s position 100 ms ahead.
- Reached 1.9 px error versus 0.7 px for a full-frame CNN, using 256× fewer readings.
- Found that mask models trained on clean scenes fail under lighting flicker (18.95 px error); adding one total-light detector cut error to 1.90 px with no cost on clean scenes.
- Benchmarked against full-frame, event-camera, random-mask and centroid-plus-Kalman baselines across five test conditions, and documented where the classical tracker still wins.
Stack
- Python
- PyTorch
- NumPy
- OpenCV
Collaboration
Developed with a collaborator, from a shared idea: let the optics choose task-relevant measurements.