Hugging Face has released ENEAS, a unified, text-promptable method designed for robust instance tracking and open-concept semantic discovery. The method targets several failure modes observed in state-of-the-art foundation segmentation models, including SAM 3, such as temporal hallucinations, spatial fragmentation, and semantic misclassification.
ENEAS addresses three key challenges: target disappearance and re-entry, extreme close-ups that cause fragmented segmentation, and semantic doppelgängers—visually similar artifacts like statues, paintings, or reflections that are often misclassified as the target entity. By extending the geometrically robust SeC architecture with a text-prompting adapter and leveraging temporal memory, ENEAS maintains tracking through disappearance without drifting to distractors, and keeps the target whole even when it fills the entire view.
For semantic discovery, ENEAS employs a verification layer that combines high-speed visual embedding matching with conditional VLM refinement. This layer invokes semantic reasoning only for ambiguous candidates, filtering out ontological errors that visual-only models cannot distinguish while keeping latency low. This design is particularly suited for 3D reconstruction, where a single misclassified distractor can corrupt the asset.
ENEAS unlocks high-quality semantic tracking and segmentation of video, broad libraries, and collections of temporally or spatially unordered data, with the discrimination to distinguish true instances from their look-alikes. The code and models are available on GitHub, along with a project page and an interactive demo on Hugging Face Spaces.