01 · AI + mental health
HARMONI
Harmonically Adaptive Real-Time Multimodal Output for Neural Integration
A portable platform that combines facial, vocal, and physiological signals to estimate emotional state, then creates personalized music and visuals for stress and anxiety support.
HARMONI uses an 11-model ensemble, multimodal fusion, retrieval-augmented personalization, agentic orchestration, and generative music and visuals. Four controlled experiments examined emotion recognition, personalization, adaptation, and AI-generated versus human-composed music.
88.3%hybrid accuracy
0.80Cohen's κ
108participants
4experiments
PerceptionFacial, vocal, HR & GSR signals
Intelligence11-model ensemble & fusion
PersonalizationRAG & agentic orchestration
GenerationMusicGen & SD-Turbo
Independently built: I conceived, designed, programmed, integrated, and tested the complete system.
