From sound to embodiment: AI sound imitation technology driven by N/CM model enables immersive communication:Take the first person monologue narration video as an example

Abstract

Recently, video commentaries featuring first-person monologues have gained popularity online. These videos use AI sound technology to reconstruct the story world through the subjective perspectives of characters in the drama, allowing viewers to ‘embody’ the characters and engage in immersive storytelling. This study employs multimodal analysis (Research 1) and controlled variable experiments (Research 2), based on the narrative-coordination model (N/CM, Narration/Coordination Model). It sets up a scenario experiment with two commentary perspectives (first-person and third-person) and two user technology acceptance levels (high-tech and low-tech users). By analyzing the dimensions of sound and visuals in first-person monologue commentaries, the study explores how AI sound technology endows characters with vivid ‘voice life’ (Research 1). It also reveals the unique advantages of first-person narration in narrative depth, emotional resonance, and audience interaction (Research 2), aiming to explore the feasibility of using AI sound technology to enhance the film and television industry and to create embodied immersive experiences.