Mutism

🔗 Demo Video

A mobile solution designed to protect the daily lives of individuals on the autism spectrum who experience difficulties due to sound sensitivity.
Through real-time sound detection, the Mutism app helps reduce anxiety from triggering noises and enhances users’ independence and safety.

🌍 UN SDGs Aligned

📑 Table of Contents

1. Overview
2. User Research & Validation Process
3. MVP
4. Tech
5. Screen
6. Expected Effects
7. Future Development
8. Member

🧩 Overview

Autism?

Autism spectrum disorder (ASD) often includes heightened sensitivity to environmental sounds.
Mutism aims to empower users with real-time detection and personalized responses to stressful auditory environments.

Currently, about 1 in 100 people worldwide are diagnosed with autism, and among them, approximately 86.6% experience auditory hypersensitivity, making daily life challenging due to sound-related difficulties.

https://www.who.int/news-room/fact-sheets/detail/autism-spectrum-disorders

Pain Point

• Severe Anxiety from Sudden Noises
  • Unexpected sounds (e.g., honking, barking) can cause sensory overload, leading to panic.
  • Individuals may not know how to cope with the real-time noise, often displaying hypersensitive reactions and having difficulty calming down.
• Limitations of Conventional Noise Blocking
  • Even individuals capable of some self-regulation can only respond by wearing noise-canceling devices or covering their ears.
  • Since users must take manual action, it’s difficult to respond quickly when exposed to sudden sounds.
• Lack of Immediate Intervention by Caregivers
  • If a caregiver is not nearby, it becomes difficult to provide real-time support.
  • There is a need for an emergency function that can instantly notify and connect with a caregiver in crisis situations.

📋 User Research & Validation Process

📌 Background Research & Problem Definition

Individuals on the autism spectrum are often highly sensitive to sensory input, especially certain sounds, which can cause extreme discomfort. After conducting desk research, we wanted to understand the real-life challenges they face in daily life. To gain deeper insight, we conducted an interview with a social worker at the Sungmin Welfare Center.

• Discomfort with Noise
  • They are especially sensitive to sudden loud noises such as motorcycle horns in daily environments. We confirmed through the interview that individuals with autism do indeed experience significant discomfort from these kinds of sounds.
• Current Coping Behaviors
  • They often try to calm themselves by wearing noise-canceling headphones or covering their ears. According to the social worker, some individuals at the center who are particularly sensitive to sound even react by suddenly covering their ears very tightly.

Through this process, we found that many autistic individuals frequently engage in physical actions to block out disturbing sounds and that it’s often difficult for them to cope without the help of a caregiver.

“There needs to be a way for autistic individuals to escape from overwhelming sounds without having to physically cover their ears.”

🛠️ Feature Specification & Iteration Through Feedback

Through a second interview with the welfare center, we refined the features of our solution focused on managing sensitivity to specific noises.

• Since each autistic individual is sensitive to different types of sounds, we designed a personalized soothing message feature tailored to each user’s noise preferences. The feedback we received was very positive, with the response that this feature could be highly beneficial in helping autistic individuals manage daily life independently without a caregiver.
• We discovered that age or gender information is not particularly relevant when setting up user profiles. Initially, we planned to collect this information to help the AI generate more personalized calming messages, but based on the feedback, we revised the user input fields accordingly.
• To prepare for situations where the user is exposed to distressing noises for extended periods, we added a feature where repeated detection of sensitive sounds triggers an alert to the caregiver, allowing them to check in on the user's status in real time.

As a result, we clearly defined the app’s direction as:

Personalized noise classification → Delivery of soothing messages,

and improved the feature set based on direct feedback.

🧪 User Testing

After developing the app, we conducted interviews and user testing with two individuals with severe autism and a social worker at Seongmin Welfare Center to evaluate the app’s usability and determine whether it would genuinely benefit autistic individuals.

According to the social worker, although the participating individuals were classified as having severe autism in terms of behavior and intellectual level, they had relatively good cognitive abilities and were capable of self-regulation thanks to education received at the center. While they could understand instructions, they had limited expressive ability. Therefore, the social worker's professional input played a crucial role in helping us further improve the app.

Since the app is in English, we first played a screen recording with Korean subtitles to help participants understand the overall flow.

At this point, regarding the white noise selection feature, the participants mentioned that it was well-designed because autistic individuals often prefer sounds like forest ambience or bubbles, and these preferences were well reflected in the app.

Finally, we received feedback on the strengths of our app and suggestions for additional features.

First, participants expressed a strong preference for the color blue in the design. They also noted that the voice conversion feature helped increase comfort and a sense of calm for autistic users. They especially appreciated that the app was designed to provide personalized calming messages rather than generic, uniform ones. Overall, the autistic participants showed a desire to download and use the app, recognizing its meaningful solution to the problem of enduring discomfort caused by noise.

Additionally, the feedback pointed out that the SOS button might be difficult for users to operate directly, and that in cases of prolonged exposure to sensitive noise, automatic calls to caregivers might not allow for immediate assistance. However, the idea of informing caregivers about the user’s situation was seen as valuable. Based on this, we plan to improve this feature by sending alert messages via SMS in the future.

🚀 MVP

1. Real-Time Noise Detection and Classification

Mutism uses the YAMNet model to classify environmental sounds collected in real-time, enabling accurate noise detection.

Before starting detection, users select the specific sounds they are sensitive to.

• The main screen displays real-time sound tags (e.g., car, crying, shouting), allowing users to intuitively understand their current auditory environment.
• The app leverages a foreground service, ensuring that even when the screen is off or the app is running in the background, it can still detect sensitive sounds and send push notifications. This enables users to use the app safely even when outside.

2. Personalized Calming Messages + White Noise Playback

When a sensitive sound is detected, a personalized calming message is generated using the Gemini API, based on the user’s profile information, and played aloud via TTS.

(e.g., “You’re safe now. Let’s try finding a quieter place.”)

• The calming message takes into account the user's name, age, and preferred relaxation methods.
• Once the voice message ends, the user's pre-selected white noise (e.g., ocean waves, birds chirping) automatically plays to help promote emotional stability.

3. Automatic Guardian Notification for Prolonged Exposure

If sensitive sounds are detected repeatedly over a certain threshold, the situation is regarded as potentially dangerous, and the app will automatically call the guardian registered in the user’s profile.

• The guardian’s phone number can be entered through the user profile, and in emergencies, the app initiates a call to notify someone nearby of the user’s condition.
• This feature serves as a practical safety net for users on the autism spectrum who are sensitive to sound.

Key Features

✨ Tech

Tech stack

Language

• Kotlin: Used for Android app development.

Architecture

• MVC (Model - View - Controller)
  • A simple structure with minimal asynchronous network response handling or complex state management.
  • Since the service primarily operates on-device with minimal external communication, the MVC pattern was considered more structurally efficient.

Data Management

• EncryptedSharedPreferences: Used to securely store user information.

AI

• Gemini API (Gemini-2.0-flash)
  • Suitable for generating personalized text considering various individual characteristics.
  • Capable of creative text generation, making it well-suited for TTS (Text-to-Speech) applications.
• YAMNet
  • A lightweight sound classification model for real-time environmental noise detection.
  • Optimized for mobile environments.
• TensorFlow Lite
  • A lightweight ML framework that enables efficient execution of models on mobile and IoT devices.
  • Used to run the YAMNet model quickly and efficiently on mobile devices.

Network

• OkHttp: Supports efficient HTTP network communication.

System Workflow

Architecture

📱 Screen

Home

• When the Start button is pressed, sound classification begins in the background.
• The classified sound is displayed on the screen.

Emergency contact

• When the user presses the SOS button, the app immediately dials the emergency contact registered in their profile.
• If a sensitive sound is continuously detected for a certain period during real-time noise monitoring, the app automatically connects a call to the emergency contact.

Real-time Noise Detection / White noise playing

• When real-time noise detection begins, tags representing the currently detected sound are displayed at the bottom of the screen.
• If a sensitive sound is detected, a calming voice message is played to help soothe the user (refer to the PPT video).
• After the calming message finishes, the white noise previously selected by the user is automatically played.
• While the white noise is playing, a ‘Stop White Noise’ button appears on the screen.
  • Every 5 minutes, a toast message and notification prompt the user to check their current state and encourage them to stop the white noise if they feel calm.

Mypage / NoiseSelect / WhiteNoise

Mypage	NoiseSelect	WhiteNoise

• On the My Page screen, users can enter their personal information and access buttons that lead to the Noise Selection and White Noise Selection pages.
• In the Noise Selection page, sounds are categorized into main, sub, and detailed categories. Users can select tags using tabs and save their preferences.
• In the White Noise Selection page, users can choose and save one preferred white noise; only one option can be selected at a time.

Enter User Information & Select Noises Dialog

User Information Input Dialog	Noise Selection Dialog

• If the user attempts to start real-time noise detection without entering all required information, a modal appears guiding them to the My Page screen to complete their profile.
• Similarly, if the user hasn't selected any sensitive sounds, a modal prompts them to go to the My Page screen to complete sound selection before proceeding.

Real-time Noise Detection Alert

💡 Expected Effects

• Supporting Safe Daily Life for Individuals on the Autism Spectrum

  • Mutism classifies and detects ambient sounds in real time, alerting users when sensitive sounds are detected.

  • This helps users avoid overstimulation and enhances their sense of safety in everyday life.

• Promoting Emotional Stability with Personalized Calming Messages and White Noise

  • The app helps users who are sensitive to external stimuli regain emotional stability quickly and provides an environment to prevent sensory overload.

• Enhancing Independence and Autonomy

  • Users can manage their own calming process without relying on caregivers, promoting greater independence in daily routines.

• Reducing Caregiver Burden with Emergency Alerts

  • If a user is repeatedly exposed to a sensitive sound, the app automatically notifies the caregiver.

  • This allows for real-time monitoring and quick intervention, even when caregivers are not physically present.

• Delivering Personalized Calming Strategies Based on User Information

  • Using Gemini, Mutism provides personalized calming messages tailored to each user.

  • By analyzing both sound sensitivities and sound preferences, the app delivers truly customized support.

🏃‍♀️ Future Development

• User-Specific On-Device AI

  • We plan to enhance personalized calming solutions by allowing the on-device AI model to learn and adapt to each user's unique data.

• Integration with Noise-Canceling Wearable Devices

  • By connecting with wearable devices equipped with active noise canceling, we aim to provide fundamental sound-blocking functionality.

• Caregiver Voice-Based Calming Messages

  • Calming messages will be delivered using the caregiver’s voice to offer greater emotional comfort and a sense of security.

• Personalized Safe Space Recommendations

  • The app will suggest nearby safe and quiet places tailored to the user's preferences and needs.

💙 Member

Member	Eunsoo Kim	Jaewon Hong	Sieun ko	Hyojin Lim
Role	AI	Design	Android	Android/Server
profile