Problem

Solution

Research

Design/dev

Evaluate

Further dev

Takeaway

AR Accident Simulation 

The Accident Simulator is designed to prevent traffic accidents by simulating frequently occurring accidents as AR and 360 video experiences.

I designed an XR app that allows users to easily navigate different scenarios and also created an interaction flow for the XR experience.

Work timeline

2019 – 2021

Category

Augmented Reality
User Research

Outcome

XR book and app launch
Selected for Gyeonggi Content Agency AR/VR competition
Success in attracting a series of fundings

Responsibilities

I designed an XR web app and developed AR/360 video accident simulation scenarios that are integrated into the app. In addition, I collaborated with the team to plan user research and was responsible for directing the visual elements, including marketing materials.

PROJECT Timeline

2019 – 2021

Category

Augmented Reality
User Research

Responsibilities

AR Programming
Interface Design
3D Character Design
Research Planning

Team

2 developers
1 deep learning engineer
1 designer
1 project manager

Tools

Unity – ARFoundation
App – Sketch
Fusion 360
Blender

About the project

  • Phoenix On the Ground(POG) is a startup aimed at creating safer cities through XR solutions.
  • The team consists of engineers developing apps and utilizing deep learning, PMs, and a CEO.
  • We launched our service, XR book, in 2023 and installed QR code devices in a few locations to attract users and conduct further user testing.
  • As the only designer on the team, I was responsible for app, web, 3D assets, and marketing material design.
  • I also developed 30 different AR and 360 video simulations using Unity ARfoundation and collaborated with PMs and engineers to conduct user research using methods such as eye-tracking and EEG.
see more (in Korean)

Process

I joined the team during the design and prototyping phase, after the team leader had developed the business concepts and plans. I contributed to refining the value propositions and creating the designs and interactions to bring the idea to the viable product.

The process I have not participated in is marked in grey.

Research
(2019)

Field research
Competitive analysis

Design and develop
(2020)

Scenario design
Character design
Unity AR development
Unity 360 video dev
App design
Deep learning integration

Evaluation
(2020)

User testing

Development
(2021~)

Simulation cases expansion
Android App development

Research (2019)

  • Market research
  • Competitive analysis

Design and develop (2020)

  • Interaction scenario design
  • Character design
  • Unity AR/360 video development
  • Deep learning integration

Evaluate (2020)

  • User testing

Refine (2021 ~)

  • Simulation cases expansion
  • Web app design and development

Context

  • South Korea is spending a significant amount of money on social expenses related to traffic accidents, with a particular focus on education.
  • However, education for accident prevention often fails to address the main cause of accidents – negligence in maintaining proper focus while driving or walking.
  • Therefore, we proposde to design immersive educational content that will guard drivers and pedestrians against accidents.

Result

The project successfully got funded by multiple public and private corporations and we are aiming to launch the app to the public.

A user scanning the QR code at the POG station to start the experience

Fundings

Korea Expressway Corporation
Korea Institute of Civil Engineering and Building Technology (KICT)
Yonsei University Techstars
Goyang K-startup

App

Aim to publicize the app on 22’

QR CODE device

QR code device installed in a few locations for further user testing

Preliminary research

Our team conducted field research to explore the current status and mitigation strategies for traffic accidents. We also performed a competitive analysis to determine how user-based insurance (UBI) can contribute to minimizing traffic accidents.

Field research

We found that the number of traffic accidents is increasing every year, with a decrease in fatalities but an increase in injuries. It is important to identify the primary causes of accidents and their different patterns. We found that the failure of drivers and pedestrians to keep their eyes on the road is the main issue, which is often overlooked.

Competitive analysis

The user-based insurance (UBI) market is growing. Most companies reflect a driver’s behavior on their insurance bill based on vehicle miles and acceleration/deceleration. However, these UBI strategies only focus on the car and not on the different actions drivers take.

Field research

Looked into the current status and mitigation strategy for traffic accidents

Competitive analysis

Looked into user-based insurance market(UBI) checking their strategy and how they contribute to minimizing traffic accidents

Co-design Workshop

From the primary research, I found the design opportunity of creating embodied PSAs where the audience gets augmented to the victim of the PSA. To get more ideas on how to design the experience, I conducted co-design workshops.
Participants, interaction designers, were told to design interaction scenarios under the issue of domestic child abuse. PSA videos and posters were given for reference along with an augmented experience where participants can embody themselves onto.

WHO

Three groups, totall of 10 interaction designers

Process

  1. Introductin (15 min)

  2. Ideation on interactive PSA scenario on child abuse issue (25 min)

  3. Focus Group Interview (20 min)

CO-DESIGN WORKSHOP Insights

Restrictions

Intended restriction of the augmented characters’ movements was often brought up. Participants used the restricted interaction to symbolize enervation and helplessness feeling to the audience.

Posture

The posture of the character in the designed PSA scenario shared the commonality. Upright postures were often chosen to draw connections between the passersby and virtual characters.

Design opportunity

Planting memories of near-miss traffic accidents

Based on preliminary research, we found that helping users improve their ability to prepare for similar accidents by instilling memories of accidents is the most effective solution for preventing traffic accidents.

AR/VR for first-person view accident simulation

AR/VR simulations enable individuals to put themselves in the position of a subject at the site of an accident. This allows them to train themselves to take action when similar incidents occur in real-time.

New UBI model

The UBI model reflects the user’s daily effort in preventing traffic accidents. The more the user participates in the immersive experience, the more benefits they receive on their insurance bill.

Design process

I began by designing an interaction scenario for the entire experience, from the first encounter with the service to turning an immersive experience into educational moments. With the established scenario, I developed content that includes 3D characters and AR and 360 scenarios in the Unity game engine. Additionally, I designed a mobile app to store our AR experience and other features to create profit model for the product.

User scenario design

Create an interaction scenario for a safe immersive experience

Character design

Design a 3D model for simulation without traumatic influence

Develop, phase1

Create an AR app and integrate deep learning for precise adjustment

Develop, phase2

Create 360 videos and design an app to publicize the service

CHARACTER DESIGN

I created a 3D character Pog Bot.
The bot substitutes the pedestrian, reducing the shock that may come from the accident simulation.

User scenario design

Traffic accident simulations occur at locations where accidents are likely to happen, and users can use their phones for an augmented reality experience.
Considering the environment, we aimed to design a safe service that reminds users of their surroundings through visual and audio cues.

A. The user finds the POG marker on the street and scans the QR code to start the AR experience that fits the corresponding location. The marker devcie helps users become aware of their surroundings and notifies them when the traffic signal changes.

B. The user follows the instructions on the AR app. The guidelines direct the user to stand in a secure position and point the camera at the proper angle.

C. The user learns about the frequent accidents that occur at the location through an AR simulation.

D. The user reviews the information on the injury and the cause of the accident.

See design iterations

Challenge and outcome

Initially, we planned to design a geo-location-based AR app to educate drivers and pedestrians about traffic accidents. To achieve this, I developed AR simulations in Unity and worked with a deep learning engineer to provide a seamless user experience where the AR simulations are augmented at the exact lane.

However, during the process, we encountered two following limitations of integrating AR into a mobile app.

  • AR is strongly impacted by the environment and unable to augment the simulation during nighttime or when the location is too crowded.
  • Deep learning recognition is often influenced, resulting in lower accuracy. Additionally, it can be too heavy and take up too much data to run in a mobile app.

10 AR simulations

Designed immersive content simulating frequently happening accidents

Drivers & pedestrians

Created separate content with ‘driver point of view’ and  ‘pedestrian point of view’ to effectively communicate how accidents occur

Initial plan:
AR contents (10 cases)

I developed an AR app that simulates accidents between pedestrians and drivers. The app integrates deep learning to detect lanes.

An AR demo on display, simulating pedestrian trying AR experience

Modification

We developed a new plan for the problem we encountered. We decided to use a 360 video solution until we find a solution for AR. This decision solved other issues of simulating an outdoor environment, while also opening up opportunities for a new business model, the XR book. This will expand our reach of solutions not only for pedestrians, but also for drivers.

Modified plan:
360 videos (30 cases)

I developed 360 videos to supplement the engineering limitations. We created content for both drivers and pedestrians to effectively communicate how accidents occur from different points of view.

A pedestrian playing 360 video simulation

Evaluation

We used our 360 video solution to conduct user tests and evaluate the effectiveness of an immersive accident simulation. Participants wore eye-tracking devices and Bluetooth Low Energy, and went through four different phases of testing to compare and contrast the impact of the experience. I contributed to planning the user testing. 

30

Participants

We recruited six people per age group
between teens and fifties

4

Phases of test

We conducted four phases of testing to examine the differences before and after

Brainwave & eye tracking

We tested the user’s response to simulation, trauma stability, and perception of reaching a risky area

A participant going through user testing

Research question

The survey and interview sought out to answer the following:

  • Text

User test process

Pre-test evaluating participants’ behavior  before the 360 exeprience 

The participant views a 360-degree video simulation

The participants views real traffic accident videos

Post-test evaluating paritipants’ behavior after 360 simulation

User test result

“The AR traffic accident simulation system is a psychologically stable training method that has a significant impact on preventing incidents.”

(Eye-tracking)
Field distribution adequacy

The ability to respond to accidents
increased by 69.7 percent

(Brainwave)
Trauma stability

360 simulation is safer than educating with real accident videos by 14.8 percent

(Brainwave)
Perception of reaching risk area

In a week, the perception of reaching a risk area increased by 43.3 percent

App design

Our service not only provides educational content for new user-based insurance (UBI), but also offers a platform for easily connecting to the insurance agency prior to or after accidents. I designed an app that allows users to play 360 video content, track their UBI benefit status, and access different services provided by the business.

develop phase 1

AR DEVELOPMENT

With the designed character and user scenario, we developed an AR app, coming up with ten different cases demonstrating frequently happening accidents.

AR simulation on the accident between car and pedestrian 

DEVELOP PHASE 1

DEEP LEARNING INTEGRATION

The service aims to provide a seamless user experience where the AR simulations are augmented at the right position, especially in vehicles.

I worked with a deep learning engineer who trained a deep learning model to differentiate between lanes and not-lanes and converted the data into coordination usable at Unity. I was responsible for using those data and augmenting the simulation at the right location on the screen.

Scene to train

Trained result showing lane in white and non-lane in black

Change of car position after applying the trained result, moving from the leftmost to the rightmost corner of the lane

DEVELOP PHASE 2

360 VIDEO DESIGN

From the AR development, we found the limitation of AR. It is impacted strongly by the environment, unable to augment the simulation during the night time or when the place is too crowded. Deep learning recognition also got influenced, often showing lower accuracy.

We devised a new plan for the problem we encountered, choosing to use a 360 video solution until we come up with a solution for the AR.

Demo of 360 video

3D character designed in Fusion 360

Nodding and looking around motion applied in Unity

Further development

After demonstrating the effectiveness of immersive simulation in traffic accident education, we expanded the scope of traffic accident simulations to include three different perspectives.

We manufactured a QR code device and installed it in a few locations for user testing.

Pedestrian point-of-view simulating accidents between pedestrians and cars

Driver point-of-view simulating accidents between pedestrian and car

Driver point-of-view simulating accidents between cars

Takeaways & inclusive design

As a founding member of a startup, I had the opportunity to wear different hats, including those of a developer, researcher, and designer. Working with a small team, I gained insights of both PM and engineers, and learned business insights. Facing challenges during AR development has enabled me to shift direction quickly and work within contrains.

The current project targeted users who are familiar with emerging technology. However, a recent research study suggests that our approach excluded an important demographic: seniors, who may benefit most from immersive simulations. Specifically, a 360 video version would be appropriate, as the simulation would be accessible at home for seniors with different physical difficulties and undergoing loss of recognition and memory loss. Therefore, I personally worked on creating an inclusive version of the product.

Wireframe

I focused on the following points for POG for seniors.

  • Customizable settings to match the user’s physical and mental abilities
  • An easy-to-grasp point and discount system to encourage learning
  • Easy hand controller to assist with 360-degree experience

App usage video

Skip to content