Problem

Solution

Research

Design/dev

Evaluate

Further dev

Takeaway

AR Accident Simulation 

Accident Simulator is an AR product designed at Phoenix On the Ground(POG) to prevent traffic accidents by simulating frequently happening accidents between cars and pedestrians. The application works based on deep learning data that perceive lanes.

I worked as an AR programmer and product designer to build a safe city together with a deep learning engineer and a project manager. We are continuously developing our content through user testing and researching the effectiveness of the app in decreasing future traffic accidents.

PROJECT Timeline

2019 – 2021

Category

Augmented Reality
User Research

Responsibilities

AR Programming
Interface Design
3D Character Design
Research Planning

Team

2 developers
1 deep learning engineer
1 designer
1 project manager

Tools

Unity – ARFoundation
App – Sketch
Fusion 360
Blender

01. Problem statement

South Korea is spending a large number of social expenses due to traffic accidents. Prevention by education is more important than handling accidents.
Education for accident prevention, however, does not reflect the main cause of the accidents – negligence in keeping eyes forward.

We seek to guard drivers and pedestrians against accidents by designing immersive educational content.

02. Solution

We designed a geo-location-based AR app and 360 videos to educate the drivers and pedestrians about traffic accidents.

The experience brought other’s traffic accidents to an individual level, impacting users to prevent the accident by reminding the implanted memories.

10 AR simulations

Designed immersive content simulating frequently happening accidents

Drivers & pedestrians

Created separate content with ‘driver point of view’ and  ‘pedestrian point of view’ to effectively communicate how accidents occur

AR contents(10 CASES)

Developed AR app simulating
accidents between pedestrians and drivers

Integrated deep learning to
detect the lanes

A pedestrian playing AR simulation demo on display

360 video(30 CASES)

Developed 360 video version of the app to supplementation issues in AR app

Created separate content with ‘driver point of view’ and ‘pedestrian point of view’ to effectively communicate how accidents occur

A pedestrian playing 360 video simulation

03. Result

The project successfully got funded by multiple public and private corporations and we are aiming to launch the app to the public.

A user scanning the QR code at the POG station to start the experience

Fundings

Korea Expressway Corporation
Korea Institute of Civil Engineering and Building Technology (KICT)
Yonsei University Techstars
Goyang K-startup

App

Aim to publicize the app on 22’

QR CODE device

QR code device installed in a few locations for further user testing

04. Process

For this project, I joined the team in the design and prototype phase. Business concepts and plans were built by the team leader.

The process I have not participated in is marked in grey.

Research
(2019)

Field research
Competitive analysis

Design and develop
(2020)

Scenario design
Character design
Unity AR development
Unity 360 video dev
App design
Deep learning integration

Evaluation
(2020)

User testing

Development
(2021~)

Simulation cases expansion
Android App development

05. Research

Traffic accidents are increasing year by year, showing a decrease in kill figures and an increase in injured figures. It is important to figure out the focal reason for accidents and identify the different patterns. We found drivers’ and pedestrians’ failure to keep their eyes on the road is the main issue, which is less recognized.

The user-based insurance (UBI) market is increasing. Most of the companies reflect driver’s behavior on the bill, such as vehicle miles and acceleration/ decelerationHowever, those UBI strategies only focus on the cars, not focusing on how different actions drivers take.

 

Field research

Looked into the current status and mitigation strategy for traffic accidents

Competitive analysis

Looked into user-based insurance market(UBI) checking their strategy and how they contribute to minimizing traffic accidents

Co-design Workshop

From the primary research, I found the design opportunity of creating embodied PSAs where the audience gets augmented to the victim of the PSA. To get more ideas on how to design the experience, I conducted co-design workshops.
Participants, interaction designers, were told to design interaction scenarios under the issue of domestic child abuse. PSA videos and posters were given for reference along with an augmented experience where participants can embody themselves onto.

WHO

Three groups, totall of 10 interaction designers

Process

  1. Introductin (15 min)

  2. Ideation on interactive PSA scenario on child abuse issue (25 min)

  3. Focus Group Interview (20 min)

CO-DESIGN WORKSHOP Insights

Restrictions

Intended restriction of the augmented characters’ movements was often brought up. Participants used the restricted interaction to symbolize enervation and helplessness feeling to the audience.

Posture

The posture of the character in the designed PSA scenario shared the commonality. Upright postures were often chosen to draw connections between the passersby and virtual characters.

06. Opportunity

“Planting memory of near-miss traffic accident”

Developing the ability to prepare for similar accidents by instilling memories of accidents is the most effective solution we found to the main cause of traffic accidents.

AR/VR, FIRST-PERSON ACCIDENT SIMULATION

AR/VR simulation allows an individual to situate oneself as the subject at the accident and train oneself to take action when similar incidents happen in real-time

New UBI model

UBI model reflects the user’s daily effort in preventing traffic accidents. The more user participate in the immersive experience, the more benefits one gets on the insurance bill

07. Design & develop

User sCENARIO DESIGN

Create an interaction scenario for a safe immersive experience

character DESIGN

Design a 3D model for simulation without 
traumatic influence

Develop, phase1

Create an AR app and integrate deep learning for precise adjustment

DEVELOP, PHASE2

Create 360 videos and design an app to publicize the service

USER SCENARIO DESIGN

The traffic accident simulation happens at the site where accidents are prone to happen. The user uses one’s phone and goes through a vicarious experience.

Considering the context, we aimed to design a safe service, reminding users of the surrounding through visual and audio cues.

A. The user finds the POG marker on the street and scans the QR code to start the AR experience that fits the corresponding location. The marker alerts the user of the surroundings and notifies them once the traffic signal changes.

B. The user follows the instruction on the AR app. The guideline allows the user to stand at the secure position and point the camera at the proper angle.

C. The user learns about the accident frequently happening on the location through AR simulation.

D. The user checks out the information on the injury and a written description of the cause of the accident.

See design iterations
CHARACTER DESIGN

I created a 3D character Pog Bot.
The bot substitutes the pedestrian, reducing the shock that may come from the accident simulation.

3D character designed in Fusion 360

Nodding and looking around motion applied in Unity

develop phase 1

AR DEVELOPMENT

With the designed character and user scenario, we developed an AR app, coming up with ten different cases demonstrating frequently happening accidents.

AR simulation on the accident between car and pedestrian 

DEVELOP PHASE 1

DEEP LEARNING INTEGRATION

The service aims to provide a seamless user experience where the AR simulations are augmented at the right position, especially in vehicles.

I worked with a deep learning engineer who trained a deep learning model to differentiate between lanes and not-lanes and converted the data into coordination usable at Unity. I was responsible for using those data and augmenting the simulation at the right location on the screen.

Scene to train

Trained result showing lane in white and non-lane in black

Change of car position after applying the trained result, moving from the leftmost to the rightmost corner of the lane

DEVELOP PHASE 2

360 VIDEO DESIGN

From the AR development, we found the limitation of AR. It is impacted strongly by the environment, unable to augment the simulation during the night time or when the place is too crowded. Deep learning recognition also got influenced, often showing lower accuracy.

We devised a new plan for the problem we encountered, choosing to use a 360 video solution until we come up with a solution for the AR.

Demo of 360 video

DEVELOP PHASE 2
APP DESIGN

Our service not only provides educational content for new user-based insurance(UBI) but also provides a platform to easily connect to the insurance agency prior or after to the accidents.
I designed an app where user can play AR content, track UBI benefit status, and access different services business provides.

08. Evaluate and result

Selected for the Scale-Up project from the Ministry of Land, Infrastructure and Transport, we got funded to conduct user testing to find out the effectiveness of the AR accident simulation.

30

Participants

Recruited six people per age group
between teens and fifties

4

phases of test

Conducted four phases of test examining the difference before and after

Brainwave & eye tracking

Tested response, trauma stability, perception of reaching risk area

A participant going through user testing

Research question

The survey and interview sought out to answer the following:

  • Text
TEST procedure
PHASE 1

Pre-test
(Before any experience)

PHASE 2

AR experience

PHASE 3

Audit video on real traffic accidents

PHASE 4

Post-test
(After AR experience)

USER TEST RESULT

“AR traffic accident simulation system is a psychologically stable training method and shows a significant impact on preventing incidents”

(Eye-tracking)
Field distribution adequacy

The ability to respond to accidents
increased by 69.7 percent

(Brainwave)
Trauma stability

AR simulation is safer than educating with
real accident video by 14.8 percent

(Brainwave)
Perception of reaching risk area

In the second week,
increased by 43.3 percent

09. Further development

After we proved the effectiveness of augmented traffic accidents education, we expanded the scope of traffic accident simulation into three different types of perspectives.

Pedestrian point of view simulating accident between pedestrian and car

Driver point of view simulating accident between pedestrian and car

Driver point of view simulating accident between cars

We manufactured a QR code device and installed it in a few locations for user testing.

Finally, we are working on stabilizing the AR experience and launching the app in 22′

10. Takeaways & Inclusive service design

From working at a start-up as a founding member, I was able to wear different hats as a developer, researcher, and designer.
Working with the small team, I was able to situate myself on the manager’s and engineer’s point of few and learn how to stretch out the business.
The challenge we had during our AR development enabled me to seek and shift new direction and also get ready for future challenges.

Current projects are targeting drivers who are familiar with XR experiences. From recent research(2022) I found our approach is excluding a critical target, seniors, who may benefit most from immersive simulations. Especially 360 video version would be appropriate as accident simulation is accessible at home for seniors with different physical difficulties and undergoing loss of recognition and memory loss. Therefore, I personally dived into creating an inclusive version of the service.

WIREFRAME
OVERVIEW OF POG FOR SENIORS
Skip to content