Realingo was inspired by the growing need for today’s society to be more fluent in different languages. As people are becoming more connected around the world through social media and easier access to traveling, the need to communicate with others from different cultures and languages is becoming prominent. 

Personal contributions:

• Logo design;
• UX/UI design;
• AR adventure feature design;
• Usability testing;
• Video production.

Realingo is a course project supervised by professor Thomas Furness, a virtual reality pioneer at University of Washington. At the beginning of the project, I was given a challenging topic: design a solution for language learning that applies virtual reality technology. Language learning is never easy, as a language learner myself, I felt the most challenging aspect is to find an environment that you can immerse yourself in a different language. However, a paradox is that we often learn language because we are going to be in a place where people speak a different language, so we want to prepare for it in advance. Hence, if there is a language tutoring solution that virtually immerse users in a language environment, we could significantly speed up their language learning processes.

Problem Statement: How can we leverage advances in technology to provide a more effective and motivating solution to learn languages?

Phase 1: Research

Learning a new language is difficult for many reasons, some of which include: steep learning curve, resource limitations, lack of commitment, etc. The first phase of the project was to conduct research to identify the pain point of language learners and to examine contemporary solutions.

Literature Review

The first step of the research is to conduct literature review to examine current solutions. During the summer of 2016, the world was going crazy by a mobile game called Pokémon Go that utilizes AR in its gaming experience. This Pokemon Go phenomena triggered my interest. If a mobile game can make users playing it by letting them to explore surroundings and interact with other users, then a learning APP might also have such power. In 2012, a group of researchers developed a Spanish learning game called Mentira. The game lets users to explore locations using real map data. Instead of catching Pokémon, users catch interesting sights and stories at certain locations. Such idea resulted in a great success. 

Survey

Realingo administered an online questionnaire through Google Forms to obtain a better understanding of our problem space. Participants all agreed to let Realingo use the information they provided for academic purposes, and their privacy is well protected. Below are our significant findings based on this study:

• Speaking is the most challenging aspect when people learn a new language as 71% of our users indicated it was their problem area.
• 57% of users answered that their favorite way to approach learning new languages is by attending class.
• Many of our users responded that they like the flexibility and responsiveness that mobile/interactive language tutors give them.
• 100% of our users answered that they enjoy learning a language with other people.

Competitive Analysis

Aiming to bring business mindset to the design process, I proposed to conduct the competitive analysis to understand the strengths and weaknesses of our competitors.

Research Findings

Overall, the study provided us many insights of what to focus on our product design, such as developing location-based tools, classroom mimicking, and reducing user’s stress. I also learned that several aspects of design are vital and may even cause failure if not handled well, like that we need abundant contents to keep users interested. Also, designing an appropriate achievement/rewarding system that is both motivating and lasting could be fairly important.

Product Requirements

After the analysis of the survey responses in addition to our other research findings, I believed our product should meet the following requirements:

• The product should be simple and easy to use with a minimalist user interfaces that makes the learning process intuitive and enjoyable to all user groups.
• The product should provide verbal interactions that engage and challenge the user in order to increase their knowledge of the chosen language.
• The product should offer personalized lesson content based on the needs of the user. The user should be able to pick challenges that are geared toward specific skills the user is lacking, and track their progress through gamification features such as achievements and badges.
• The product should positioning the service with relevant AR implementations. According to Godwin’s article, if an AR gaming element is well designed, it offers engagement and entertainment along the way, improving language learner’s interest (Godwin).

Phase 2: Persona

The persona was developed based on a typical collage student who is learning a new language for upcoming study abroad program. A typical day of such persona could be:

Samantha wakes up in her cozy, two-bedroom apartment she shares with her roommate. She rushes to get ready for class as she grabs a coffee and a bagel on her way to her 8:30 Psychology lecture. She attends two more classes, grabs a quick lunch in the cafeteria, and then hits the library to do some studying till about 4:00. Samantha gets home around 4:30 and goes for a short run around campus before it gets too dark outside. She makes dinner with her roommate and calls home to check in with her family. After dinner she sits down to read through her textbook in preparation for the next day and flips through a study guide she has on her phone. Samantha ends the night by searching through travel magazines of Spain as she plans her upcoming study abroad trip.  

Phase 3: Storyboards

A Typical Day in the Life of Samantha Turner before Realingo

Samantha wakes up early to get ready for another busy day. She grabs a quick breakfast and heads to campus where she meets up with her friend Brian before class. Since Samantha is a busy college student she had completely forgot about the assignment that is due in her Spanish class at 2:00pm. With only a half hour before class Samantha panics and runs to the library to start on the assignment.

The assignment covers an extensive amount of reading material and Samantha fears she may not get it done in time. Samantha sits in the library frantically trying to read through all the pages and due to her poor translating skills she ends up only completing half the assignment. The tool she uses is Google Translate where she manually types the Spanish words and the website shows their English translations. Due to this manual process, Samantha runs out her time quickly.

Samantha enters class feeling unprepared and frustrated with her assignment progress. She hands in the assignment and reluctantly takes the 50% grade, dropping her overall class grade. After a long day of classes and studying Samantha heads back to her apartment where she unwinds by flipping through magazines of Spain as she plans her upcoming study abroad trip. She is disheartened by her Spanish grade and fears that living in Spain for a semester may not be the best experience anymore. She also worries about her speaking skills because she does not know any native Spanish speaker, so she has few chances to practice speaking.

A Typical Day in the Life of Samantha Turner after Realingo

Instead of panicking that her Spanish assignment is due in 30 minutes, Samantha knows that Realingo will help her complete her assignment. Once Samantha gets to the library she pulls out her augmented reality glasses provided by Realingo and has her textbook translated to Spanish. She reads her textbook in English and the Spanish translation appears in text-form right in front of her, making it easy to complete her assignment. Thanks to Realingo Samantha is able to finish her assignment with time to spare! Samantha walks to class feeling relaxed and prepared for the lecture. She hands in her assignment and sits down, excited to learn more about the Spanish language. 

Samantha attends the rest of her classes and heads home to relax. After having such a successful experience with Realingo earlier that day, Samantha decides to use Realingo to help her plan her upcoming study abroad trip next semester. She uses Realingo’s object recognition feature to practice learning Spanish in her own bedroom by identifying different furniture in Spanish. She also uses the AR function to have a Spanish conversation with a Realingo avatar, aiding in her Spanish speaking ability. All this interaction with the Spanish language leaves Samantha feeling prepared and excited for her upcoming study abroad trip, and it’s all because of Realingo!

Phase 4: Task Analysis

Although the focus of Realingo is the AR experiences, to increase the chance that users open the APP, I decided to add a translation feature. Lots of mobile APP could not survive several times of use before getting deleted. Therefore, I believe that the translation function would help Realingo stay on users' phones. Another consideration was to couple Realingo with external VR headsets to achieve an immersive learning experiences. Therefore, three features (tasks) of Realingo was developed:

1. Translator (task 1)
2. VR language lessons (task 2)
3. AR language learning game (task 3)

Phase 5: Paper Prototype

The first design iteration was the paper prototype. Here I designed the logo of the APP as well. The first level menu includes lessons, AR/VR experience, progress, and help. The translator feature was included in AR features. Personally, I did not like this structure since I thought users would get confused by what AR/VR experience really means.

We conducted a round of usability testing to 5 university students. The structure was task based, where we gave them three tasks (translating a paragraph of text, taking a lesson, and using VR/AR experiences) and recorded how they completed tthem.

Design Recommendations

• Courses should be designed for different skill levels
• Design more VR scenes
• Add introduction/tutorial section on front page
• Add introduction for VR/AR - not everybody will know what these terms mean and how they’re different from one another
• Keep the simple interface when adding any other features
• Incorporate personal achievements depending on the user’s activity/goals

Phase 6: Mid-fi digital prototype

After the first iteration, I collected valuable feedback from tested users and moved on to the second iteration. In this iteration, to better represent the VR/AR scenarios, I developed a set of mid-fidelity digital prototype.

In this iteration, in addition to addressing the feedback from the previous iteration, I made big design changes to the front page of the APP. The layout was inspired from an instant messaging (IM) APP called WeChat, where four first level sections were placed on the bottom of the page and the open area was to display the current progress. The design was thus cleaner and had a clear structure. 

Design Recommendations

• Change “help” to “profile” and remove account icon on the top right. Put settings icon under “profile” page
• Provide instructions in VR mode to inform users when they should point on screen and when they should point to the air
• Avatar selection page only appears in “new track” and will not appear in existing tracks
• Remove “would you like to continue this track” page
• Separate countries and places in VR mode
• Add more pages in VR mode, including feedback when user point to an object
• Add Exit icon in VR and AR mode

Phase 7: Hi-Fi Digital Prototype

The final iteration addressed the recommendations from the previous testing.  I strongly insisted that we should put translation on the front page to avoid any inconvenience, and the team finally accepted my suggestions. Hence, Realingo reached its final state: three major functions on the front page, translation (with image recognition feature), VR language lessons, and AR language game (Pokemon Go treasure hunt stype)

Translator as a convenient feature is expected to be used often, so I put the button on the center in a distinct color/shape; Lessons and VR experiences were combined into VR lessons focusing on speaking practices; the UI elements were redesigned from stock images into customized contents.

Future Work

• Continue investigation of related products on the market
• What educational VR and AR tools are most successful today, and why?
• What language-learning tools are most successful today, and why?
• Continue implementing improvements to Realingo platform based on user study findings, followed by more user studies
• Review demographics - is our audience of young adults appropriate for our product, or do we need to change our features and/or target population?

You don’t fully understand a language until you know two

Other Contributors:

Melissa Galloway, Luis Gonsalez, Miranda Weipert

Reference

Godwin-Jones, Robert. "Augmented Reality and Language Learning: From Annotated Vocabulary to Place-Based Mobile Games." Emerging Technologies. Michigan State University, Oct. 2016. Web. 17 Oct. 2016.

Holden, Chris, and Julie Sykes. "Mentira: Prototyping language-based locative gameplay." Mobile Media Learning. Springer-Verlag, 2012.

Hsaio, Kuei-Fang, and Habib F. Rashvand. "Body Language and Augmented Reality Learning Environment." Human-centric Computing and Information Sciences. FTRA, 2011. Web. 17 Oct. 2016.

Li, Kuo-Chen, Chung-Wei Tsai, Cheng-Ting Chen, Shein-Yong Cheng, and Jia-Sheng Heh. "The Design of Immersive English Learning Environment Using Augmented Reality." 8th International Conference on Ubi-Media Computing. N.p., 2015. Web. 17 Oct. 2016.

Perry, Bernadette. "Gamifying French Language Learning: A Case Study Examining a Quest-Based, Augmented Reality Learning Tool." Procedia - Social and Behavioral     Sciences. Sciencedirect, 2014. Web. 17 Oct. 2016.