Language learners listen to music and refer dual lyrics that lay emphasis on comprehension rather than vocabulary & provide translations of phrases not words, thus not improving their vocabulary learning drastically.
Smart Indic Lyrics, an interactive web-based Indian language music player oriented towards effective vocabulary learning while maintaining an enjoyable experience. The design of Smart Indic lyrics proved to be 1.5x effective as dual lyrics in a pilot user study.
Feel free to contribute to this project!
Lightning talk at Grace Hopper Celebration of Women in Computing India '16 AND
Poster at India HCI '16
Interviews, Competitive Analysis, Lit review, Sketching, User feedback, HTML, CSS, AngularJS, MongoDb, T-tests, Quantitative and qualitative analysis
WORK SETTING & ROLE
Priya Lakshman & Sonali Tandon,
UX Design Lead & Co-Web developer
June 2016 - Oct 2016
My first step into wonderful world of HCI
This is a self-initiated project born out my passion to learn User Experience and Human Computer Interaction. This project had three phases - UX Design, Development & Pilot User Study. I led the complete UX Design process and was involved in every stage of work. Individually, I conducted user research (literature review, interviews), made design decisions and conducted user feedback on designs. I coded features related to syncing the music & lyrics and vocabulary look ups. My teammate and I were equally involved in competitive analysis and the pilot study.
The cherry on top was presenting this project at 2 conferences - Grace Hopper Celebration Women in Computing India 2016 and India HCI 2016.
MOTIVATION & INITIAL LITERATURE RESEARCH
Music helps in language learning
While volunteering for a school in India, a language teacher told me she encourages her students to listen to music in the language being learnt. I found this interesting and wanted to apply my newly learnt skills from Interaction Design Coursera course to design a real-world project. So, I first started exploring this space by reading research literature about how music helps in language learning.
Language learners face difficulties in vocabulary building & revision while listening to music
I wanted to know if students of the school listen to music in the language being learnt, so I conducted user interviews with 5 language learners learning Hindi. These were intermediate learners who had a basic grasp of comprehension of foreign language and continuously learning new words. The conversations were around listening to songs in the language being learnt and their experiences of building vocabulary. Some common patterns emerged as shown below.
What do current tools offer and don't offer?
My teammate and I then did a competitive analysis to see what current tools offer, a few of which are listed below. Findings were consistent with interviews in that most applications don't support vocabulary building with music. I also found applications that combine music/video and lyrics/ subtitles at the same place for language learning making it more time-effective experience.
OVERARCHING DESIGN DECISIONS
What features would help language learners?
Based on findings from literature review, interviews & competitive analysis, I made the overarching design decisions.
How can I design the lyrics?
To design lyrics, I analyzed how existing native-foreign language lyrics are represented and kept user needs in mind.
ITERATIVE SKETCHING AND FEEDBACK
What do users think of Smart Indic Lyrics?
I sketched paper wireframes of an interactive music player, showed users for their feedback and iterated over the design.
What are the design changes?
DESIGN CHANGE 1
Users preferred seeing only meaning of the word on hover, so I re-designed the hover to have two states - 1) meaning of the word on hover with a "Learn more" to expand it 2) Meaning, contextual usage and part of speech on expanded hover.
DESIGN CHANGE 2
Users wanted to see a picture representation along with vocabulary definition. In addition, I read research which states pictures are recognized more than words thus helping in language learning. Hence I decided to include pictures for nouns and verbs in lookups.
Smart Indic Lyrics : Interactive Music Player for Language Learning
#1 : Music + Lyrics + Vocabulary Learning
All in the same place for a time effective and enjoyable experience
#2 : Lyrics in language being learnt
To focus learner's attention on the language
being learnt, the lyrics are in the foreign
language script by default
#3: Easy & quick vocabulary building
Learners can quickly view word definitions on hover. On clicking "Learn More", learners can understand part of speech, example sentences and see picture representations
#3: Highlight currently played word
The currently played word is highlighted so that learners can quickly correlate hearing and written components.
#4 : Book & Download for Long Term Learning
Learners can bookmark newly learnt words and download them onto their computers for
Web Development of Smart Indic Lyrics
With the aim of building a real world application for language learners and also evaluating our design through a pilot user study, my teammate and I coded Smart Indic Lyrics using HTML, CSS, AngularJS and MongoDb. We first explored online APIs to obtain dictionary definitions for words, however they didn't support all languages and all words. So we obtained a starter online bilingual dictionary, and manually created JSONS for word level translations, sentences and parts of speech. We implemented the system for Hindi & Tamil songs.
PILOT USER STUDY
Does the design work?
Quantitative & Qualitative Comparison of Smart Indic Lyrics Versus Dual Lyrics
We conducted a study with 10 participants, 5 males and 5 females. The participants were all students aged between 11-12 and studying Hindi. They self-reported that they weren’t from a Hindi speaking background. Our tool was evaluated by comparing the number of newly learnt words learnt while listening to songs using Smart Indic Lyrics to that in the dual Hindi-English lyrics. Participants listened to two 4-minute Hindi songs, one on each tool. Half the participants listened to the first song using Smart Indic Lyrics followed by dual lyrics and the other half followed the reverse order. After hearing both songs, a vocabulary test was conducted and a feedback questionnaire was answered.
Vocabulary Quiz : Quantifying number of newly learnt words
Once participants completed listening to songs on both tools, a vocabulary quiz consisting of 20 questions (10 on each song) of 2 types were asked: 1) Definition when given the word 2) Definition given the word in the context of a sentence. Additional 10 questions were asked to test the definition of the word given its picture. This was asked only for words that appeared using Smart Indic Lyrics. Participants could opt to listen to the pronunciation of the word while taking the quiz. We additionally found out whether the participant already knew the meaning of the word before listening to the song or it was newly learnt. This helped in quantifying the number of learnt words correctly knew the meaning of the word before listening to the song or was it newly learnt.
Feedback Questionnaire : Quantifying the experience
Participants gave feedback of their listening and learning experience with each tool on a 5-point Likert scale.
PILOT USER STUDY
Results of the study
Vocabulary Quiz Results
Dual Lyrics VS Smart Indic Lyrics (without pictures)
Users correctly defined 1.5 times as many new words, more questions were correctly answered (t = 2.4097, df = 9, p < 0.0393) and more new words were correctly defined (t = 2.6234, df=9, p= 0.0277) when using Smart Indic Lyrics.
Smart Indic Lyrics VS Smart Indic Lyrics (with pictures)
Users correctly defined 1.6 times as many new words, more questions were correctly answered (t = 1.2910, df = 9, p =0.2289) and more new words were correctly defined (t = 7.3193, df=9, p= 0.0001) when using Smart Indic Lyrics.
Feedback Questionnaire Results
Users considered it easier to use Smart Indic Lyrics. Listeners enjoyed using both tools almost equally & rated their overall understanding of songs as similar in both cases.
Monitoring features used in Smart Indic Lyrics using Code
We monitored user behavior using code in the tool for feature tracking.
What can be done in the future?
We prototyped our Smart Indic Lyrics for Hindi and Tamil songs only. The tool can be extended to other languages if bilingual dictionaries are available. The users of our pilot study were intermediate level language learners only. Further investigation must be performed to understand its effectiveness for beginner and advanced level language learners. Since vocabulary quizzes were conducted immediately after listening to the songs, the results of our study evaluated short- term vocabulary retention. Although our tool allows learners to store words with complete vocabulary annotations, further evaluation is needed to judge how helpful it proved to be in long-term learning.
Several failures makes perfect
After multiple failed attempts, I learnt how to conduct interviews, sketch my ideas on paper, iterate my design based on user feedback and conduct user study. As it was hard to emulate an experience related to music on paper, I used hi fidelity prototyping for conducting user testing. Instead, I could have done medium-fi testing first and gathered feedback. I also learnt that during user testing, my users were not comfortable using a laptop without a mouse, something which I should have found out during user interviews.
Overall, this independent project was a tremendous learning experience for me. While presenting this project at two major conferences, I met wonderful people in the field of HCI and learnt that HCI was much more than imagined - it had data visualization, social computing, designing for tangible interfaces and so on. This was when I decided to pursue a career in this field, and applied for a Master's program to formally learn it.