San Francisco Bay Area

ಸmart इndic લyrics

 Interactive music player to improve language learning

Presented talk at Grace Hopper Celebration India 2016 & poster at India HCI 2016

Long Story Short  ​

Problem

Intermediate level Indian language learners listen to music and refer dual lyrics that lay emphasis on comprehension rather than vocabulary & provide translations of phrases not words, thus not improving their language learning drastically.

 

Skills developed

Interviews, User study & evaluation using T-tests,

Web development

Duration

June 2016 - August 2016

Solution

Smart Indic Lyrics, an interactive web-based Indian language music player oriented towards effective vocabulary learning while maintaining an enjoyable experience. A demo can be viewed here.

 

Tools 

HTML, CSS, Bootstrap, AngularJs

Work Setting

Team of 2 

The Long Story

 

Problem Statement 

Currently, if a language learner is listening to a song and doesn’t recognize a word, the learner needs to listen to the word carefully, pause the song, and look it up in the dictionary. This process is time-consuming and reduces the enjoy-ability of the song. Alternatively, language learners use dual lyrics, which display lyrics in both foreign language and the viewer’s native language. These lay emphasis on comprehension rather than vocabulary learning as they provide translation of phrases and not words.

Dual Hindi-English lyrics

Bookmarked words which can be downloaded and saved to the local file system.

The Design Process 

Exploratory Interviews 

Informal interviews were conducted with a Hindi teacher and 3 students enrolled in Hindi language classes. We asked students the following questions : 

  • Do you listen to music in Hindi?

  • If so, what do you do if you come across a word you don't recognize in a song?

  • Do you recognize new words in a song?

  • Do you note down newly learnt words anywhere?

  • Do you revise the words, if noted?

  •  How often do you revise newly learnt words?

The teacher reported that they encourage students to incorporate music into language learning and students self-reported that they often listen to songs outside class. Students seldom looked up words they didn’t know as it was time consuming. Many indicated they weren’t meticulous in collating newly learnt words and revisiting them.

Design & Prototyping 

Our features were designed to address the issues our interviewees were facing. Smart Indic Lyrics, an interactive web-based Indian language music player oriented towards effective vocabulary learning while maintaining an enjoyable experience. We prototyped our solution for Hindi & Tamil songs.

  • It displays lyrics in the language being learnt 

  • To focus the learners’ attention on the currently played word and its pronunciation, a highlight feature is implemented. Learners can view vocabulary definitions of words by hovering over them.

  • For further understanding, contextual usage of words in sentences and pictorial representation of nouns and verbs are provided

  • The learner can bookmark and download newly learnt words with complete information for long-term learning.

  • If word wise translations are not sufficient to understand the current phrase of the song, a complete translation can be viewed. 

.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

User Study

We conducted a study with 10 participants, 5 males and 5 females. The participants were all students aged between 11-12 and studying Hindi. They self-reported that they weren’t from a Hindi speaking background. Our tool was evaluated by comparing the number of vocabulary words learnt while listening to songs using Smart Indic Lyrics to that in the dual Hindi-English lyrics. Participants listened to two 4-minute Hindi songs, one on each tool. Half the participants listened to the first song using Smart Indic Lyrics followed by dual lyrics and the other half followed the reverse order. After hearing both songs, a vocabulary test was conducted and a feedback questionnaire was answered.

 

 

 

 

 

 

Vocabulary Quiz

Once participants completed listening to songs on both tools, a vocabulary quiz consisting of 20 questions (10 on each song) of 2 types were asked: 1) Definition when given the word 2) Definition given the word in the context of a sentence. Additional 10 questions were asked to test the definition of the word given its picture. This was asked only for words that appeared using Smart Indic Lyrics. Participants could opt to listen to the pronunciation of the word while taking the quiz. We additionally found out whether the participant already knew the meaning of the word before listening to the song or it was newly learnt. This helped in quantifying the number of learnt words correctly knew the meaning of the word before listening to the song or was it newly learnt.

Feedback Questionnaire

Participants gave feedback of their listening and learning experience with each tool on a 5-point Likert scale.

Vocabulary Quiz Results 

  • Dual Lyrics VS Smart Indic Lyrics (without pictures)

Users correctly defined 1.5 times as many new words, more questions were correctly answered (t = 2.4097, df = 9, p < 0.0393) and more new words were correctly defined (t = 2.6234, df=9, p= 0.0277) when using Smart Indic Lyrics.

  • Smart Indic Lyrics  VS Smart Indic Lyrics (with pictures)

Users correctly defined 1.5 times as many new words, more questions were correctly answered (t = 2.4097, df = 9, p < 0.0393) and more new words were correctly defined (t = 2.6234, df=9, p= 0.0277) when using Smart Indic Lyrics.

Feedback Questionnaire Results

Users correctly defined 1.5 times as many new words, more questions were correctly answered (t = 2.4097, df = 9, p < 0.0393) and more new words were correctly defined (t = 2.6234, df=9, p= 0.0277) when using Smart Indic Lyrics.

 

Feature Usage During Study

 

 

 

 

 

 

 

 

 

 

 

 

 

Demo

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Future Work 

We prototyped our Smart Indic Lyrics for Hindi and Tamil songs only. The tool can be extended to other languages if bilingual dictionaries are available. The users of our pilot study were intermediate level language learners only. Further investigation must be performed to understand its effectiveness for beginner and advanced level language learners. Since vocabulary quizzes were conducted immediately after listening to the songs, the results of our study evaluated short- term vocabulary retention. Although our tool allows learners to store words with complete vocabulary annotations, further evaluation is needed to judge how helpful it proved to be in long-term learning.

Smart Indic Lyrics Interface

English tab displays phrase-level translation of song in

English language.

Dictionary definition and ability to star or bookmark word on hover.

Contextual usage of word in sentences & part of speech on expanded hover.

Bookmark and download newly for long-term learning