{"id":245712,"date":"2017-05-16T12:38:16","date_gmt":"2017-05-16T16:38:16","guid":{"rendered":"http:\/\/www.rochester.edu\/newscenter\/?p=245712"},"modified":"2017-05-18T08:49:00","modified_gmt":"2017-05-18T12:49:00","slug":"automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712","status":"publish","type":"post","link":"https:\/\/www.rochester.edu\/newscenter\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\/","title":{"rendered":"New system displays song lyrics in real time, multiple languages"},"content":{"rendered":"

A recent choral concert reflecting on the past with the theme \u201cMemories,\u201d also featured the debut of a new technological advancement in music.<\/p>\n

The Chinese Choral Society of Rochester<\/a>\u00a0<\/strong>presented a concert that included Chinese and English art and folk songs and also marked the launch of an automatic lyrics display system developed by Zhiyao Duan, an assistant professor of electrical and computer engineering<\/a><\/strong> at the University of Rochester, and his PhD student, Bochen Li. The system is able to follow a performance and display lyrics on a large screen in real time.<\/p>\n

During the May concert, the system displayed the lyrics in both Chinese and English to a multicultural audience composed of people with a variety of native languages.<\/p>\n

\u201cMusical performance is all about communication between musicians and the audience,\u201d Duan says. \u201cThis system helps musicians convey ideas\u2014in this case the lyrics\u2014to the audience and helps the audience better appreciate the performance.\u201d<\/p>\n

Duan sings tenor with the Chinese Choral Society and brainstormed with Youti Kuo, secretary of the choral society, and Ming-Lun Lee, an assistant professor of audio and music engineering at the University of Rochester and the director of the choral society, about ways he could also contribute his knowledge as a researcher in artificial intelligence and music.<\/p>\n

\"laptop
Rather than relying on someone to manually type song lyrics during a live performance, professor Zhiyao Duan’s system uses an algorithm to compare a reference recording with a live performance and display the matching song lyrics automatically. (University photo \/ Yichi Zhang)<\/figcaption><\/figure>\n

The choir performs in various languages, and previously, in order to display lyrics in real time, a person very familiar with the songs would have to manually type the words during the live performance. Duan\u2019s innovation allows an audience to view the lyrics at the exact moment they are performed, without the need for manual lyric input during the performance.<\/p>\n

\u201cIt is very important to have the audience understand the lyrics in foreign languages,\u201d says Lee. \u201cWith the help of the automatic lyrics displaying system, we can train the system before the concert and avoid human errors.\u201d<\/p>\n

Duan\u2019s system uses a dynamic time warping algorithm, which reconciles differences between two sequences that may vary in speed. For the system to work, it needs a reference recording to compare with the live performance. The choral society taped a reference recording during a rehearsal and pre-aligned the lyrics to this recording. During the live performance, the algorithm aligned the live performance with the reference recording and displayed the lyrics in real time.<\/p>\n

Although manual work is still required to add lyrics to the reference recording prior to the concert, it can be completed in a much more flexible fashion: it can be done any time before the concert and allows for revisions. In addition, once the lyrics alignment is completed, it can be used as a reference for any live performance of the same piece.<\/p>\n

\u201cWe hope this will soon be available to choirs worldwide, either for commercial use or for free,\u201d Duan says. \u201cWe are working on building a system and a repertoire that are scalable.\u201d<\/p>\n

In addition to the debut of the lyrics system, the concert featured the US premiere of the suite\u00a0A Journey Named Life\u00a0<\/em>for chorus, flute, and piano, composed by Ching-Mei Lin \u201906 (MM).<\/p>\n","protected":false},"excerpt":{"rendered":"

Zhiyao Duan, an assistant professor of electrical and computer engineering who also sings in the Chinese Choral Society of Rochester, lent his engineering skills to an innovation that provides the audience with live lyrics and translations. <\/p>\n","protected":false},"author":912,"featured_media":245942,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[116],"tags":[31972,19382,18632,30532],"class_list":["post-245712","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-sci-tech","tag-audio-and-music-engineering","tag-department-of-electrical-and-computer-engineering","tag-hajim-school-of-engineering-and-applied-sciences","tag-zhiyao-duan"],"acf":[],"yoast_head":"\nNew system displays song lyrics in real time, multiple languages<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.rochester.edu\/newscenter\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"New system displays song lyrics in real time, multiple languages\" \/>\n<meta property=\"og:description\" content=\"Zhiyao Duan, an assistant professor of electrical and computer engineering who also sings in the Chinese Choral Society of Rochester, lent his engineering skills to an innovation that provides the audience with live lyrics and translations.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.rochester.edu\/newscenter\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\/\" \/>\n<meta property=\"og:site_name\" content=\"News Center\" \/>\n<meta property=\"article:published_time\" content=\"2017-05-16T16:38:16+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2017-05-18T12:49:00+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.rochester.edu\/newscenter\/wp-content\/uploads\/2017\/05\/fea-Duan-lyrics.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1000\" \/>\n\t<meta property=\"og:image:height\" content=\"600\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Lindsey Valich\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Lindsey Valich\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\\\/\"},\"author\":{\"name\":\"Lindsey Valich\",\"@id\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/#\\\/schema\\\/person\\\/fcd7d29a5b8e855924bf73b764dcd827\"},\"headline\":\"New system displays song lyrics in real time, multiple languages\",\"datePublished\":\"2017-05-16T16:38:16+00:00\",\"dateModified\":\"2017-05-18T12:49:00+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\\\/\"},\"wordCount\":570,\"image\":{\"@id\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/wp-content\\\/uploads\\\/2017\\\/05\\\/fea-Duan-lyrics.jpg\",\"keywords\":[\"Audio and Music Engineering\",\"Department of Electrical and Computer Engineering\",\"Hajim School of Engineering and Applied Sciences\",\"Zhiyao Duan\"],\"articleSection\":[\"Science & Technology\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\\\/\",\"url\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\\\/\",\"name\":\"New system displays song lyrics in real time, multiple languages\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/wp-content\\\/uploads\\\/2017\\\/05\\\/fea-Duan-lyrics.jpg\",\"datePublished\":\"2017-05-16T16:38:16+00:00\",\"dateModified\":\"2017-05-18T12:49:00+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/#\\\/schema\\\/person\\\/fcd7d29a5b8e855924bf73b764dcd827\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/wp-content\\\/uploads\\\/2017\\\/05\\\/fea-Duan-lyrics.jpg\",\"contentUrl\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/wp-content\\\/uploads\\\/2017\\\/05\\\/fea-Duan-lyrics.jpg\",\"width\":1000,\"height\":600,\"caption\":\"The Chinese Choral Society of Rochester performs, with lyrics displayed in real-time in both English and Chinese. (University photo \\\/ Yichi Zhang )\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"New system displays song lyrics in real time, multiple languages\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/#website\",\"url\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/\",\"name\":\"News Center\",\"description\":\"University of Rochester\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/#\\\/schema\\\/person\\\/fcd7d29a5b8e855924bf73b764dcd827\",\"name\":\"Lindsey Valich\",\"url\":\"https:\\\/\\\/www.rochester.edu\\\/newscenter\\\/author\\\/lvalich\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"New system displays song lyrics in real time, multiple languages","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.rochester.edu\/newscenter\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\/","og_locale":"en_US","og_type":"article","og_title":"New system displays song lyrics in real time, multiple languages","og_description":"Zhiyao Duan, an assistant professor of electrical and computer engineering who also sings in the Chinese Choral Society of Rochester, lent his engineering skills to an innovation that provides the audience with live lyrics and translations.","og_url":"https:\/\/www.rochester.edu\/newscenter\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\/","og_site_name":"News Center","article_published_time":"2017-05-16T16:38:16+00:00","article_modified_time":"2017-05-18T12:49:00+00:00","og_image":[{"url":"https:\/\/www.rochester.edu\/newscenter\/wp-content\/uploads\/2017\/05\/fea-Duan-lyrics.jpg","width":1000,"height":600,"type":"image\/jpeg"}],"author":"Lindsey Valich","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Lindsey Valich","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.rochester.edu\/newscenter\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\/#article","isPartOf":{"@id":"https:\/\/www.rochester.edu\/newscenter\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\/"},"author":{"name":"Lindsey Valich","@id":"https:\/\/www.rochester.edu\/newscenter\/#\/schema\/person\/fcd7d29a5b8e855924bf73b764dcd827"},"headline":"New system displays song lyrics in real time, multiple languages","datePublished":"2017-05-16T16:38:16+00:00","dateModified":"2017-05-18T12:49:00+00:00","mainEntityOfPage":{"@id":"https:\/\/www.rochester.edu\/newscenter\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\/"},"wordCount":570,"image":{"@id":"https:\/\/www.rochester.edu\/newscenter\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\/#primaryimage"},"thumbnailUrl":"https:\/\/www.rochester.edu\/newscenter\/wp-content\/uploads\/2017\/05\/fea-Duan-lyrics.jpg","keywords":["Audio and Music Engineering","Department of Electrical and Computer Engineering","Hajim School of Engineering and Applied Sciences","Zhiyao Duan"],"articleSection":["Science & Technology"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.rochester.edu\/newscenter\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\/","url":"https:\/\/www.rochester.edu\/newscenter\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\/","name":"New system displays song lyrics in real time, multiple languages","isPartOf":{"@id":"https:\/\/www.rochester.edu\/newscenter\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.rochester.edu\/newscenter\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\/#primaryimage"},"image":{"@id":"https:\/\/www.rochester.edu\/newscenter\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\/#primaryimage"},"thumbnailUrl":"https:\/\/www.rochester.edu\/newscenter\/wp-content\/uploads\/2017\/05\/fea-Duan-lyrics.jpg","datePublished":"2017-05-16T16:38:16+00:00","dateModified":"2017-05-18T12:49:00+00:00","author":{"@id":"https:\/\/www.rochester.edu\/newscenter\/#\/schema\/person\/fcd7d29a5b8e855924bf73b764dcd827"},"breadcrumb":{"@id":"https:\/\/www.rochester.edu\/newscenter\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.rochester.edu\/newscenter\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.rochester.edu\/newscenter\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\/#primaryimage","url":"https:\/\/www.rochester.edu\/newscenter\/wp-content\/uploads\/2017\/05\/fea-Duan-lyrics.jpg","contentUrl":"https:\/\/www.rochester.edu\/newscenter\/wp-content\/uploads\/2017\/05\/fea-Duan-lyrics.jpg","width":1000,"height":600,"caption":"The Chinese Choral Society of Rochester performs, with lyrics displayed in real-time in both English and Chinese. (University photo \/ Yichi Zhang )"},{"@type":"BreadcrumbList","@id":"https:\/\/www.rochester.edu\/newscenter\/automatic-lyric-display-system-projects-song-lyrics-in-real-time-245712\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.rochester.edu\/newscenter\/"},{"@type":"ListItem","position":2,"name":"New system displays song lyrics in real time, multiple languages"}]},{"@type":"WebSite","@id":"https:\/\/www.rochester.edu\/newscenter\/#website","url":"https:\/\/www.rochester.edu\/newscenter\/","name":"News Center","description":"University of Rochester","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.rochester.edu\/newscenter\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.rochester.edu\/newscenter\/#\/schema\/person\/fcd7d29a5b8e855924bf73b764dcd827","name":"Lindsey Valich","url":"https:\/\/www.rochester.edu\/newscenter\/author\/lvalich\/"}]}},"_links":{"self":[{"href":"https:\/\/www.rochester.edu\/newscenter\/wp-json\/wp\/v2\/posts\/245712","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.rochester.edu\/newscenter\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.rochester.edu\/newscenter\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.rochester.edu\/newscenter\/wp-json\/wp\/v2\/users\/912"}],"replies":[{"embeddable":true,"href":"https:\/\/www.rochester.edu\/newscenter\/wp-json\/wp\/v2\/comments?post=245712"}],"version-history":[{"count":5,"href":"https:\/\/www.rochester.edu\/newscenter\/wp-json\/wp\/v2\/posts\/245712\/revisions"}],"predecessor-version":[{"id":247262,"href":"https:\/\/www.rochester.edu\/newscenter\/wp-json\/wp\/v2\/posts\/245712\/revisions\/247262"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.rochester.edu\/newscenter\/wp-json\/wp\/v2\/media\/245942"}],"wp:attachment":[{"href":"https:\/\/www.rochester.edu\/newscenter\/wp-json\/wp\/v2\/media?parent=245712"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.rochester.edu\/newscenter\/wp-json\/wp\/v2\/categories?post=245712"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.rochester.edu\/newscenter\/wp-json\/wp\/v2\/tags?post=245712"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}