About me

I am a fourth-year Ph.D. student at Toyota Technological Institute at Chicago (TTIC). I am advised by Prof. Karen Livescu. My research interests are in speech and language processing. I am particularly interested in topics related to learning from multimodal data and generative modeling for speech.

Professional Experiences

• Microsoft
• Research Intern, Mentors: Dr. Jinyu Li, (2024.06 - 2024.08).
• Incorporating extra text data for streaming speech translation..
• Meta AI
• Research Intern, Mentors: Dr. Wei-Ning Hsu, (2022.06 - 2022.12).
• Joint language modeling for speech units and text [arxiv].
• TTIC
• Ph.D. student, Advisor: Prof. Karen Livescu, (2021.04 - Present).
• Audio-visual speech enhancement with diffusion models [arxiv, demo].
• Layer-wise Analysis of wav2vec 2.0 [arxiv].
• Montreal Institute for Learning Algorithms (MILA)
• Intern for SpeechBrain project, advisors: Dr. Mirco Ravanelli, Prof. Yoshua Bengio (2020.03 - 2020.08).
• Developing an open source toolkit for end-to-end ASR.
• Speech Processing and Machine Learning Laboratory
• Research assistant, advisors: Prof. Hung-yi Lee and Prof. Lin-shan Lee (2015.09 - 2019.07).
• Unsupervised learning for disentangled speech representations, with a focus on the application of voice conversion [arxiv1, arxiv2].
• Institude of Information Science, Academia Sinica
• Research intern, advisor: Prof. Wei-Yun Ma (2016.07 - 2016.09)
• Name entity recognition with tree-stutured neural networks [link].

Education

• Toyota Technological Institute at Chicago (TTIC)
• Ph.D. in Computer Science (2021.03 - )
• National Taiwan University
• M.S. in Computer Science and Information Engineering (2017.07 - 2020.06)
• National Taiwan University
• B.S. in Computer Science and Information Engineering (2013.09 - 2017.07)

CV

Here is my CV (last update: 2024/10/27).

Publications

Here is my Google scholar.

Awards

• Artificial Intelligence Research Scholarship, Appier (2019.08).
• Advanced Speech Technologies Scholarship (2018.09).
• One of the 12 finalists for the best student paper award in Interspeech 2018 (2018.09).
• Interspeech 2018 Travel Grant (2018.09).
• Artificial Intelligence Research Scholarship, Appier (2018.09).
• Undergraduate research competition 2017 held by CSIE, NTU: First prize (2017.06). [ Poster Link ].
• Undergraduate research competition 2017 held by CSIE, NTU: Appier second prize (2017.06).
• Undergraduate research competition 2017 held by CSIE, NTU: Viscovery third prize (2017.06).
• Undergraduate research competition 2017 held by CSIE, NTU: Leopard Mobile first prize (2017.06).

Talks

• One-shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalization. NTU, Speech lab (2019.12) [ pptx | pdf ].
• One-shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalization. Appier (2019.08) [ pptx | pdf ].
• Unsupervised Voice Conversion by Separately Embedding Speaker and Content Information with Deep Generative Model. Master thesis defense (2019.07) [ pptx | pdf ].
• Multi-target Voice Conversion without Parallel Data by Adversarially Learning Disentangled Audio Representations. Appier (2018.10) [ pptx | pdf ].
• Multi-target Voice Conversion without Parallel Data by Adversarially Learning Disentangled Audio Representations. Interspeech 2018 (2018.09) [ pptx | pdf ].

Teaching

• Teaching Assistant for TTIC 31110 (CMSC 35110): Speech Technologies, 2022 Spring.
  • Instructor: Dr. Karen Livescu.
• Teaching Assistant for Introduction to Digital Speech Processing, NTU CSIE, 2018 Spring.
  • Instructor: Dr. Lin-shan Lee.
• Teaching Assistant for Introduction to Digital Speech Processing, NTU CSIE, 2017 Fall.
  • Instructor: Dr. Lin-shan Lee.
• Teaching Assistant for Machine Learning, NTU EE, 2017 Fall.
  • Instructor: Dr. Hung-yi Lee.
• Teaching Assistant for Applied Deep Learning / Machine Learning and Having It Deep and Structured, NTU CSIE and NTU EE, 2017 Fall.
  • Instructor: Dr. Yun-Nung (Vivian) Chen and Dr. Hung-yi Lee.
• Teaching Assistant for Machine Learning, NTU EE, 2017 Spring.
  • Instructor: Dr. Hung-yi Lee.