Research

Research Theme

My research goals are aimed at extracting knowledge from acoustic information (aka acoustic features). Examples of this theme include: speech emotion recognition, abnormal sound detection, and audio classification. Moreover, the research can be extended to vibration signals. My approaches to achieve these goals are defined by: (1) data-driven approach (instead of physical modeling), (2) focus on practical implementation (not necessarily following human mechanisms), and robustness (how stable/consistent the model is given any perturbation instead of correctness). For me, science should be evidence-based, implementable, and consistent. My research is result-oriented instead of process-oriented. This doesn’t mean that the process (physical phenomena, modeling, math, and algorithms) is not important. If we understand the process very well, the solution may appear by itself. Still, there must be a reason (rationale) for doing such research. Then, I judge my research mainly based on the results. My research contributes to developing technologies to solve issues in Society 5.0 (What is Society 5.0? Read here in Indonesian language).

Offered Research Themes

The following are past research themes that I offer:

1. Speech emotion recognition using multilayer perceptron with CCC loss, dataset: IEMOCAP
2. Indonesian speech recognition using Wav2Vec2/Hubert/WavLM/UniSpeech-SAT, etc.
3. Toward universal acoustic features for multi-corpus speech emotion recognition, 30+ datasets.
   4. Predicting Alzheimer’s disease using speech analysis.
4. Development of Calfem-Python
5. Development of Vibration Toolbox
   7. Abnormal sound detection for predictive maintenance (the method is from you/your idea), dataset: DCASE
6. Indonesian emotional speech synthesis using FastSpeech
   9. COVID-19 diagnosis using COUGH sound with deep learning
7. COVID-19 diagnosis using SPEECH sound with deep learning, dataset: ComParE CSS 2021
8. Predicting pathological voice disorders with speech processing techniques, dataset: SVD, Voiced, HUPA
9. Detection of emotion intensity in non-speech sounds (laughter, crying, etc.)
10. Detecting/predicting stuttering (bahasa: gagap) in speech with machine learning
11. Predicting the intensities of seven self-reported emotions (Adoration, Amusement, Anxiety, Disgust, Empathic Pain, Fear, Surprise) from user-generated reactions to emotionally evocative videos
12. Few-shot learning on acoustic data to capture 10 dimensions of emotions reliably perceived in distinct vocal bursts: Awe, Excitement, Amusement, Awkwardness, Fear, Horror, Distress, Triumph, Sadness, and Surprise
13. Multimodal learning (audio+video+text) to capture 10 dimensions of emotions reliably perceived in distinct vocal bursts: Awe, Excitement, Amusement, Awkwardness, Fear, Horror, Distress, Triumph, Sadness, and Surprise
14. Inferring self-reported emotions from multimodal expression, using multi-output regression to predict fine-grained self-report annotations of seven ‘in-the-wild’ emotional experiences

For undergraduate level, I will try to provide the baseline method, and you will improve the results using your proposed method.

Other Topics/Themes

Read my papers. Usually, I write down the remaining tasks for future work in that topic.

For master’s level, you can also propose your research theme. Contact me by email for details.

Typical Timeline

Green: work hardest, play hard phases; Blue: work harder phases, play hard phases; Yellow: work hard, play harder phases

The timeline is in a yearly basis; it can be adapted for undergraduate students (S1, last year); master (S2, 2 years) and PhD levels (S3, three years of research from the beginning).

Style Guide

An example repository for undergraduate final projects can be seen in this guide.

Teaching Policy

There is no specific requirements for master; however, for doctoral students, I required them to have minimum a publication, at least one that hosted in IEEE Xplore or equivalent. A strong background in signal processing and machine learning is plus for student candidates. I prefer to supervise students who are highly motivated and passionate about research and previously (or willingly) have use either Nkululeko or Speechain toolkits. Master students may work on my given research themes but PhD students should proposed their theme independently. I will provide guidance and support. Regular discussions and meetings are keys to discuss progress, challenges, and next steps. Students are encouraged to actively read relevant literature, do experiment with different approaches, and critically analyze their results. While research is important, health is also a priority, both physically and mentally. Ultimately, I aim to help students develop their research skills and contribute meaningfully to their field of study. —

Contact email: bagustris[at]outlook.com