ITS Campus, ITS News – The widespread use of smartphones and social media in this globalization era causes the public’s mental condition to become more dynamic. Answering this challenge, a doctoral program graduate from the Department of Electrical Engineering at Institut Teknologi Sepuluh Nopember (ITS), Dr Gede Aditra Pradnyana SKom MKom, designed a depression detection system model based on data sources from social media usage using multimodal Artificial Intelligence (AI).
The man familiarly called Adit explained that suicide cases caused by depression have become a critical humanitarian issue. The fear of expressing life problems directly, whether to psychologists, psychiatrists, or close relatives, remains an obstacle for the majority of people. In fact, some people more frequently vent their anxious feelings and complex problems on social media.
Starting from this problem, this lecturer from Universitas Pendidikan Ganesha proposed a non-intrusive depression detection approach based on digital footprints. Through this approach, the built system can provide initial information quickly, support self-intervention, and function as a complement to conventional clinical assessments. “This approach not only requires physiological sensors but also utilizes multimodal expression patterns as an initial indicator,” he explained.
The developed model is called DeXMAG, which is a combination of Cross-Modal Attention and Adaptive Gated Fusion with the Myers Briggs Type Indicator (MBTI) feature. More clearly, it is a combination of a multimodal deep learning framework personalized by the user’s personality type. Therefore, the system can improve the performance of depression detection from social media data.
Architecturally, the model design starts from the earliest and main stage, namely identifying the text and image modalities uploaded by the user. After the identification process, the modality content is processed by pre-trained RoBERTa and VGG-16 models, and the personality decision by GloVe-BiLSTM. The process completion is done using Weighted Fused Representation, resulting in a detection output of depressed or not depressed.
Furthermore, the obtained results show eight personality traits related to depression indications. These personality traits consist of perceiving, judging, intuition, and thinking. “Not only that, personality traits regarding feelings, such as feeling, introversion, sensing, and extroversion, are also displayed in the form of a radar chart,” added the Universitas Udayana alumnus.
Overall, the detection results are proven by an ablation study showing that each modality contributes to predictive performance. Every linguistic and visual signal provides a significant performance increase compared to a unimodal approach. “Hopefully, this approach can be utilized by many people and can help determine depression vulnerability solely through application features,” he said optimistically.
This research is also in line with the Sustainable Development Goals (SDGs) agenda. Specifically on point 3 regarding Good Health and Well-being and point 9 regarding Industry, Innovation and Infrastructure. Through the integration of artificial intelligence and digital mental health, this innovation contributes to the development of technological solutions that support the psychological well-being of the community sustainably. (ITS Public Relations)
Reporter: Mohammad Fariz Irwansyah