Optimization of Signature Language Tracking Objects Using GMM Models and Kalman Filters Including ROI
DOI:
https://doi.org/10.62951/ijeemcs.v1i3.7Keywords:
Sign Language, Kalman Filter, Gaussian Mixture Model, Region of Interest, Hand DetectionAbstract
Sign language recognition is one of the areas of image recognition and image processing technology that is developing rapidly in human-computer interaction. This technology really helps the deaf and speech impaired in communicating with non-disabled people. This research aims to examine the optimization of an object tracking system in sign language using the Gaussian Mixture Model (GMM) and Kalman Filter by including the Region of Interest (ROI). The proposed system consists of three main components, namely hand detection, object extraction, and classification. Hand detection is done using the Kalman Filter to track hand movements accurately. Next, Region of Interest (ROI) features, such as shape, direction and movement features, are extracted from the detected part of the hand. These features are fed into a Gaussian Mixture Model (GMM) classifier, which can recognize sign language based on the extracted features. With the combination of GMM and Kalman Filter in this research, it can increase accuracy in object tracking, reduce interference from the background, and ensure the tracking focus remains on important objects. The dataset used is in the form os SIBI alphabet symbols, namely A-Z with the amount of data for each class, namely 620 images. Based on the research result, model testing using GMM, Kalman Filter and ROI produces higher accuracy of 99%, while model testing using GMM and ROI produces accuracy of 90%.
References
Peling, I. B. A., Ariawan, I. M. P. A., & Subiksa, G. B. (2024). Deteksi Bahasa Isyarat Menggunakan Tensorflow Lite dan American Sign Language (ASL). Jurnal Krisnadana, 3(2), 90–100. https://doi.org/10.58982/krisnadana.v3i2.534
Ardiansyah, A. R., Nur’azizan, A. H., & Fernandis, R. (2024). Implementasi Deteksi Bahasa Isyarat Tangan Menggunakan OpenCV dan MediaPipe. Stain. (Seminar Nasional Teknologi Sains), 3(1), 331–337.
Sani, A., & Rahmadinni, S. (2022). Deteksi Gestur Tangan Berbasis Pengolahan Citra. Jurnal Rekayasa Elektrik, 18(2), 115–124. https://doi.org/10.17529/jre.v18i2.25147
Thira, I. J., Riana, D., Ilhami, A. N., Dwinanda, B. R. S., & Choerunisya, H. (2023). Pengenalan Alfabet Sistem Isyarat Bahasa Indonesia (SIBI) Menggunakan Convolutional Neural Network. Jurnal Algoritm, 20(2), 421–432. https://doi.org/10.33364/algoritma/v.20-2.1480
Widya Agata, A., Saputra, W. S. J., & Aji Putra, C. (2024). Pengenalan Bahasa Isyarat Indonesia (Bisindo) Menggunakan Algoritma Scale Invariant Feature Transform (Sift) Dan Convolutional Neural Network (Cnn). JATI (Jurnal Mahasiswa Teknik Informatika), 8(1), 1054–1061. https://doi.org/10.36040/jati.v8i1.8917
Putri, H. M., Fadlisyah, F., & Fuadi, W. (2022). Pendeteksian Bahasa Isyarat Indonesia Secara Real-Time Menggunakan Long Short-Term Memory (Lstm). Jurnal Teknologi Terapan Sains 4.0, 3(1), 663. https://doi.org/10.29103/tts.v3i1.6853
Amalia, N., #1, H., Arnia, F., #3, R. M., Khamdan, M., & #4, M. (2022). Pengenalan Gerakan Isyarat Bahasa Indonesia Menggunakan Algoritma Surf Dan K-Nearest Neighbor. Jurnal Komputer, Informasi Teknologi dan Elektro, 7(1), 2022. https://jurnal.unsyiah.ac.id/kitektro/article/view/23262
Gustsa, A. H., & Permadi, G. S. (2023). Sistem Deteksi Bahasa Isyarat Secara Realtime Dengan Tensorflow Object Detection dan Python Menggunakan Metode Convolutional Neural Network. Inovasi: Jurnal Ilmiah Inovasi. https://ejournal.unhasy.ac.id/index.php/inovate/article/view/4116
Iskandar, D. M., Yel, M. B., & Sitohang, A. (2022). Detection of the Deaf Signal Language Using the Single Shot Detection (SSD) Method, *4*(1), 215–222.
Muzami, A., et al. (2016). Aplikasi Identifikasi Citra Telur Ayam Omega-3 Dengan Metode Segmentasi Region Of Interest Berbasis Android, *4*(2), 380–388.
Marpaung, F., Aulia, F., & Nabila, R. C. (2022). Computer Vision Dan Pengolahan Citra Digital.
Suharjito, Wiryana, F., Kusuma, G. P., & Zahra, A. (2018). Feature Extraction Methods in Sign Language Recognition System: A Literature Review. *1st 2018 Indonesian Association for Pattern Recognition International Conference (INAPR) - Proceedings*, 11–15. https://doi.org/10.1109/INAPR.2018.8626857
Nafiudin, I. A., Hidayat, R. T., Putri, A. M., & Maulana, A. R. (2021). Deteksi Jumlah Kendaraan Dengan Algoritma Gaussian Mixture Model Di Area Jalan Raya. Method. Jurnal Teknik Informatika dan Sistem Informasi, 7(1), 37–44. https://doi.org/10.46880/mtk.v7i1.258
Liu, C., Li, Z., Zhang, C., Yan, Y., & Zhang, R. (2019). An Improved hand tracking algorithm for Chinese sign language recognition. Proceedings of the 2019 IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), 229–232. https://doi.org/10.1109/ITNEC.2019.8729147
Sholehurrohman, R., Habibi, M. R., & Ilman, I. S. (2023). Analisis Metode Kalman Filter, Particle Filter dan Correlation Filter Untuk Pelacakan Objek. Komputika, 12(2), 141–148. https://doi.org/10.34010/komputika.v12i2.9567
Sumarsono, A. S., & Sumarsono, A. S. (2023). Optimasi Klasifikasi Deteksi Senjata Api Pada Animasi Black Lagoon Dengan Algoritma Deepsort Dan Kalman Filter. Jurnal Indonesia Manajemen Informatika dan Komunikasi, 4(3), 1835–1843. https://doi.org/10.35870/jimik.v4i3.424
Pratomo, A. H., Kaswidjanti, W., & Korespondensi, P. (2020). Implementasi Algoritma Region of Interest (ROI) untuk Meningkatkan Performa Algoritma Deteksi dan Klasifikasi. Jurnal Teknologi Informasi dan Ilmu Komputer, 155–162. https://doi.org/10.25126/jtiik.202071718
Shivashankara, S., & Srinath, S. (2019). American sign language video hand gestures recognition using deep neural networks. International Journal of Engineering and Advanced Technology, 8(5), 2742–2751.
Fatmi, R., Rashad, S., Integlia, R., & Hutchison, G. (2017). American Sign Language Recognition using Hidden Markov Models and Wearable Motion Sensors. Transactions on Machine Learning and Data Mining, 10(2), 41–55.
Deep, A., Litoriya, A., Ingole, A., Asare, V., Bhole, S. M., & Pathak, S. (2022). Realtime Sign Language Detection and Recognition. 2022 2nd Asian Conference on Innovation Technology (ASIANCON). https://doi.org/10.1109/ASIANCON55314.2022.9908995
Chaitra, P., Poojitha, B., Nikitha, G. V., & Neelima, N. (2024). Real-Time Gesture and Sentence Level Sign Language Translator. 2024 4th International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT). https://doi.org/10.1109/ICAECT60202.2024.10469646
Surya, B., Suresh Krishna, N. V., Sankarreddy, A. S., Prudhvi, B. V., Neeraj, P., & Deepthi, V. H. (2023). An Efficient Real-Time Indian Sign Language (ISL) Detection using Deep Learning. Proceedings of the 7th International Conference on Intelligent Computing and Control Systems (ICICCS), 430–435. https://doi.org/10.1109/ICICCS56967.2023.10142596
Kapoor, A., Yadav, R., Jangra, V., Mishra, A., & Singh, P. (2023). Sign Language Predictor System Using Human Activity Recognition. Proceedings - International Conference on Technology Advances in Computer Science (ICTACS), 301–304. https://doi.org/10.1109/ICTACS59847.2023.10390112
Sonsare, P., Gupta, D., Sayyed, J., Nandha, P., & Laira, S. (2024). Sign Language to Text Conversion Using Deep Learning Techniques. 2024 2nd International Conference on Computing, Communication and Control (IC4). https://doi.org/10.1109/IC457434.2024.10486568
Gunawan, H., Suharjito, & Pratama, D. (2021). Comparison of Gaussian Hidden Markov Model and Convolutional Neural Network in Sign Language Recognition System. Proceedings of the 2021 1st International Conference on Computer Science and Artificial Intelligence (ICCSAI), 5–10. https://doi.org/10.1109/ICCSAI53272.2021.9609718
Ajay, J., Sumathi, R., Arjun, K., Durga Hemanth, B., & Nihal Saneen, K. (2023). Analyses of Machine Learning Techniques for Sign Language to Text conversion for Speech Impaired. 2023 International Conference on Computer Communication and Informatics (ICCCI). https://doi.org/10.1109/ICCCI56745.2023.10128515
Amaliya, S., Handayani, A. N., Akbar, M. I., Herwanto, H. W., Fukuda, O., & Kurniawan, W. C. (2021). Study on Hand Keypoint Framework for Sign Language Recognition. 7th International Conference on Electrical, Electronics and Information Engineering (ICEEIE). https://doi.org/10.1109/ICEEIE52663.2021.9616851
Bhavana, N., & Shenoy, G. S. (2023). Empowering Communication: Harnessing CNN and Mediapipe for Sign Language Interpretation. International Conference on Recent Advances in Science and Engineering Technology (ICRASET). https://doi.org/10.1109/ICRASET59632.2023.10420395
Abdulhamied, R. M., Nasr, M. M., & Abdulkader, S. N. (2023). Real-time recognition of American sign language using long-short term memory neural network and hand detection. Indonesian Journal of Electrical Engineering and Computer Science, 30(1), 545–556. https://doi.org/10.11591/ijeecs.v30.i1.pp545-556
Tung, N. (2021). Vietnamese sign language detection using Mediapipe. Proceedings, 162–165.
Luna-Jiménez, C., Gil-Martín, M., Kleinlein, R., San-Segundo, R., & Fernández-Martínez, F. (2023). Interpreting Sign Language Recognition using Transformers and MediaPipe Landmarks. ACM International Conference Proceeding Series, 373–377. https://doi.org/10.1145/3577190.3614143
Alyami, S., Luqman, H., & Hammoudeh, M. (2024). Isolated Arabic Sign Language Recognition Using a Transformer-based Model and Landmark Keypoints. ACM Transactions on Asian and Low-Resource Language Information Processing, 23(1). https://doi.org/10.1145/3584984
Shah, F., Shah, M. S., Akram, W., Manzoor, A., Mahmoud, R. O., & Abdelminaam, D. S. (2021). Sign Language Recognition Using Multiple Kernel Learning: A Case Study of Pakistan Sign Language. IEEE Access, 9. https://doi.org/10.1109/ACCESS.2021.3077386
Saunders, B., Cihan, N., & Richard, C. (2021). Continuous 3D Multi-Channel Sign Language Production via Progressive Transformers and Mixture Density Networks. International Journal of Computer Vision, 129(7), 2113–2135. https://doi.org/10.1007/s11263-021-01457-9
Mokhtar, H. (2021). Automatic system for Arabic Sign Language Recognition and. International Journal of Advanced Trends in Computer Science and Engineering. https://doi.org/10.30534/ijatcse/2020/37952020
Language, J. S. (2020). Recognition of Non-Manual Content in Continuous Japanese Sign Language. Sensors, 20(19), 5621. https://doi.org/10.3390/s20195621
Sutabri, T. (2012). Analisa Sistem Informasi (Edisi Pertama). Andi Yogyakarta.
Rinalduzzi, M., et al. (2021). Gesture Recognition of Sign Language Alphabet Using a Magnetic Positioning System. Applied Sciences.
Moape, T. G., Muzambi, A., & Chimbo, B. (2024). Convolutional Neural Network Approach for South African Sign Language Recognition and Translation. *2024 Conference on Information Communications Technology and Society (ICTAS) - Proceedings*, 101–106. https://doi.org/10.1109/ICTAS59620.2024.10507130
Gangadia, D. (2020). Indian Sign Language Interpretation and Sentence Formation. Proceedings, 71–76.
Faiza Ansari, I., Ansari, N., Khanche, M. (2023). Real Time Sign Language to Text Conversion. International Journal of Research in Applied Science and Engineering Technology, 11(April).
Mejía-peréz, K., Terven, J., García-ramírez, T., & Ramírez-pedraza, A. (2022). Automatic Recognition of Mexican Sign Language Using a Depth Camera and Recurrent Neural Networks. Applied Sciences, 1–16.
Himmah, D., et al. (2019). Analisis Metode Pengurangan Latar Belakang Berbasis Mixture of Gaussian pada Kasus Deteksi Objek. Jurnal Teknologi Informasi dan Ilmu Komputer, 6(2), 3268–3276.
Apendi, S., Setianingsih, C., & Paryasto, M. W. (2023). Deteksi Bahasa Isyarat Sistem Isyarat Bahasa Indonesia Menggunakan Metode Single Shot Multibox Detector. eProceedings of Engineering, 10(1), 249–255. https://openlibrarypublications.telkomuniversity.ac.id/index.php/engineering/article/view/19322
Vita Via, Y., Saputra, W. S. J., Fachrurrozi, M. I., Puspaningrum, E. Y., Tri Anggraeny, F., & Nudin, S. R. (2023). Object Localization and Detecting Alphabet in Sign Language BISINDO Using Convolution Neural Network. Technium: Romanian Journal of Applied Sciences and Technology, 16, 143–149. www.techniumscience.com
Muchsin, A., & Abdul, B. (2021). Mouse Tracking Tangan dengan Klasifikasi Gestur Menggunakan OpenCV dan Mediapipe. Jurnal Teknologi Informasi dan Komputer, 1(2), 10–18.
Fadlilah, U., et al. (2024). Sign Language Word Detection Using LRCN. IOP Conference Series: Materials Science and Engineering, 1305(1), 012023. https://doi.org/10.1088/1757-899X/1305/1/012023
M, M. H., S, M. B., Ravi Iyer, R., & J, S. J. (2023). Sign Language Detection and Classification using Hand Tracking and Deep Learning in Real-Time. International Research Journal of Engineering and Technology. www.irjet.net
Halder, A., & Tayade, A. (2021). Real-time Vernacular Sign Language Recognition using MediaPipe and Machine Learning. International Journal of Research and Publication Reviews, 2(5), 9–17. www.ijrpr.com
Pandapotan, B. R., Aulia, S., & Hadiyoso, S. (2023). Perancangan sistem penerjemah sign language to text berbasis image processing. Jurnal Teknologi Informasi dan Komunikasi, 9(1), 1–9.
Rejeki, S., Ramdhania, K. F., & Hantoro, K. (2009). Sistem Informasi Pemesanan Menu Makanan Berbasis Web. Jurnal Ilmu Komputer.
Ua, A. M. T. I. S., et al. (2023). Penggunaan Bahasa Pemrograman Python Dalam Analisis Faktor Penyebab Kanker Paru-Paru. Jurnal Ilmiah Teknologi Informasi, 2(2).
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 International Journal of Electrical Engineering, Mathematics and Computer Science
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
