Sign Language Detector Using TensorFlow with Convolutional Neural Network (CNN) Method
DOI:
https://doi.org/10.33998/jakakom.2025.5.2.2386Keywords:
Detector, Sign Language, Image Processing, TensorFlow, CNNAbstract
Sign language recognition plays a vital role in facilitating communication for individuals with hearing impairments. This study proposes a Convolutional Neural Network (CNN) model trained to recognize patterns in sign language images with the aim of improving the accuracy and efficiency of sign language recognition systems. The model was trained in two stages with the first training session achieving a validation accuracy of around 63%, while the second training session yielded an impressive validation accuracy exceeding 92% at epoch 29. This significant improvement demonstrates the model’s ability to effectively learn and generalize complex patterns in sign language images, signaling its potential for practical applications in sign language interpretation. The high accuracy achieved by the CNN model demonstrates its suitability for use in a variety of real-world scenarios, such as assistive technology for the deaf community or automation systems requiring hand gesture recognition. Thus, the trained CNN model has the potential to be a valuable tool in improving the accessibility and efficiency of communication for individuals who rely on sign language.
Downloads
References
M. A. Imaddudin, I. W. Hamzah, and S. Astuti, “Simulasi Penerjemah SIBI (Sistem Isyarat Bahasa Indonesia) Menggunakan Tensorflow Dan Convolutional Neural Network (CNN),” e-Proceeding Eng., vol. 8, no. 6, pp. 3911–3918, 2022, [Online]. Available: https://openlibrarypublications.telkomuniversity.ac.id/index.php/engineering/article/view/19146
N. T. Adam, Z. A. Tyas, and T. Hardiani, “Deteksi Gestur Sistem Isyarat Bahasa Indonesia Menggunakan Metode Deep learning SSD MobileNet V2 FPNLite,” Sainteks, vol. 21, no. 2, p. 129, Oct. 2024, doi: 10.30595/sainteks.v21i2.24006.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification with deep convolutional neural networks,” Commun. ACM, vol. 60, no. 6, pp. 84–90, May 2017, doi: 10.1145/3065386.
J. Anggara, E. Ryansyah, and B. Arif Dermawan, “IMPLEMENTASI OBJECT DETECTION DALAM KLASIFIKASI SAMPAH UNTUK MENINGKATKAN EFISIENSI PENGELOLAAN LIMBAH,” JATI (Jurnal Mhs. Tek. Inform., vol. 9, no. 3, pp. 4923–4930, May 2025, doi: 10.36040/jati.v9i3.13813.
A. Rohim, Y. A. Sari, and T. Tibyani, “Convolution Neural Network (CNN) Untuk Pengklasifikasian Citra Makanan Tradisional,” J. Pengemb. Teknol. Inf. dan Ilmu Komput., vol. 3, no. 7, pp. 7038–7042, 2019, [Online]. Available: https://j-ptiik.ub.ac.id/index.php/j-ptiik/article/view/5851
Nasha Hikmatia A.E. and M. I. Zul, “Aplikasi Penerjemah Bahasa Isyarat Indonesia menjadi Suara berbasis Android menggunakan Tensorflow,” J. Komput. Terap., vol. 7, no. 1, pp. 74–83, Jun. 2021, doi: 10.35143/jkt.v7i1.4629.
D. R. R. Putra and R. A. Saputra, “IMPLEMENTASI CONVOLUTIONAL NEURAL NETWORK (CNN) UNTUK MENDETEKSI PENGGUNAAN MASKER PADA GAMBAR,” J. Inform. dan Tek. Elektro Terap., vol. 11, no. 3, pp. 710–714, Aug. 2023, doi: 10.23960/jitet.v11i3.3286.
I. N. T. A. Putra, K. S. Kartini, Y. K. Suyitno, I. M. Sugiarta, and N. K. E. Puspita, “Penerapan Library Tensorflow, Cvzone, dan Numpy pada Sistem Deteksi Bahasa Isyarat Secara Real Time,” J. Krisnadana, vol. 2, no. 3, pp. 412–423, May 2023, doi: 10.58982/krisnadana.v2i3.335.
I. B. A. Peling, I. M. P. A. Ariawan, and G. B. Subiksa, “Deteksi Bahasa Isyarat Menggunakan Tensorflow Lite dan American Sign Language (ASL),” J. Krisnadana, vol. 3, no. 2, pp. 90–100, Jan. 2024, doi: 10.58982/krisnadana.v3i2.534.
R. A. Malik and E. Zuliarso, “Metode Convolutional Neural Network Untuk Mendeteksi Jenis Sayur Menggunakan Tensorflow,” Media Bina Ilm., vol. 15, no. 12, pp. 5873–5882, 2021, [Online]. Available: https://ejurnal.binawakya.or.id/index.php/MBI/article/view/1147
A. H. Gustsa and G. S. Permadi, “Sistem Deteksi Bahasa Isyarat Secara Realtime Dengan Tensorflow Object Detection dan Python Menggunakan Metode Convolutional Neural Network,” Inov. J. Ilm. Inov. …, vol. 7, no. 2, pp. 1–10, 2023, [Online]. Available: https://ejournal.unhasy.ac.id/index.php/inovate/article/view/4116
C. N. Insani, N. Arifin, and M. R. Rasyid, “Deteksi Gerakan Bahasa Isyarat Menggunakan Euclidean Distance,” Inform. J. Ilmu Komput., vol. 19, no. 1, pp. 99–106, May 2023, doi: 10.52958/iftk.v19i1.5658.
H. M. Putri, F. Fadlisyah, and W. Fuadi, “PENDETEKSIAN BAHASA ISYARAT INDONESIA SECARA REAL-TIME MENGGUNAKAN LONG SHORT-TERM MEMORY (LSTM),” J. Teknol. Terap. Sains 4.0, vol. 3, no. 1, p. 663, Mar. 2022, doi: 10.29103/tts.v3i1.6853.
D. Robert, M. Nababan, and Z. Budiarso, “Sistem Pendeteksi Gerakan Bahasa Isyarat Indonesia Menggunakan Webcam Dengan Metode Supervised Learning,” J. Ilm. Komputasi, vol. 22, no. 3, pp. 449–456, Oct. 2023, doi: 10.32409/jikstik.22.3.3403.
A. R. Ardiansyah, A. H. Nur’azizan, and R. Fernandis, “Implementasi Deteksi Bahasa Isyarat Tangan Menggunakan OpenCV dan MediaPipe,” Stain. (Seminar Nas. Teknol. Sains), vol. 3, no. 1, pp. 331–337, 2024, [Online]. Available: https://proceeding.unpkediri.ac.id/index.php/stains/article/view/4337
F. Chollet, Deep Learning with Python, Second Edi. Shelter Island: Manning Publications Co., 2021. [Online]. Available: https://www.manning.com/books/deep-learning-with-python-second-edition
D. P. Mawardi, M. Novita, and N. Dwi Saputro, “Deteksi Awal Klasifikasi Jenis Penyakit Kanker Kulit Dengan Algoritma Convolutional Neural Network (CNN) Berbasis Mobile Apps,” Adopsi Teknol. dan Sist. Inf., vol. 3, no. 2, pp. 1–6, Dec. 2024, doi: 10.30872/atasi.v3i2.2305.
