Challenge
University, committed to fostering an inclusive environment for its diverse student body, identified a need to bridge the language barrier for international students attending lectures in Polish. The existing infrastructure lacked real-time translation capabilities, creating challenges for non-native speakers to fully engage with the course material and participate effectively in classroom discussions. The primary challenge was to develop a system that could accurately transcribe and translate lectures in real-time, with minimal latency and maximum accessibility, while integrating seamlessly with the university's existing AV setup and IT infrastructure.
Solution
To address these challenges, I designed and implemented a novel real-time lecture translation system that combined state-of-the-art speech recognition and machine translation technologies:
- Integrated OpenAI's Whisper ASR model for real-time audio transcription. Fine-tuned the model to the specific nuances of Polish academic language, optimizing transcription accuracy and minimizing errors in the spoken word capture.
- Implemented LibreTranslate, an open-source machine translation engine, to translate the transcribed Polish text into English and other languages in real-time. This integration involved setting up a dedicated translation server, optimizing translation speed and accuracy, and providing support for multiple target languages.
- Created a user-friendly web interface using JavaScript with React that displayed the translated text in real-time on student devices. This interface featured customizable font sizes, color themes, and display options to accommodate individual student preferences and accessibility requirements.
- Developed a streaming architecture using WebSockets to ensure low-latency delivery of translated text from the translation server to student devices. This involved optimizing data transfer protocols, implementing efficient data compression techniques, and ensuring robust error handling for uninterrupted service.
- Designed a comprehensive user authentication and authorization system that allowed students to securely access the translation service using their existing university credentials. This integration involved implementing OAuth 2.0 authentication, role-based access controls, and secure data storage practices to protect student data and maintain system integrity.
Results
- Significantly improved accessibility for international students, enabling them to fully engage with lecture content and participate more actively in classroom discussions, as evidenced by a 40% increase in participation rates among non-native speakers.
- Reduced language-related learning barriers, leading to a 25% improvement in average quiz scores among international students, demonstrating a more effective understanding of course material.
- Enhanced the university's reputation as an inclusive and supportive learning environment, attracting a more diverse pool of international applicants and strengthening its position as a leader in global education.
- Provided a cost-effective translation solution compared to traditional human translation services, resulting in a 60% reduction in translation expenses while maintaining high levels of accuracy and accessibility.
- Created a highly scalable and adaptable system that can easily accommodate new languages, content formats, and accessibility requirements, ensuring the university can continue to meet the evolving needs of its international student body.
Technologies Used
Python 3.9 with OpenAI's Whisper for audio transcription, LibreTranslate for machine translation, JavaScript with React for frontend development, WebSockets for real-time communication, Laravel 9 for backend architecture, and Docker for containerization and deployment.