Building a Multilingual Question Generation Pipeline with T5
Architecture and engineering behind SmartQ: combining T5 transformers, Speech-to-Text (STT), multi-language translation, and Text-to-Speech (TTS) into a unified pipeline.
Introduction & Technical Context
Educational platforms and study tools often require automated question generation from diverse input modalities including multi-language text and spoken audio. In the SmartQ Generator project, I built a full NLP pipeline leveraging sequence-to-sequence T5 transformers, Speech-to-Text (STT), translation modules, and Text-to-Speech (TTS).
1. Sequence-to-Sequence Modeling with T5
The Text-to-Text Transfer Transformer (T5) frames every NLP task into a unified text-to-text format. For question generation, text context passages are formatted with prefix prompts (e.g., 'generate question: <context>'), enabling the transformer to predict syntactically and semantically coherent questions.
2. Connecting Multimodal & Multilingual Components
The pipeline integrates speech recognition for voice inputs, machine translation for multilingual source documents, T5 model inference for question synthesis, and TTS audio rendering for final interactive playback.
from transformers import T5ForConditionalGeneration, T5Tokenizer
tokenizer = T5Tokenizer.from_pretrained("valhalla/t5-base-qg-hl")
model = T5ForConditionalGeneration.from_pretrained("valhalla/t5-base-qg-hl")
text_input = "generate question: <hl> Deep learning <hl> models learn representations from complex data."
inputs = tokenizer(text_input, return_tensors="pt", max_length=512, truncation=True)
outputs = model.generate(
input_ids=inputs.input_ids,
max_length=64,
num_beams=4,
early_stopping=True
)
question = tokenizer.decode(outputs[0], skip_special_tokens=True)3. Engineering Challenges: Latency & Post-Processing
Chaining STT, Translation, T5, and TTS can create multi-second latency bottlenecks. Optimizing model loading, utilizing PyTorch torch.no_grad(), and stripping duplicate or nonsensical question outputs improved end-to-end responsiveness.
Key Engineering Takeaways
- T5's unified prefix architecture allows versatile task fine-tuning for question generation.
- Modular NLP pipelines allow easy swapping of STT/TTS components without altering core generative transformer weights.
- Robust text normalization and quality heuristics are crucial to prevent grammatically invalid outputs.
Sujan K S — AI/ML Engineer