DeTR-OCR: Combining DETR and Transformer-Based OCR for Real-Time Video Caption Extraction

Abstract

The rapid production of video content across digital platforms requires strong systems for extracting textualslogans in real time video.This paper represents DeTR-OCRi.e a novel framework that integrates DetectionTransformer(DETR) and Transformerbased Optical Character Recognition (TrOCR) to focuses on these difficulties. utilizing DETR’s set forecast pa-radigm and TrOCR’s recognition ability to DeTR-OCR accomplishes end-to-end caption extraction from video streams. ICDAR 2017 Multi-Lingual Scene Text (MLT) and News VideosText datasets are used to test accuracy of proposed model. The system yields a mean Average Precision (mAP) of 0.82 and Character Error Rate (CER) of 0.09 on ICDARtogether with 28 frames per second (FPS) for realtime achievement. Provisional analyses expose superiority over baselines like as Efficient and Accurate Scene Text Detector (EAST) and Tesseract,while destruction studies underscore the effectiveness of flexible frame sampling. Regardless of challenges with lowcontrast text, DeTR-OCR suggestions a promising outcome for live sub-titling and accessibility with concrete way for future improvements in video text processing.