Abstract

Dot plots are widely used for data visualization across various domains due to their simplicity. However, extracting data from dot plot images remains a challenge, primarily because of the lack of dedicated methodologies and comprehensive datasets. Existing chart analysis approaches often overlook dot plots, and resources such as the Benetech – Making Graphs Accessible dataset fail to adequately capture the diversity of dot plot designs. To address this gap, a synthetic dataset has been developed, incorporating a wide range of variations in dot size, shape, color, background, and grid configurations. Additionally, a novel pipeline is proposed for extracting data from dot plot images. This pipeline leverages the context-aware chart extraction and data encoding (CACHED) model to detect key chart components, including plot areas, axis titles, legends, labels, and ticks. For dot detection, a you only look once (YOLO)-based model has been developed and trained on the synthetic dataset. To evaluate performance, 124 real-world dot plot images were manually annotated with dot positions and plot areas. A comparative analysis was conducted by training EfficientUNet and YOLOv5 models on both the Benetech dataset and the proposed synthetic dataset, followed by testing on the annotated real-world images. On the synthetic hold-out set, the YOLOv5 model achieved 99.1% precision, 98.5% recall, and a mAP@0.5 of 99.5%. When evaluated on real-world dot plot images, it maintained high performance, achieving 95.99% precision and a 90.19% F1-score at a normalized distance threshold of 0.02—demonstrating strong accuracy and generalization capability.

Keywords

Dot plot extraction, Synthetic datasets, Chart component detection, Data visualization, Object detection models, YOLO and EfficientUNet.

References

[1] Healy K. Data visualization: a practical introduction. Princeton University Press; 2024.

[2] Milev P. The role of data visualization in enhancing textual analysis. Trakia Journal of Sciences. 2023; 21(1):248-53.

[3] Farahani AM, Adibi P, Ehsani MS, Hutter HP, Darvishy A. Automatic chart understanding: a review. IEEE Access. 2023; 11:76202-21.

[4] Huang KH, Chan HP, Fung YR, Qiu H, Zhou M, Joty S, et al. From pixels to insights: A survey on automatic chart understanding in the era of large foundation models. IEEE Transactions on Knowledge and Data Engineering. 2025; 37(5): 2550-68.

[5] Yau N. Visualize this: the flowing data guide to design, visualization, and statistics. John Wiley & Sons; 2024.

[6] Jacoby WG. The dot plot: a graphical display for labeled quantitative values. The Political Methodologist. 2006; 14(1):6-14.

[7] Sönning L. The dot plot: a graphical tool for data analysis and presentation. University of Bamberg Press; 2015.

[8] Rodrigues N, Weiskopf D. Nonlinear dot plots. IEEE Transactions on Visualization and Computer Graphics. 2017; 24(1):616-25.

[9] Masson D, Malacria S, Vogel D, Lank E, Casiez G. Chartdetective: easy and accurate interactive data extraction from complex vector charts. In proceedings of the CHI conference on human factors in computing systems 2023 (pp. 1-17). ACM.

[10] https://www.datadigitization.com/. Accessed 08 May 2025.

[11] https://plotdigitizer.com/. Accessed 08 May 2025.

[12] Rohatgi A. Automeris. io: computer vision assisted data extraction from charts using WebPlotDigitizer. Automeris. 2025.

[13] Luo J, Li Z, Wang J, Lin CY. Chartocr: data extraction from charts images via a deep hybrid framework. In proceedings of the winter conference on applications of computer vision 2021 (pp. 1917-25). IEEE/CVF.

[14] Kato H, Nakazawa M, Yang HK, Chen M, Stenger B. Parsing line chart images using linear programming. In proceedings of the winter conference on applications of computer vision 2022 (pp. 2109-18). IEEE/CVF.

[15] Rane C, Subramanya SM, Endluri DS, Wu J, Giles CL. Chartreader: automatic parsing of bar-plots. In 22nd international conference on information reuse and integration for data science (IRI) 2021 (pp. 318-25). IEEE.

[16] Cheng ZQ, Dai Q, Hauptmann AG. Chartreader: a unified framework for chart derendering and comprehension without heuristic rules. In proceedings of the international conference on computer vision 2023 (pp. 22202-13). IEEE/CVF.

[17] Mustafa O, Ali MK, Moetesum M, Siddiqi I. Charteye: a deep learning framework for chart information extraction. In international conference on digital image computing: techniques and applications (DICTA) 2023 (pp. 554-61). IEEE.

[18] Liang C, Xue F, Li Z. Automatic chart decoding system based on deep learning and image processing. In 4th international symposium on artificial intelligence and intelligent manufacturing (AIIM) 2024 (pp. 499-504). IEEE.

[19] Hassan MY, Singh M. LineEX: data extraction from scientific line charts. In proceedings of the winter conference on applications of computer vision 2023 (pp. 6213-21). IEEE/CVF.

[20] Lal J, Mitkari A, Bhosale M, Doermann D. Lineformer: line chart data extraction using instance segmentation. In international conference on document analysis and recognition 2023 (pp. 387-400). Cham: Springer Nature Switzerland.

[21] Al-kady Y, Abdel-wahab N, Hassanein F, Al-alfy R, El-malatawy L, Ammar R, et al. Chartlytics: chart image classification and data extraction. In eleventh international conference on intelligent computing and information systems (ICICIS) 2023 (pp. 515-22). IEEE.

[22] Liu F, Eisenschlos J, Piccinno F, Krichene S, Pang C, Lee K, et al. DePlot: one-shot visual language reasoning by plot-to-table translation. In findings of the association for computational linguistics: ACL 2023 (pp. 10381-99). Association for Computational Linguistics.

[23] Liu F, Piccinno F, Krichene S, Pang C, Lee K, Joshi M, et al. MatCha: enhancing visual language pretraining with math reasoning and chart derendering. In the 61st annual meeting of the association for computational linguistics 2023 (pp. 5464–79).

[24] Chen J, Kong L, Wei H, Liu C, Ge Z, Zhao L, et al. Onechart: purify the chart structural extraction via one auxiliary token. In proceedings of the 32nd international conference on multimedia 2024 (pp. 147-55). ACM.

[25] Wang Y. Beyond heuristics: multimodal transformer for chart data extraction. In international conference on computers, information processing and advanced education (CIPAE) 2024 (pp. 156-9). IEEE.

[26] Meng F, Shao W, Lu Q, Gao P, Zhang K, Qiao Y, et al. ChartAssistant: a universal chart multimodal language model via chart-to-table pre-training and multitask instruction tuning. In findings of the association for computational linguistics 2024 (pp. 7775-803). ACL.

[27] Kim W, Park S, In Y, Han S, Park C. SIMPLOT: enhancing chart question answering by distilling essentials. In findings of the association for computational linguistics: NAACL 2025 (pp. 573-93). ACL.

[28] Wu Y, Yan L, Shen L, Wang Y, Tang N, Luo Y. ChartInsights: evaluating multimodal large language models for low-level chart question answering. In findings of the association for computational linguistics: EMNLP 2024 (pp. 12174-200). ACL.

[29] Zhou Z, Wang H, Zhao Z, Zheng F, Wang Y, Chen W, et al. ChartKG: a knowledge-graph-based representation for chart images. IEEE Transactions on Visualization and Computer Graphics. 2024:1-15.

[30] Kanroo MS, Kawoosa HS, Rana K, Goyal P. C3E: a framework for chart classification and content extraction. Computers and Electrical Engineering. 2025; 121:1-18.

[31] Davila K, Lazarus R, Xu F, Rodríguez AN, Setlur S, Govindaraju V, et al. Chart-info 2024: a dataset for chart analysis and recognition. In international conference on pattern recognition 2024 (pp. 297-315). Cham: Springer Nature Switzerland.

[32] Masry A, Do XL, Tan JQ, Joty S, Hoque E. ChartQA: a benchmark for question answering about charts with visual and logical reasoning. In findings of the association for computational linguistics: 2022 (pp. 2263-79). ACL.

[33] Methani N, Ganguly P, Khapra MM, Kumar P. Plotqa: reasoning over scientific plots. In proceedings of the winter conference on applications of computer vision 2020 (pp. 1527-36). IEEE/CVF.

[34] Jobin KV, Mondal A, Jawahar CV. Docfigure: a dataset for scientific document figure classification. In international conference on document analysis and recognition workshops (ICDARW) 2019 (pp. 74-9). IEEE.

[35] Kafle K, Price B, Cohen S, Kanan C. Dvqa: understanding data visualizations via question answering. In proceedings of the conference on computer vision and pattern recognition 2018 (pp. 5648-56). IEEE.

[36] Siegel N, Horvitz Z, Levin R, Divvala S, Farhadi A. Figureseer: parsing result-figures in research papers. In European conference on computer vision 2016 (pp. 664-80). Cham: Springer International Publishing.

[37] Caldwell B, Cooper M, Reid LG, Vanderheiden G, Chisholm W, Slatin J, et al. Web content accessibility guidelines (WCAG) 2.0. WWW Consortium (W3C). 2008; 290(1-34):5-12.

[38] Smith AR. Color gamut transforms pairs. ACM Siggraph Computer Graphics. 1978; 12(3):12-9.

[39] Gonzalez RC. Digital image processing. Pearson Education India; 2009.

[40] https://roboflow.com/. Accessed 08 May 2025.

[41] https://www.kaggle.com/competitions/benetech-making-graphs-accessible/data. Accessed 10 May 2025.

[42] Yan P, Ahmed S, Doermann D. Context-aware chart element detection. In international conference on document analysis and recognition 2023 (pp. 218-33). Cham: Springer Nature Switzerland.

[43] Jocher G, Chaurasia A, Stoken A, Borovec J, Kwon Y, Michael K, et al. Ultralytics/yolov5: v7. 0-yolov5 sota realtime instance segmentation. Zenodo. 2022.

[44] Baheti B, Innani S, Gajre S, Talbar S. Eff-unet: a novel architecture for semantic segmentation in unstructured environment. In proceedings of the conference on computer vision and pattern recognition workshops 2020 (pp. 358-9). IEEE/CVF.

Cite this article