Abstract
The emerging versatile video coding (VVC) standard adopted an innovated multi-type tree (MTT) versatile block structure, comprising binary trees (BTs) and ternary trees (TTs) pruning. This new tree structures’ flexibility, induced by the MTT module, significantly improved the compression performance. However, it dramatically increased the coding complexity due to the brute force search for rate distortion optimization (RDO). To cope with this issue, we proposed a fast decision approach using a lightweight neural network (LNN) with an early direction determination scheme to avoid redundant MTT pruning and hence, reduced considerable computing complexity. The I-frame processing significantly affected the coding efficiency. Thus, the main goal of the suggested LNN-based approach is to substitute the brute force RDO search, used to check all block decision candidates, without affecting the compression efficiency performance. Based on the BT RD cost, the TT splitting direction was selected in a first step. Subsequently, an adequate LNN-based model was applied to predict the corresponding VVC TT partition, which deeply optimized the VVC coding unit partition module. Experiments over various test sequences showed that the proposed method substantially decreased the total encoding time by up to 46% with negligible compression efficiency loss under the all-intra configuration.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Gary, J.S., Jens-Rainer, O., Woo-Jin, H., Thomas, W.: Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Circuits Syst. Video Technol. 22, 1649–1668 (2012)
Bossen, F., Li, X., Suehring, K.: AHG Report: Test Model Software Development (AHG3), JVET-J0003 (2018)
Amna, M., Imen, W., Ezahra, S.F., et al.: Fast intra-coding unit partition decision in H. 266/FVC based on deep learning. J. Real Time Image Process. 17(6), 1971–1981 (2020)
Jin, Z., An, P., Yang, C., et al.: Fast QTBT partition algorithm for intra frame coding through convolutional neural network. IEEE Access 6, 54660–54673 (2018)
Yang, H., Shen, L., Dong, X., et al.: Low-complexity CTU partition structure decision and fast intra mode decision for versatile video coding. IEEE Trans. Circuits Syst. Video Technol. 30(6), 1668–1682 (2019)
Cho, S., Kim, M.: Fast CU splitting and pruning for suboptimal CU partitioning in HEVC intra coding. IEEE Trans. Circuits Syst. Video Technol. 23(9), 1555–1564 (2013)
Lee, D., Jeong, J.: Fast intra coding unit decision for high efficiency video coding based on statistical information. Signal Process. Image Commun. 55, 121–129 (2017)
Sun, X., Chen, X., Xu, Y., et al.: Fast CU partition strategy for HEVC based on Haar wavelet. IET Image Process. 11(9), 717–723 (2017)
Zhang, M., Lai, D., Liu, Z., et al.: A novel adaptive fast partition algorithm based on CU complexity analysis in HEVC. Multimed. Tools Appl. 78(1), 1035–1051 (2019)
Huang, C., Peng, Z., Chen, F., et al.: Efficient CU and PU decision based on neural network and gray level co-occurrence matrix for intra prediction of screen content coding. IEEE Access 6, 46643–46655 (2018)
Kim, K., Ro, W.W.: Fast CU depth decision for HEVC using neural networks. IEEE Trans. Circuits Syst. Video Technol. 29(5), 1462–1473 (2018)
Amna, M., Imen, W., Ezahra, S.F.: Lenet5-based approach for fast intra coding. In: 2020 10th International Symposium on Signal, Image, Video and Communications (ISIVC), pp. 1–4. IEEE (2021)
Xu, M., Li, T., Wang, Z., et al.: Reducing complexity of HEVC: a deep learning approach. IEEE Trans. Image Process. 27(10), 5044–5059 (2018)
Amna, M., Imen, W., Soulef, B., Fatma, E.Z.: Machine Learning-Based approaches to reduce HEVC intra coding unit partition decision complexity. Multimed. Tools Appl. 81, 1–26 (2021)
Wang, Z., Wang, S., Zhang, J., et al.: Probabilistic decision based block partitioning for future video coding. IEEE Trans. Image Process. 27(3), 1475–1486 (2017)
Ryu, S., Kang, J.: Machine learning-based fast angular prediction mode decision technique in video coding. IEEE Trans. Image Process. 27(11), 5525–5538 (2018)
Lin, T.L., Jiang, H.Y., Huang, J.Y., et al.: Fast intra coding unit partition decision in H. 266/FVC based on spatial features. J. Real Time Image Process. 17(3), 493–510 (2020)
Park, S.H., Kang, J.: Fast multi-type tree partitioning for versatile video coding using a lightweight neural network. IEEE Trans. Multimed. 23, 4388–4399 (2020)
Park, S.H., Kang, J.W.: Context-based ternary tree decision method in versatile video coding for fast intra coding. IEEE Access 7, 172597–172605 (2019)
Chen, F., Ren, Y., Peng, Z., et al.: A fast CU size decision algorithm for VVC intra prediction based on support vector machine. Multimed. Tools Appl. 79(37), 27923–27939 (2020)
VVC Test Model (VTM) Software. Accessed: 24 July 2019. Available: https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/
Frank, B., Jill, B., Karsten, S., Xiang, L., Vadim, S.: JVET Common Test Conditions and Software Reference Configurations for SDR Video, Document JVET-M1010, Joint Video Experts Team (JVET) of ITU-T and ISO/IEC (2019)
Fernández, D.G., Del Barrio, A.A., Botella, G., Meyer-Baese,U., Meyer-Baese, A., Grecos, C.: Information fusion based techniques for HEVC. In: Real-Time Image and Video Processing 2017, vol. 10223, p. 102230M (2017)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Amna, M., Imen, W. & Fatma Ezahra, S. Fast multi-type tree partitioning for versatile video coding using machine learning. SIViP 17, 67–74 (2023). https://doi.org/10.1007/s11760-022-02204-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-022-02204-4