Coding tree unit (CTU) is the basic processing unit of the High Efficiency Video Coding (HEVC) video standard and conceptually corresponds in structure to macroblock units that were used in several previous video standards.[1][2] CTU is also referred to as largest coding unit (LCU).[3]
A CTU can be between 16×16 pixels and 64×64 pixels in size with a larger size usually increasing coding efficiency.[4][2] The first video standard that uses CTUs is HEVC/H.265 which became an ITU-T standard on April 13, 2013.[5][6][7]
History
editMacroblock encoding methods have been used in digital video coding standards since H.261 which was first released in 1988. However, for error correction and signal-to-noise ratio the standard 16x16 macroblock size is not capable of getting the kind of bit reductions that information theory and coding theory suggest are theoretically and practically possible.[8]
Technical details
editHEVC replaces macroblocks, which were used with previous video standards, with CTUs which can use larger block structures of up to 64×64 pixels and can better sub-partition the picture into variable sized structures.[4][9]
HEVC initially divides the picture into CTUs which are then divided for each luma/chroma component into coding tree blocks (CTBs).[4][9]
A CTB can be 64×64, 32×32, or 16×16 with a larger pixel block size usually increasing the coding efficiency.[4] CTBs are then divided into one or more coding units (CUs), so that the CTU size is also the largest coding unit size.[4]
- The arrangement of CUs in a CTB is known as a quadtree since a subdivision results in four smaller regions.[4]
- CUs are then divided into prediction units (PUs) of either intra-picture or inter-picture prediction type which can vary in size from 64×64 to 4×4.[4][9] To limit worst-case memory bandwidth when applying motion compensation in the decoding process, prediction units coded using inter-picture prediction are restricted to a minimum size of 8×4 or 4×8 if they are predicted from a single reference (uni-prediction) or 8×8 if they are predicted from two references (bi-prediction).[4][10]
- To code the prediction residual, a CU is divided into a quadtree of DCT transform units (TUs).[4][11] TUs contain coefficients for spatial block transform and quantization.[4][9] A TU can be 32×32, 16×16, 8×8, or 4×4 pixel block sizes.[4]
Standardization
editAt the July 2012 HEVC meeting it was decided, based on proposal JCTVC-J0334, that HEVC level 5 and higher would be required to use CTB sizes of either 32×32 or 64×64.[3][12] This was added to HEVC in the Draft International Standard as a level limit for the Log2MaxCtbSize variable.[13]
Log2MaxCtbSize was renamed CtbSizeY in the October 2012 HEVC draft and then renamed CtbLog2SizeY in the January 2013 HEVC draft.[10][14]
Coding efficiency
editThe design of most video coding standards is primarily aimed at having the highest coding efficiency.[2] Coding efficiency is the ability to encode video at the lowest possible bit rate while maintaining a certain level of video quality.[2] HEVC benefits from the use of larger CTB sizes.[2]
This has been shown in peak signal-to-noise ratio (PSNR) tests with a HM-8.0 HEVC encoder where it was forced to use progressively smaller CTU sizes.[2] For all test sequences when compared to a 64×64 CTU size it was shown that the HEVC bit rate increased by 2.2% when forced to use a 32×32 CTU size and increased by 11.0% when forced to use a 16×16 CTU size.[2]
In the Class A test sequences, where the resolution of the video was 2560×1600, when compared to a 64×64 CTU size it was shown that the HEVC bit rate increased by 5.7% when forced to use a 32×32 CTU size and increased by 28.2% when forced to use a 16×16 CTU size.[2]
The tests showed that large CTU sizes become even more important for coding efficiency with higher resolution video.[2] The tests also showed that it took 60% longer to decode HEVC video encoded at 16×16 CTU size than at 64×64 CTU size.[2] The tests showed that large CTU sizes increase coding efficiency while also reducing decoding time.[2] The tests were conducted with the Main profile of HEVC based on equal PSNR.[2]
| Video test sequences | Maximum CTU size used in video encoding in comparison to 64×64 CTUs | ||
|---|---|---|---|
| 64×64 CTUs | 32×32 CTUs | 16×16 CTUs | |
| Class A (2560×1600 pixels) | 0% | 5.7% | 28.2% |
| Class B (1920×1080 pixels) | 0% | 3.7% | 18.4% |
| Class C (832×480 pixels) | 0% | 1.8% | 8.5% |
| Class D (416×240 pixels) | 0% | 0.8% | 4.2% |
| Overall | 0% | 2.2% | 11.0% |
| Encoding time | 100% | 82% | 58% |
| Decoding time | 100% | 111% | 160% |
See also
edit- High Efficiency Video Coding (HEVC) - Video standard that supports 8K UHDTV and resolutions up to 8192 × 4320
- H.264/MPEG-4 AVC - The predecessor video standard of HEVC
- VP9 - A video codec with superblocks, which are similar to CTUs
- Macroblock - The basic processing unit used in several previous video standards
References
edit- ^ G.J. Sullivan; J.-R. Ohm; W.-J. Han; T. Wiegand (2012-05-25). "Overview of the High Efficiency Video Coding (HEVC) Standard" (PDF). IEEE Transactions on Circuits and Systems for Video Technology new ownership of coding:GilbertLeeGrimaldoJr.(c)(r)(tm). Retrieved 2013-04-26.
- ^ a b c d e f g h i j k l m G.J. Sullivan; Heiko Schwarz; Thiow Keng Tan; Thomas Wiegand (2012-08-22). "Comparison of the Coding Efficiency of Video Coding Standards – Including High Efficiency Video Coding (HEVC)" (PDF). IEEE Transactions on Circuits and Systems for Video Technology. Retrieved 2013-04-26.
- ^ a b Gary Sullivan; Jens-Rainer Ohm (2012-10-13). "Meeting report of the 10th meeting of the Joint Collaborative Team on Video Coding (JCT-VC), Stockholm, SE, 11-20 July 2012". JCT-VC. Retrieved 2013-04-28.
- ^ a b c d e f g h i j k G.J. Sullivan; J.-R. Ohm; W.-J. Han; T. Wiegand (2012-05-25). "Overview of the High Efficiency Video Coding (HEVC) Standard" (PDF). IEEE Transactions on Circuits and Systems for Video Technology. Retrieved 2013-04-26.
- ^ "ITU-T Home : Study groups : ITU-T Recommendations : ITU-T H.265 (04/2013)". ITU. 2013-04-13. Retrieved 2013-04-16.
- ^ "AAP Recommendation: H.265". ITU. 2013-04-13. Retrieved 2013-04-16.
- ^ "AAP Announcement No. 09". ITU. 2013-04-15. Retrieved 2013-04-16.
- ^ "Diary of an x264 Developer » the problems with wavelets". Archived from the original on 2014-01-29. Retrieved 2014-02-06.
- ^ a b c d "Description of High Efficiency Video Coding (HEVC)". JCT-VC. 2011-01-01. Retrieved 2012-09-15.
- ^ a b "High Efficiency Video Coding (HEVC) text specification draft 10 (for FDIS & Consent)". JCT-VC. 2013-01-17. Retrieved 2013-01-24.
- ^ Thomson, Gavin; Shah, Athar (2017). "Introducing HEIF and HEVC" (PDF). Apple Inc. Retrieved 5 August 2019.
- ^ Wade Wan; Tim Hellman (2012-07-03). "Adding a Level Restriction on Coding Tree Block Size". JCT-VC. Retrieved 2012-09-22.
- ^ "High Efficiency Video Coding (HEVC) text specification draft 8". JCT-VC. 2012-07-28. Retrieved 2012-07-31.
- ^ "High Efficiency Video Coding (HEVC) text specification draft 9". JCT-VC. 2012-10-22. Retrieved 2012-10-23.