Rendering Techniques through Images in Computer Graphics
Keywords:
Differentiable rendering, Dynamic scene modeling, Image-based rendering, Implicit neural representations, Multi-view geometry, Neural Radiance Fields (NeRF), Neural scene representation, Novel view synthesis, Real-time neural graphics, Volumetric renderingAbstract
The field of computer graphics is currently undergoing a transformative shift from traditional explicit geometric modeling toward data-driven neural implicit representations. This review paper provides an extensive and structured survey of contemporary image-based rendering (IBR) techniques, with a primary focus on the evolution and diversification of neural radiance fields (NeRF). Historically, IBR relied on complex geometric proxies, multi-view stereo, and depth-map interpolation, which frequently struggled with non-Lambertian surfaces, transparency, and thin structures. The emergence of coordinate-based neural networks has redefined this paradigm by encoding 3D scenes as continuous volumetric functions of spatial position and viewing direction. This study presents a meticulous analysis of thirty influential research contributions that have shaped the modern landscape of neural rendering. These works are categorized based on their technical innovations, including multi-scale anti-aliasing, hash-grid acceleration, tensor factorization for memory efficiency, and the modeling of non-rigid deformations in dynamic environments. By synthesizing findings from these thirty pivotal papers, this review highlights significant milestones in achieving photorealistic novel view synthesis, reducing computational latency for real-time applications, and automating the reconstruction of large-scale outdoor scenes. The study concludes by situating these advancements within the broader context of AR/VR, digital twins, and autonomous systems, offering a comprehensive look at how neural rendering has established a new gold standard for immersive visual computing.
References
B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng, “NeRF: Representing scenes as neural radiance fields for view synthesis,” Communications of the ACM, vol. 65, no. 1, pp. 99–106, 2022.
J. T. Barron, B. Mildenhall, M. Tancik, P. Hedman, R. Martin-Brualla, and P. P. Srinivasan, “Mip-NeRF: A multiscale representation for anti-aliasing neural radiance fields,” in Proc. IEEE/CVF Int. Conf. Computer Vision (ICCV), Montreal, QC, Canada, 2021, pp. 5835–5844.
A. Yu, R. Li, M. Tancik, H. Li, R. Ng, and A. Kanazawa, “PlenOctrees for real-time rendering of neural radiance fields,” in Proc. IEEE/CVF Int. Conf. Computer Vision (ICCV), Montreal, QC, Canada, 2021, pp. 5732–5741.
P. P. Srinivasan, B. Deng, X. Zhang, M. Tancik, B. Mildenhall, and J. T. Barron, “NeRV: Neural reflectance and visibility fields for relighting and view synthesis,” in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 2021, pp. 7491–7500.
D. T. Waide, D. G. Green, and G. F. Gribakin, “B-spline basis implementation of the Hartree-Fock method for arbitrary central potentials,” Journal of Physics: Conference Series, vol. 1412, no. 13, Art. no. 132029, 2020.
H. Hashmi and D. S. Kalogerias, “Model-free learning of optimal deterministic resource allocations in wireless systems via action-space exploration,” in Proc. IEEE 31st Int. Workshop Machine Learning for Signal Processing (MLSP), Gold Coast, Australia, 2021, pp. 1–6.
M. Tancik et al., “Fourier features let networks learn high frequency functions in low dimensional domains,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 33, 2020, pp. 7537–7547.
G. Gkioxari, J. Malik, and J. Johnson, “Mesh R-CNN,” in Proc. IEEE/CVF Int. Conf. Computer Vision (ICCV), Seoul, South Korea, 2019, pp. 9784–9794.
R. Martin-Brualla, N. Radwan, M. S. M. Sajjadi, J. T. Barron, A. Dosovitskiy, and D. Duckworth, “NeRF in the wild: Neural radiance fields for unconstrained photo collections,” in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 2021, pp. 7206–7215.
A. Belavin, V. Belavin, and G. Koshevoy, “Periods of the multiple Berglund–Hübsch–Krawitz mirrors,” Letters in Mathematical Physics, vol. 111, no. 93, Jul. 2021.
S. Peng et al., “Neural body: Implicit neural representations with structured latent codes for novel view synthesis of dynamic humans,” arXiv preprint arXiv:2012.15838, 2020.
J. Gwak, C. Choy, and S. Savarese, “Generative sparse detection networks for 3D single-shot object detection,” arXiv preprint arXiv:2006.12356, 2020.
V. Sitzmann, J. N. P. Martel, A. W. Bergman, D. B. Lindell, and G. Wetzstein, “Implicit neural representations with periodic activation functions,” arXiv preprint arXiv:2006.09661, 2020.
S. J. Garbin, M. Kowalski, M. Johnson, J. Shotton, and J. Valentin, “FastNeRF: High-fidelity neural rendering at 200FPS,” arXiv preprint arXiv:2103.10380, 2021.
A. Chen, Z. Xu, A. Geiger, J. Yu, and H. Su, “TensoRF: Tensorial radiance fields,” arXiv preprint arXiv:2203.09517, 2022.
T. Müller, A. Evans, C. Schied, and A. Keller, “Instant neural graphics primitives with a multiresolution hash encoding,” ACM Transactions on Graphics, vol. 41, no. 4, pp. 1–15, Jul. 2022.
J. T. Barron, B. Mildenhall, D. Verbin, P. P. Srinivasan, and P. Hedman, “Mip-NeRF 360: Unbounded anti-aliased neural radiance fields,” arXiv preprint arXiv:2111.12077, 2021.
L. Seco and M. Patrão, “Counting geodesics on compact symmetric spaces,” arXiv preprint arXiv:2204.11984, 2022.
R. Chandra et al., “Fine structures of an EUV wave event from multi-viewpoint observations,” The Astrophysical Journal, vol. 919, no. 1, p. 9, Sep. 2021.
J. Ost, F. Mannan, N. Thuerey, J. Knodt, and F. Heide, “Neural scene graphs for dynamic scenes,” arXiv preprint arXiv:2011.10379, 2020.
K. Park et al., “Nerfies: Deformable neural radiance fields,” arXiv preprint arXiv:2011.12948, 2021.
C. Reiser, S. Peng, Y. Liao, and A. Geiger, “KiloNeRF: Speeding up neural radiance fields with thousands of tiny MLPs,” arXiv preprint arXiv:2103.13744, 2021.
K. Deng, A. Liu, J.-Y. Zhu, and D. Ramanan, “Depth-supervised NeRF: Fewer views and faster training for free,” arXiv preprint arXiv:2107.02791, 2021.
S. Lombardi, T. Simon, J. Saragih, G. L. Schwartz, A. M. Lehrmann, and Y. Sheikh, “Neural volumes,” ACM Transactions on Graphics, vol. 38, no. 4, pp. 1–14, Aug. 2019.
O. Wiles, G. Gkioxari, R. Szeliski, and J. Johnson, “SynSin: End-to-end view synthesis from a single image,” arXiv preprint arXiv:1912.08804, 2019.
T. Zhou, R. Tucker, J. Flynn, G. Fyffe, and N. Snavely, “Stereo magnification: Learning view synthesis using multiplane images,” arXiv preprint arXiv:1805.09817, 2018.
T. G. Khunjua, V. C. Zhukovsky, K. G. Klimenko, and R. N. Zhokhov, “Duality and charged pion condensation in chirally asymmetric dense quark matter in the framework of an NJL2 model,” International Journal of Modern Physics: Conference Series, vol. 47, Art. no. 1860093, Jan. 2018.
C. B. Choy, D. Xu, J. Gwak, K. Chen, and S. Savarese, “3D-R2N2: A unified approach for single and multi-view 3D object reconstruction,” arXiv preprint arXiv:1604.00449, 2016.
K.-A. Aliev, A. Sevastopolsky, M. Kolos, D. Ulyanov, and V. Lempitsky, “Neural point-based graphics,” arXiv preprint arXiv:1906.08240, 2019.
