Neural Radiance Projection

Abstract

The proposed method, Neural Radiance Projection (NeRP), addresses three fundamental challenges in training convolutional neural networks for X-ray image segmentation: handling limited or missing human-annotated datasets, dealing with ambiguity in per-pixel labeling, and managing class imbalance between positive and negative classes. By leveraging a generative adversarial network (GAN), NeRP synthesizes a large volume of physics-based X-ray images known as Variationally Reconstructed Radiographs (VRRs). These images are paired with more accurately labeled 3D Computed Tomography data for segmentation purposes. As a result, VRRs demonstrate higher fidelity in terms of photorealistic metrics compared to other projection methods. Integrating NeRP outputs also outperforms standard UNet models trained on the same X-ray image pairs.