Objectives Gadolinium-based contrast agents (GBCAs) have become an integral part in daily clinical decision making in the last 3 decades. However, there is a broad consensus that GBCAs should be exclusively used if no contrast-free magnetic resonance imaging (MRI) technique is available to reduce the amount of applied GBCAs in patients. In the current study, we investigate the possibility of predicting contrast enhancement from noncontrast multiparametric brain MRI scans using a deep-learning (DL) architecture. Materials and Methods A Bayesian DL architecture for the prediction of virtual contrast enhancement was developed using 10-channel multiparametric MRI data acquired before GBCA application. The model was quantitatively and qualitatively evaluated on 116 data sets from glioma patients and healthy subjects by comparing the virtual contrast enhancement maps to the ground truth contrast-enhanced T1-weighted imaging. Subjects were split in 3 different groups: enhancing tumors (n = 47), nonenhancing tumors (n = 39), and patients without pathologic changes (n = 30). The tumor regions were segmented for a detailed analysis of subregions. The influence of the different MRI sequences was determined. Results Quantitative results of the virtual contrast enhancement yielded a sensitivity of 91.8% and a specificity of 91.2%. T2-weighted imaging, followed by diffusion-weighted imaging, was the most influential sequence for the prediction of virtual contrast enhancement. Analysis of the whole brain showed a mean area under the curve of 0.969 ± 0.019, a peak signal-to-noise ratio of 22.967 ± 1.162 dB, and a structural similarity index of 0.872 ± 0.031. Enhancing and nonenhancing tumor subregions performed worse (except for the peak signal-to-noise ratio of the nonenhancing tumors). The qualitative evaluation by 2 raters using a 4-point Likert scale showed good to excellent (3–4) results for 91.5% of the enhancing and 92.3% of the nonenhancing gliomas. However, despite the good scores and ratings, there were visual deviations between the virtual contrast maps and the ground truth, including a more blurry, less nodular-like ring enhancement, few low-contrast false-positive enhancements of nonenhancing gliomas, and a tendency to omit smaller vessels. These “features” were also exploited by 2 trained radiologists when performing a Turing test, allowing them to discriminate between real and virtual contrast-enhanced images in 80% and 90% of the cases, respectively. Conclusions The introduced model for virtual gadolinium enhancement demonstrates a very good quantitative and qualitative performance. Future systematic studies in larger patient collectives with varying neurological disorders need to evaluate if the introduced virtual contrast enhancement might reduce GBCA exposure in clinical practice.