This study presents a hybrid deep learning framework for early cancer prediction and risk analysis using multimodal biomedical imaging, specifically MRI and CT scans. A total of 512 images from 178 TCIA patients and 3,260 images from 369 BraTS patients were used. Images were preprocessed with Gaussian smoothing, contrast enhancement, and augmentation. Feature extraction combined handcrafted ORB descriptors with Inception V4 deep features, followed by classification using Sparse Logistic Regression and a Multi-Scale Graph Wavelet Neural Network (MS-GWNN). The proposed method achieved a classification accuracy of 94.5% (BraTS validation), sensitivity of 93.2%, and specificity of 93.8%, outperforming single-modality CNNs (89.4% accuracy) and wavelet-based fusion (91.7%). The model also demonstrated an AUC of 99.6% for distinguishing benign from malignant cases, with particularly strong performance in detecting invasive cancers (85% correctly classified) and maintaining 100% accuracy in normal cases. However, misclassification occurred between benign and in situ categories (~50% overlap), highlighting challenges in fine-grained differentiation. Statistical validation with Wilcoxon and Kruskal–Wallis tests confirmed the improvements were significant.