This study demonstrates the design of an automated, precise 3D segmentation of muscle and bone in the shoulder using convolutional neural networks (CNN) to measure intramuscular fat (IMF), a key predictor of rotator cuff repair failure. Using high resolution, 3D Dixon MRI sequences from 80 shoulders were used to train the model, followed by external validation of 25 shoulder scans from different scanners and scans of different orientations. The CNN was trained to segment 8 regions of interest simultaneously, including the four rotator cuff muscles (supraspinatus, subscapularis, infraspinatus, teres minor), the large surrounding muscles (deltoid, teres major), and respective bones (scapula, clavicle), segmenting each region within 50 seconds. Compared to manual segmentation performed by humans, the model achieved a Dice Similarity Coefficient (DSC) of 0.89 and higher for most regions, an intraclass correlation coefficient (ICC) of 0.93 and higher for most regions in muscle volume, and reliability in IMF measurements except in smaller, more complex muscles. Traditionally, Goutallier Classification (GC), a single-slice and subjective grading scale to measure IMF percentages in muscle, is used, where a 3 or higher represents an IMF percentage that compromises surgical outcomes. Importantly, the 3D IMF model was able to accurately predict muscle regions that were above the threshold, with near-perfect discrimination (AUC greater than or equal to 0.93; infraspinatus AUC = 1.0). On the other hand, 2D simulated slices failed to differentiate muscles above and below the GC threshold, achieving an average AUC of 0.50, demonstrating the requirement of 3D IMFs for accurate, reliable IMF classifications.