Because if you are a eunuch, it does seem to put you at an elevated risk for this disease.

http://www.cancerhelp.com/malebreastcancer.htm