Mathematically, both situations are identical. You save 5 lives compared to 1; however, the second case directly involves our responsibility. WE kill the fat man to save the others. The latter situation is thus highly emotional and most people prefer to do nothing. Contrary to the first situation, pushing the fat man involves our ventromedial prefrontal cortex (VPMC) which has been shown to be responsible for emotions. People with a lesion on this part of the brain for instance tend to be very utilitarian and choose systematically to kill the fat man.
Reference: Moll, J., & de Oliveira-Souza, R. (2007). Moral judgments, emotions and the utilitarian brain. Trends in cognitive sciences, 11(8), 319-321.