Because Goku was crying when Broly was a baby and that annoyed him so. The crying would drive Broly to insanity, and a hatred towards Goku that he carried all throughout his life, so it would make sense for him to take revenge on Goku, not only stopping the memories of Goku perhaps crying inside of his head, but just to calm himslef and his senses at least temporarily or for a while.