It was only a suggestion but at the earliest chapter where some punks tried to mess with Shanks, Lucky Roo killed the guy who was pointing his gun to Shanks. At that time, the readers suggested that he is the fastest man on the Earth, (by seeing him appearing all of a sudden) but this is only a myth. Kind of hard to believe looking at his size..