He isn't the weakest. He just hasn't been shown fighting many opponents, and when he has they are all powerful. Given his experiance and Yamamoto's discipline in choosing his subordinates, we can all assume that he is quite powerful, rather.