He was never said to be afraid of the Royal Guard. He just said that he would return to the Vandenreich base and wait for them to make their move.