There is also a bunch of Barons "soldiers" most likely not to guard but to find troublemakers (or make them).