If no close-by building has room left to garrison any more units, the Villagers will just keep working. The Town Bell is a very useful function (especially in earlier stages of a game) to prevent the loss of Villagers and to rout the attackers when the town is under attack.