Armenia is perfect for adventurous backpackers. There is only one traditional hostel (the Envoy at $25/night) and a couple of hostelly homes for much less, right by the Opera, run by Anahit, so in Yerevan, you should be able to find what you're looking for. In other parts of the country, there are no hostels, and it's you, you're backpack, and a very friendly people who you just have to be open to communicating with in order to 1) get around on the minibuses and 2) figure out where you can sleep for cheap.