A nearby gardener will not watch over your growing marigolds. Seeds can be purchased from Olivia's Draynor Seed Market for 2 coins. It is also a common Thieving item from Olivia's seed stall with a Thieving level 27, and it is also possible to pickpocket it from Master Farmers with level 38 Thieving.