A typical trip from Earth Transfer Orbit to lunar orbit is roughly 3 days, plus Apollo 11 did 30 lunar orbit before it commenced he landing operation, which added a few more hours. Plus, Neil burnt almost all fuel looking for a suitable landing site, they almost had to abort the landing.