Most people who notice the Moon in daylight are looking at a waning one, through a windscreen, on the way to work.

After full moon the Moon rises later each night and is still well up after sunrise, so the waning gibbous and last quarter spend their mornings in a bright blue sky. The waxing phases do exactly the same thing in the afternoon. There are simply fewer people outside with a reason to look up at four o'clock.