Fact. Temperature, time of day, humidity, and rain can affect levels of the yellow, sneezy stuff. If you have allergies, the best time to go outside is right after heavy rains. Pollen counts run lowest on chilly, soggy days. They tend to run highest between 10 a.m. and 4 p.m., especially on hot, dry, and windy days. If you want to go outdoors, try to wait until the late afternoon, as the counts start to fall.