It isn't a big allergy problem indoors unless mold finds a damp spot to grow. Then, look out, almost any damp spot will do. That includes wood, carpet, paper, and insulation.
Clean up mold where it grows and fix the cause of the moisture, such as a leaky pipe. To keep indoor humidity low, vent bathrooms and dryers, use a dehumidifier, and run vent fans when you cook, clean, and wash dishes.