If the flashing is seriously damaged or cut, it might need replacing. If this is the case, you will have to lift up or remove the roofing material around, as the flashing goes underneath it. If your ceiling has a plastic vapor barrier between the drywall and the attic insulation, push the insulation aside and look for flow stains on the plastic.