If it happens only occasionally, it's not likely to be serious. All men have problems with erections at some time in their lives.
If it slowly but consistently gets worse, there's probably a physical cause. This is generally what happens with chronic impotence.
If it happens suddenly but you're still stiff early in the morning and can get an erection while masturbating, that suggests your mind is involved. There could be something going on physically, too.
When to Call Your Doctor
Pick up the phone if your ED worries you so much that it causes anxiety or threatens your sexual relationship. At the very least, your doctor can clear up misinformation, which often makes sexual problems worse. Sometimes taking medication for a short time can get you through a rough patch, too.
Also talk to the doctor if it's painful to get an erection or difficult because your penis is curved (a condition called Peyronie's disease).
If the problem doesn't go away, it could be an early warning sign of a more serious, larger condition. For example, the penile artery can get narrower because of coronary artery disease or diabetes.
To help your erections, your doctor may suggest taking medication as a pill or as an injection in your penis, or using a mechanical device. And you'll need to deal with the underlying medical condition, too.