The reasons for that decline aren't clear, and participants' average testosterone levels are in the normal range, the study shows.
The report appears in The Journal of Clinical Endocrinology and Metabolism's early online edition.
The researchers work at the New England Research Institutes in Watertown, Mass. They included Thomas Travison, PhD.
Travison's team studied the testosterone levels of 1,500 Boston men.
Blood levels of testosterone were first checked in the late 1980s. The second testing was in the mid-1990s. The third testing was in 2002-2004.
All of the men were 45 to 79 years old when they took the testosterone test.
As men age, their testosterone levels naturally drop. So the researchers made sure their comparisons were fair in terms of the men's age.
The key finding: Testosterone levels declined over time beyond what would be expected with age.
Travison comments on the findings in a news release from The Endocrine Society (the journal's publisher).
"In 1988, men who were 50 years old had higher serum testosterone concentrations than did comparable 50-year-old men in 1996," Travison says.
"This suggests that some factor other than age may be contributing to the observed declines in testosterone over time," he adds.
Obesity Obesity and smoking don't appear to explain the results, the study shows.
However, average testosterone levels were within the normal range for all generations studied, and testosterone levels can vary throughout the day.
Other studies are needed to double-check the results, Travison's team notes in the news release.