There are lots of things that can impact your cholesterol. One of the main ones is your genes. If your parents or grandparents had high levels, there's a good chance you could, too. Despite your genes, diet does matter. It's best to limit foods with cholesterol, saturated fats, and trans fats. That means try to cut back on fatty meats, full-fat dairy products, and deep-fried and processed foods.