If you're lactose intolerant, you may not have to give up cheese. Much of the lactose drains away with the whey, and what's left breaks down as the cheese ages, so aged hard cheeses like cheddar, Swiss and Parmigiano Reggiano generally have far less than fresh ones. Check with your doctor first, and read Can I eat cheese if I am lactose intolerant? A milk allergy is a different thing and means avoiding cheese entirely.
Also see: Is cheese vegetarian? · Is cheese gluten free? · What is processed cheese?