Do you know how much dairy your body can handle? I do. It's important to know how much dairy your specific body can take. I know that the only dairy product, so far, that I can handle is butter. The reason is that there isn't much lactose in butter, so that's why my body can handle it. (http://paleohacks.com/questions/87374/does-butter-contain-lactose#axzz2JIOw7LHh)
As a test, try to have a small amount of something with lactose in it, like butter or yogurt and see how your body reacts. If you have no ill effects, than your body can take it.