Well lets put it this way, you only getting in less calories that would be burnt out during your workout. So no, as long as the diet is structured correctly, then it wouldn't be a problem... If one doesn't understand enough about diets and our bodies, then yes that can become a problem if you do not do the adjustment properly..
"Whether You Think You Can or Can't, You're Right"--Henry Ford