If the point of such slogans is merely that no one is born with a belief in God’s existence, and that such a belief develops later, I agree. Babies do not have beliefs regarding such things. This much is obvious (although it is irrelevant to the question of the truth of theism). But if their point is that babies should be described as atheists, this is patently absurd. Indeed, it’s because babies do not have any beliefs regarding God that they can be neither a theist nor an atheist.
No one is born an atheist. To claim otherwise is to employ a faulty definition of atheism as “a lack of belief in God.” So defined, atheism is relegated to a psychological state rather than a rational claim regarding the veracity of a particular proposition. This is not only a departure from the historic definition of atheism, but it guts it of any rational significance. Atheism is not a lack of belief in God. That is more properly described as “agnosticism.” Atheism is the belief that the proposition “God exists” is false. No baby is born with that belief, and thus no baby is born an atheist.