What do you say when a fwb tells you he wants to have a baby with you?

Recently my friend with benefits told me that he wants to have a baby with me. We had been drinking and I know sometimes people say things that they are afraid to say sober. Sometimes people also talk out of their a**. I didn't know what to say so I left. Was this the wrong way to handle this situation? He hasn't really said much to me since. Do you think I hurt him, or is he just mad at me for leaving?