If the car was sold "as is", I'm surprised the OP is even entertaining the idea of continuing communication with the buyer. It does not matter what was or wasn't said, the car was sold, collected, the funds were exchanged and the change of ownership occurred (I presume it did with the DMV).
If that's the case this is the end of story. I would tell the guy to stop calling me, explain why and if he persisted I would go to the police for harassment.
At the very least, his number can be blocked by the phone service provider.
I had something similar happen when I sold my 91 Tracer back in 2000. I told the guy that the transmission was leaking ATF and he even brought his "mechanic" friend to inspect the car. I think it was a week or so after the transaction that the guy called me saying the transmission needed to be fixed and it would cost a lot of money. He sort of left it hanging there so I just replied “So, what do I have to do with this? You inspected the car, had your friend inspect the car and I told you about the transmission”. He started saying that he didn’t expect the cost to fix the transmission was so high, blah, blah, blah, basically a bunch of excuses to sucker me into feeling sorry for him. I said “Sorry can’t do anything about it, stop calling me.” That was the last I heard from him. I wasn’t yelling or showed anger in my voice, just firmly stated that I cannot do anything for him.