Don’t worry about anything, just as a precaution record all his calls from now on and screenshot texts if he gives you threats on WhatsApp etc.
This is a petty dispute and he’s just threatening you because he’s insecure.
No legal advice needed here, you haven’t done anything wrong.
