This is a terrible act and I’m sorry you had to experience this. My uncle got married decades ago(I was a kid) and his then wife for 2 days had psychological diagnosis. Her family hid it; she threw mangalasutra on my uncle’s face the very next day, screamed and behaved strangely. He dropped her at their family’s home and never let her come back in his life. They demanded alimony to sign divorce proceedings but our family refrained from paying. Both the parties had no contact for years and my uncle remained unmarried.
No, you don’t have to pay the money your ex-wife’s family is demanding. Stay firm but sadly, this will be dragged for years. More power to you!