The co-author of Virginia Giuffre’s memoirs says Prince Andrew’s behaviour is “catching up with him” as he is reportedly in discussion about leaving his Royal Lodge home.

The King’s brother is said to be in talks with Charles’ representatives about leaving Royal Lodge voluntarily following the furore over the “peppercorn” rent for his 30-room mansion, but is reluctant to give up his residence of more than 20 years.

The prince gave up the Duke of York title and other honours last Friday ahead of the publication of Ms Giuffre’s posthumous memoir, following allegations he sexually abused her.

The Daily Telegraph said the sticking points in the talks are the location of his new home and financial compensation for funds spent on the lodge.

Copies of Nobody’s Girl by Virginia Roberts Giuffre, co-written by Amy Wallace (James Manning/PA) (PA Wire)

The book’s co-author Amy Wallace said Ms Giuffre wanted “wealthy, powerful men to be held to account”.

In an interview with the Associated Press, she said: “What we’re watching is Prince Andrew’s life is being eroded for his past behaviour.

“While he’s not being tried in a court of law, his behaviour’s catching up with him”.

Ms Wallace said Andrew could still come forward about what he knew about the late paedophile financier Jeffrey Epstein and help victims of sexual abuse.

“He still denies what is in this book, we should say. But it’s not too late,” she said.

“He could continue to leave aside the things he’s accused of with Virginia. He could still come forward. He was in those houses, he was on the island, he was in the private jet numerous times.

“He could still come forward and say to the authorities, I want to validate these women’s experience.”