Former leader of the Conservative party Colin Craig, accompanied by his wife Helen, has admitted "inappropriate" conduct in his relationship with his former press secretary Rachel MacGregor but denied claims he sexually harassed her.

Colin Craig's former press secretary has accused him of trying to "frame" her "like a mistress", and has asked to be released from a confidentiality agreement so she can go public.

Craig, who on Monday, stood with his wife at his side to rebut swirling allegations of sexual harassment involving Rachel MacGregor, said he was open to it and was discussing it with his lawyers today.

He resigned as Conservative Party leader on Friday, after some members of the board tried to confront him over the allegations.

SUPPLIED Former Conservative Party press secretary Rachel MacGregor.

After days of publicly denying any knowledge of the allegations, Craig admitted "inappropriate" conduct in his relationship with MacGregor but denied claims he sexually harassed her.

He also rejected claims of a secret financial settlement with MacGregor but detailed payments including the settlement of an outstanding $50,000 invoice. He said both parties eventually agreed on a figure of $16,000.

Craig also said he and his wife Helen gave MacGregor a personal loan of $20,000 to settle significant credit card debt, which he later forgave.

Lawrence Smith Former Conservative Party leader Colin Craig and his wife Helen at a press conference in Auckland.

"In hindsight, on some occasions our conduct was inappropriate and we have acknowledged that so that we can both move on," Craig said at a press conference he attended with his wife, Helen.

After Monday's press conference, MacGregor broke her silence via a statement from her lawyer, accusing Craig of "clear factual inaccuracies", which she was happy to address, should he release her from a confidentiality agreement.

She later posted on Twitter: "Colin Craig is trying to frame me as a mistress. There was never a sexual relationship..."

Lawrence Smith Former Conservative Party leader Colin Craig and his wife Helen at a press conference in Auckland.

He and MacGregor have confirmed there was a mediated settlement. MacGregor said it was with the Human Rights Commission, under which they were both bound by confidentiality.

But Craig said he was speaking publicly in response to some of the allegations that had been made against him.

He said that in hindsight on some occasions he had engaged in "inappropriate" conduct with MacGregor.

DAVID WHITE/Fairfax NZ Former Conservative Party leader Colin Craig says he might be open to releasing Rachel MacGregor from a confidentiality agreement, but would need to discuss it with lawyers.

"Let me make it clear that I have never sexually harassed anyone and allegations to the contrary are wrong."

MacGregor said his comments amounted to a breach of their agreement, but she was still bound to honour it.

"Colin Craig's statement [yesterday], along with other public statements in recent days, breach a confidentially agreement that Mr Craig and I reached during a Human Rights Commission mediation," she said.

"Despite Mr Craig's public comments I have been advised by my lawyers that I am still bound by the confidentiality agreement.

"I am therefore unable to correct the clear factual inaccuracies contained in the statement Mr Craig made today without jeopardising my legal position."

MacGregor said she was willing to correct the statements and brief the Conservative Party board, if Craig promised he would not take legal action against her.

Craig said he would need to discuss MacGregor's request to be released from the confidentiality agreement with his lawyers. He was not ruling it out.

"She's offered the opportunity to perhaps lift some confidentiality and we might not be opposed to that."

On Radio New Zealand's Morning Report, Craig said his lawyers had confirmed her offer was genuine.

"My preference, if we're going to lift confidentiality, is that it's lifted entirely, that both parties are free to speak and all information is out in the public domain."

On MacGregor's statement about inaccuracies, Craig said he had received no correspondence "that anything I've said is inaccurate".

"That may be where we start, we invite her to explain to us what inaccuracies she thinks exist and maybe we go from there."

Party chief executive Christine Rankin said she was considering cutting all ties with the party in light of the "embarrassing" political fallout.

"I felt just awful about the way it was conducted. I don't think that's the whole story and there's more to come," she told Radio NZ.

Rankin is still making up her mind whether to quit the party altogether after losing confidence in Craig some time ago.

"I want to be part of the decision making of what happens but I don't know what the future is beyond that."

Meanwhile, Craig confirmed he would be writing to the party's grassroots in what appears to be a bid to save his political career.

Craig said he had apologised to his wife, who made a statement of her own.

Helen Craig said: "I choose to stand with my husband here today in full love and support of him whom I believe has been falsely accused."

Speculation about her husband related to "allegations already withdrawn," she said.

Board member John Stringer claimed that Craig had treated both his own wife and MacGregor "very poorly".

"It's sad to see Helen up there, Colin should have been protecting her, not the other way around."

Comments Craig made about MacGregor's finances were "despicable", Stringer said.

MacGregor abruptly resigned two days out from the 2014 election.

Craig has poured $4 million of his own money into the party he founded in 2011.