There was a time, not so long ago, that David Davis was a great fan of the EU charter of fundamental rights. He liked it so much that he used it to take up a legal challenge against the snooper’s charter (brainchild of the-then home secretary Theresa May), which ended up in Luxembourg.

How times change. Yesterday’s draft repeal bill sees Davis knocking out the protection of the charter on the day that we exit the EU. That means a whole swath of rights and protections will be lost to British citizens if it goes through unamended. It’s true that we will still have the rights inherent in the European convention on human rights (ECHR), but the two frameworks are different. While there is some overlap, the EU charter takes up a gamut of protections which are increasingly important in our fast-evolving society.

Take the right to data protection, which Davis relied on with deputy Labour leader Tom Watson for the challenge (until Davis withdrew from the case) leading to the European court of justice ruling against the general and indiscriminate retention of emails or electronic communications by government, with serious implications for the snooper’s charter. Or take the protection of children’s rights, or the freestanding right to equality. As with every argument when human rights treaties are involved, it’s always worth digging to see precisely which rights people are comfortable about losing.

Although Theresa May has declared her own commitment to workers’ rights, her record on human rights is chequered

Although May has declared her commitment to workers’ rights, her record on human rights is chequered. Meanwhile, the government’s position is dubious. The parliamentary joint committee of human rights expressed serious concern about the government’s approach to safeguarding individuals’ fundamental rights post-Brexit, other than those protected under the ECHR. It noted that the government “seemed unacceptably reluctant to discuss the issue of human rights after Brexit. The minister of state responsible for human rights was either unwilling or unable to tell us what the government saw as the most significant human rights issues.” Meanwhile the UN high commissioner for human rights recently issued strong words against May’s call for human rights to be overturned if they were to “get in the way” of the fight against terror”.

Some may argue, or at least wish, that abolition of the Human Rights Act has been kicked so far into the long grass that it may not happen at all. But exit day marks a significant change in our direction of travel. No longer subject to the protections of the EU rights framework and its court, we need to make sure our constitutional and legal framework protects us. This week’s supreme court victory for John Walker, ensuring equal pension rights for his husband, was thanks to EU law, and is a timely reminder. We cannot afford to fall (or be pushed) behind European standards; indeed, our rights framework – on employment, the environmental, human rights and other important social protections – may yet be a prerequisite for trade agreements.

Parliamentary time will only be made for new laws to safeguard our rights if MPs insist. Environmental lawyers are gearing up already. Equality lawyers and groups, as well as the Equality and human rights commission, are making the case for a freestanding constitutional right to equality.

None of this may matter to a Conservative government, with its mutterings about abolishing the Human Rights Act. But the government’s position now is precarious, and ministers may have no choice but to listen.

Labour’s position on rights is to be broadly welcomed and shadow Brexit secretary Keir Starmer has set up a red-flag challenge on the charter, opening up the possibility for proper public debate. It’s the tip of the iceberg of course, and a year late, but it’s now or never. Rather than the warm, fuzzy words of aspirational Brexit cliche, we need a serious assessment of what we want and how we will get it. That means a clear acknowledgement of what rights we stand to lose under the repeal bill as currently drafted, and as a consequence of losing the EU rights framework more broadly. The failure to conduct impact assessments is an inexcusable derogation of public duty.

Which brings me back to David Davis. He declares, on the front page of the European Union (withdrawal) bill that its provisions are compatible with the ECHR. But right there, in the explanatory notes to the bill, is an extraordinary attempted power grab: the government wants to be able to remove the rights of EU citizens in the event of no deal, without a parliamentary vote. The charter may be on its way out, but the rights of EU nationals under the European convention will still apply. Mass litigation, and mass chaos, may follow. No wonder the joint committee was “surprised to be informed that the government saw the question of domestic protection for fundamental rights as a matter for negotiation with the other EU member states”.

Rights don’t often seem as though they matter, until they do. By then, it might be too late.