You want to separate the database access logic from the business logic. You want to have the ability to use different databases - Imagine you have a product that needs to work both in Sql Server and Postgres. You want the ability to use multiple ORMs, so you abstract that.

It is an additional layer of abstraction, which tends to add to the cognitive load you already have when working on significant projects. And it needs to be maintained of course.

The repository becomes just a pass-through layer to the DbContext adding no value to your architecture.

We are throwing away the ability to use some more advanced functionalities of EF, such as Lazy Loading, accessing the ChangeTracker, etc.

The queries are also business logic. To me it makes sense to test the query along with everything else.

It adds significant clutter to things like Unit Testing. Nowadays with EF there are excellent solutions such as the InMemoryProvider specifically designed for this scenario. Do not downplay the difficulty that complex tests can present!