For most of us, our workspace is what it is. If your employer has cubicles or one long communal table, then that’s where you work, and the best you can do is adapt. The trouble is that many of us don’t adapt to less-than-ideal work environments as well as we could. More often, we just put up with them.

And in the process, we adopt a few common work habits that wind up making us needlessly spend a lot of extra mental effort just to stay focused. Chances are they won’t even come as a surprise. But here’s a look at why they’re such a drain on our productivity, and some easy steps to take to get past them.

Clutter creeps up, especially when we’re busy. The memo you almost finished, a request from a friend, receipts to reimburse, and who knows what else sit desperately demanding your attention. Like a dragged-out break-up, you’re afraid to let your clutter go, but wince when you have to confront the fact that it’s still around.

Of course it would feel nicer to sit at a clean desk, but you don’t need things to be pretty, you just need to work . . . right?

Does it really matter if your cluttered workspace is mildly annoying, as long as you’re still getting work done? It’s a fair question. Of course it would feel nicer to sit at a clean desk, but you don’t need things to be pretty, you just need to work. The time it would take to clean things up might be better spent on that project whose deadline is looming, right?

Well, not exactly. And the reason has to do with your brain’s attention systems, which aren’t designed strictly to keep you focused. They’re designed to help you pick up on what’s changing, threatening, novel, and so on. In other words, they help make sure we don’t miss what’s important in the environment. In fact, staying focused too easily and for too long would pose an evolutionary threat. We’d die if we regularly concentrated so hard on a given task that we failed to notice the people, cars, tigers, or falling objects coming at us. So we have these brain systems regularly scanning for something to shift attention onto.

We don’t usually leave tigers and moving cars on our desks, of course, but the kinds of things we do leave there are especially hard to tune out. Research shows that people tend to leave things on their desks as reminders. If you think about what kinds of things you don’t finish and need reminders for, you’ll find that many of them signal hard and time-consuming tasks, and being reminded of them feels overwhelming.

What’s more, many are tied to social obligations at work: Somebody you know personally is waiting on just about every one of those tasks that your desk clutter is reminding you about. The human brain devotes a lot of processing power to navigating social obligations, so we can reasonably expect that when our attention lands on those reminders, they’ll be especially hard to tune out.