Despite Jakob Nielsen's insisting on doing things like this there is no specific answer that you can give to this.

But speed is an important part of the user experience and one which is often lost somewhere between UX, design and engineering.

One of the best resources I have found on the subject is Stoyan Stefanov's (ex-Yahoo currently Facebook engineer and creator of the Yslow! plugin) Book of Speed. Especially the first chapter has some nice case studies of how performance affects all aspects of the user experience.

Depending on your target groups and the performance of the systems you are working with you can set ambitious targets. I think the point is that it's maybe worth dedicating a lot of engineering time to improving performance and not blowing it all on snazzy ajax front-end features. But there's no set rule of thumb.

There is a lot of potential for being innovative in the way the system handles load times. I just recently saw a presentation from an Instagram engineer where he talks about system speed vs the perceived speed to the user. Maybe the user doesn't need to see the actual uploading happening, but she can just start it and then be informed later if it has failed. When you press a button you can maybe let the state be activated right away even if there's still some work going on in the back end.