To fill in a bit beside comments.

Your question regarding technological compatibility in communication with alien species does not exclude linguistic analysis but is in the core of it.

For one they would scan a wide specter of electromagnetic waves, filter out noise, record patterns and then start to decipher. There would likely be a rather sophisticated chain of software (or probably hardware) for pattern recognition. As one recognize an artificial signal this would be filtered trough various modules to try to tune in on a likely representation via converters.

Here the linguistic part is at the core. As one are able to generate translators one should also be able to recognize patterns which are translatable / has linguistic properties. (And sometimes it would not be possible.) One would perhaps end up with n candidates, and then it would be up to technicians and communications officers like Hoshi Sato to fine tune and try to isolate the correct specimen.

I do not recall exactly episodes, but there are some examples where they do this operation.

Over time the systems would be more and more sophisticated as one encounter new languages and communications models. A self learning system with some hard core programmers to overhaul and tune.

One would also tune ones own signals to accommodate the ones used by other aliens. Sometimes one manage to decipher the signals received and as such one can immediately do a reversal and send signals using same pattern. Other times it would be the other way around: the aliens would adjust their transmission to accommodate what they have managed to decipher.

For systems one use oneself one would clearly not use algorithms with known flaws and bugs. NHL has a very good point in regards to self-describing communications protocols. Such a system would be built from ground up with one goal in mind, and not use flawed and outdated algorithms or specifications.