I was running the following PHP code:

<?php </script> ?>

There were no parse errors and the output was " ?> " (example).

In similar cases I do get a parse error:

<?php </div> ?>

Parse error: syntax error, unexpected '<' in ...