Heat oil in a large skillet over medium-high until very hot but not smoking. Cook 2 fillets, almond side down, pressing with a spatula occasionally to ensure contact with skillet, until almonds are well toasted and fish is nearly cooked through, about 3 minutes. Turn and cook until opaque throughout, about 1 minute more. Transfer to a plate. Repeat with remaining fillets.