When Siri is speaking, press the Volume buttons on the side of your iPhone.

If you're using headphones, adjust the volume from your iPhone.