It's worth remembering here how MIDI is routed through Live. Notes coming out of the piano roll pass through MIDI effects and into instruments which generate sound.

This means that the changes being made by the Velocity device aren't reflected in the piano roll; they're visibly hidden but audibly present. If you want to "see" how the velocity randomized notes look or tweak the velocities individually after the randomization process, you'll need to first record the MIDI notes coming out of the track containing your Velocity device. To do this, simply route the MIDI from this track to the MIDI input of another MIDI track. Then, simply arm the new MIDI track to record your randomized MIDI on and you'll end up with recorded notes whose randomized velocities are frozen in time, ready for further tweaking.

If you want to be sure your song sounds the same every time, you'll need to perform this process to record and lock in place any probabilistic parameters such as Random on the Velocity device.