There's not enough vacuum present to open the regulator immediately. It's why sometimes you get a no-start if you don't let the engine crank long enough when the engine is hot. Therefore they fail (the vacuum bellows leak or the hose is cracked) causing high pressure. Against full vacuum, the plunger is pulled wide open, reducing rail pressure to a prescribed minimum.
The plunger controls how much fuel flow is allowed to by-pass the injectors. In our case, the fuel pressure regulators are vacuum operated, pulling against a spring that's always pushing the plunger inside closed.