Finally got it working, with a little help from commenter WTFCircuit and after noticing that the timescale for the simulation seems to have an effect on the result;
If I "zoom out" to tens of microseconds (or further out) the output (above cap) goes almost flat but zooming in to a few microseconds and maybe opening/closing the switch once gets it going again...
I guess it has to do with the "sampling rate" within the simulation but it's still weird and frustrating as hell to troubleshoot :-/
The double diodes seems to slow down the voltage pumping a bit. Go with a single diode if you prefer a steeper voltage curve.