So.. Tried hooking up the vacuum gauge tonight to see what's going on.
Hooked it up to the plenum first. It shows a nice strong vacuum at idle and when driving with throttles more or less closed. No vacuum at WOT of course. Everything looks normal.
Second test was on the vacuum canister to see if I actually had any vacuum at WOT for the multi rams, and I certainly have - all the way. When the different solenoids open I can see a short dip in the vacuum and that tells me the flaps are operated. They operate when they should, but it gave me an indicator for something else:
When the rear multi-ram flap opens, I get the growling, shaking, struggling engine sound. If I have 95% open throttle it won't open and the sound is normal. At 100% open, the vacuum dips when the rear flap opens and it starts to growl. What on earth could be causing that?

This only happens in the 1800-4000 range (approx.)
Could this mean that the Y26SE don't like the short plenum at all? If that's the case, what's the difference?
I am pretty sure that the system is sealed properly. Changed all o-rings when I swapped. Plenum, butterflies and breathers was cleaned properly too, but I see quite a bit of oil in the plenum now (but strangely not sucked in from the butterflies, they are still perfectly clean).