It will work fine. Most people design the refugium so the pods leaving it will only be subject to one pump (return) and not the skimmer and return. The idea is that more of the pods will survive the journey up into the main display. While I am sure this is true, not all pods will die with a multi-pump route and indeed many will make it up into the display.
My newly made sump/refugium is set up similar to yours as the pods will pass the skimmer and then on to the return pump.
Remember that the refugium is not only a pod-farm but also and macro algae export center
I focus on growing/exporting macro algae in my refugium.
BTW.. for those of you with phosban reactors, take a look inside it and you'll see lots of pods swimming around. They are tough little buggers