Hm, not sure I would want to put the plunger on backwards, since the pressure at depth will push the plunger inwards.
For doing basic testing in a pool or similar you could just epoxy the whole endcap, and you should be good to go. As @Fe3C explains, you might have some more resistance when closing the endcaps, and you need to make sure the straps are properly tightened. Once I left the ROV running during a faire. It was sitting on a table, hooked up to a monitor so people would see themselves as they waked passed. After a few hours of continous running (no use of motors means you have quite a few hours of runtime), I noticed the endcaps had “popped” open, which is due to the heat generated by the electronics while running. As Lukas sais, it could be a good experiment for the kids to calculate how much the pressure would build up inside the tube at different temperatures. Maybe even have them publish the results on this forum.
Normally what we do in order to prevent the above problem is to apply a little bit of vacuum pre-dive, using a small handheld vacuum pump like the one recommended in the dozuki. This is where the plunger comes into play, as it allows you to connect the pump to the hole in the endcap, suck out some air, and then rapidly exchange the vacuum pump with the plunger.
You could off course also swap out the syring / plunger assembly with something a bit more sturdy, as the type of valve seats that are used in the first stage of scuba regulators, but this would require drilling a sufficiently big hole, and using a threading tool to make the threads.