I needed a way to simulate out of order packets in order to try to reproduce an issue I've been seeing at work, and I figured out a fairly simple way that only involves ipfw which is built-in on 10.9 but unfortunately has been removed in 10.10. The same technique should be adaptable to pfctl though.

Step 1 - Creating a pipe

To be able to simulate out of order packets, you must create a pipe in ipfw with a rule that matches the traffic you're trying to affect.

sudo ipfw add 1 pipe 1 ip from <host> to me

You must create another pipe if you want to affect traffic in the opposite direction.

sudo ipfw add 2 pipe 1 ip from me to <host>

Step 2 - Simulate out of order packets

ipfw lets you set a bunch of config options on traffic flowing through a pipe. You can limit the amount of bandwidth with bw <speed>, packet loss with plr <rate>, and latency with delay <time>.

So to simulate 1 second latency, we would run:

sudo ipfw pipe 1 config delay 1s

However, this doesn't simulate out of order packets by itself. You can simulate packet loss which can induce out of order packets with:

sudo ipfw pipe 1 config plr 0.05 # 5% packet loss

But I believe I found a better solution (IMO) by constantly setting the delay to random values every couple of milliseconds.

sudo ruby -e "loop { \\`ipfw pipe 1 config delay #{rand(500) + 100}ms\\`; sleep 0.01 }"

where the delay is randomly set to (0-500) + 100ms every 10ms.

Step 3 - Clean up

Once you're done with your testing, be sure to remove the rules.

sudo ipfw del 1

Conclusion

This worked well enough for me, and I hope it helps you too. If someone figures out how to do this with pfctl on 10.10, please let me know!