This statement threw me off a bit. Not sure if I understand, or have missed something.
Using iRules has a big security hole as iRule is executed after the connection is established.
You say this from a DDOS perspective, or how is it a security hole?
By default, the F5 only establishes the backend connection once a request has been received, and depending on your hardware/modules the unit can be very resilient SYN floods.
A few alternatives on how to separate
If hosting multiple services on one VIP you'd have to distinguish them somehow, right?
Here are two options (there are a lot more though).
Distributing HTTP services
Probably the most common way to do separate request between different pools is to use the host header. You can access this one via [HTTP::host] in the iRule if you have an HTTP profile assigned to the virtual server. You can easily combine the host header with a switch, if, data group list lookup to choose pools.
You can also use the URI, Cookies, Other headers and anything you can think of that the iRule HTTP commands supports. You can find all of them here: https://devcentral.f5.com/wiki/iRules.HTTP.ashx.
Distributing non HTTP services per port
With reservations that I consider this an ugly solution, this is an alternative. In this case you'd set up a wildcard virtual server that listens to all ports on one specific IP:
Then, you can use the event WHEN_CLIENT_ACCEPTED and throw [TCP::local_port] into a conditional statement.
Let's not forget about Traffic Policies
If one is not very versed in iRules Traffic Policies might be easier to use. They've been much improved in later versions of TMOS and they are more efficient than iRules although not as flexible.
Questions from me
I also have some questions to and . How does this work for Radware/Brocade? Never had the pleasure of working with those devices. :)