I want to calculate the security cost of middlebox traversal when VM migrate from one physical server to another. Middle boxes can be firewalls or IPS/IDS containing rules checking the VM traversing them. Now imagine the most simple scenario that the only problem is to find the cost of checking VM by middlebox rules (this is what I call it security cost), and according to this cost finding the optimum path.
However there are already some protocols out there such as BGP or OSPF, but unfortunately non of them consider the security cost.
I do not agree that the right way to arrive at the optimum path is to calculate firewall rules. Instead, I would focus on the impact of large set of rules. Instead of trying to find out how many rules are present or what security features are enabled, you should define the optimum path as the one that has lowest network latency. That is probably easily measured. If there is a firewall with lot of rules and can still process traffic at faster rate, you should not mind going through that firewall, right?