[cap-talk] "Don't Prohibit What You Can't Prevent" vs. "Security is Economics"

[zooko at zooko.com]

There is a valuable and beloved meme in the capability community which I have
heard attributed to Chip Morningstar: "Don't Prohibit What You Can't Prevent"
[1].  (Below, I will abbreviate this maxim as "DPWYCP".)

This maxim has inspired and guided me personally for years, but now I think
the time has come to revisit it and consider where it can lead us astray.  In
this note I will analyze it afresh in terms of costs, and propose a
counter-meme: "Security is Economics".  Finally, I'll suggest ways that the
"Security is Economics" approach casts light on the sticky topic of
"prohibition of delegation" including a novel (I think) mechanism for
deterring delegation.  (These also apply to the other sticky topic -- 
"Lampson confinement" -- but for brevity I will omit that part.)


The motivation for "DPWYCP" is to avoid the following failure of security
engineering:

Scenario 1: The security engineer "Alice" gives a system to a customer "Bob",
claiming that the system will prevent a certain act by "Carol".  Bob,
believing this claim, risks a high value on the chance that Carol can
accomplish the act.  Carol does accomplish the act, and Bob loses.

It is important to avoid this scenario, but the "DPWYCP" meme is too blunt an
instrument to separate the successes from the failures.  Consider Scenario 1
described in terms of costs:

Scenario 1, economically speaking: The security engineer "Alice" gives a
system to a customer "Bob", claiming that the system will impose a cost
"v_ss" on a certain act by an attacker "Carol".  We can think of v_ss as the
"value of the system strength".  Bob, believing this claim, risks a "value
risked" v_r in order to gain a "value gained" v_g.  Carol, can gain benefit
equal to v_a "value to the attacker" by accomplishing the act.  v_a is
greater than v_ss, so Carol expends v_ss to accomplish the act, and Bob loses
v_r.

>From the economic perspective, the problem with Scenario 1 is that the value
to the attacker, v_a, is greater than the system strength, v_ss.  From the
"DPWYCP" perspective, the problem with Scenario 1 is that v_ss is not
infinite.

Now clearly it is hard to estimate these values, v_ss, v_r, v_g, and v_a.  It
gets more complicated because Bob, who is not a security engineer, must
understand a good estimate of v_ss and v_a.  (The converse, in which Alice,
who is not an expert in Bob's business, tries to understand v_r, v_g, and v_a
sounds wrong to me, but it might be a good approach in some cases.)

It gets even more complicated given that the value gained by Carol -- v_a --
may be different from the value risked by Bob -- v_r.  For Alice and Bob to
estimate v_a is even harder than to estimate v_ss, v_r, and v_g.
Unfortunately if v_a is substantially different from v_r, then Alice and Bob
cannot use the simple inequality v_ss > v_r to determine if it is beneficial
to use the system.  Instead they must estimate the likelihood of Carol
committing the act in light of v_ss and v_a, and weigh that likelihood
against the loss that they would incur, v_r and the benefit that they could
gain, v_g.  (It is often reasonable to take the value risk -- v_r -- as an
estimated lower bound on v_a -- criminals rob banks because that's where the
money is.  Another reasonable consideration is that v_ss ought to be
proportional to v_r.)

(In "Secrets and Lies", Bruce Schneier gives examples of v_a > v_r, such as
when Carol is willing to spend vast resources to perform a DoS against Bob
because she hates Bob, even if she gains a net negative in terms of resources
by doing so.  He also gives examples of v_a < v_r, such as the story that a
thief stole 3.5" floppies, containing valuable data, causing significant
damage to the owner, only because the thief wanted some floppies.)

A compounding difficulty is that it might be costly to innovate an attack, 
but that once someone has done so it might be extremely cheap for others to 
use the attack.  In such a case, Alice's and Bob's estimates of v_ss may be 
much higher than the cost to the ultimate attacker.

Given the difficulties of safely estimating these values, the appeal of
"DPWYCP" is that it is conservative -- by prohibiting Alice from delivering
systems which offer any v_ss less than an "effectively infinite" v_ss, it
seeks to avoid Scenario 1.  Unfortunately there are three other economic
losses that can result from this conservative strategy.

The first economic loss of "DPWYCP" is due to the fact that in practice no
cost is effectively infinite.  If Alice follows "DPWYCP", then she delivers a
system with a "very high" v_ss.  If Alice is skillful, v_ss may be high
enough that the value the the attacker, v_a, happens to be lower than
v_ss. Unfortunately there is no guarantee of this inequality, and in
particular since there was no explicit estimation of v_ss conveyed to Bob, he
may progressively entrust greater and greater value to the system as his
observation of no successful attacks gives him faith in its security.

The insidious thing about this kind of failure is that the frequency of
successful attack is rare (but each one is costly).  Since it is rare, people
may fail to see the causal pattern of the "DPWYCP" strategy.  Even when
successful attacks are observed, they may be blamed on specific details of
the incident rather than on the security strategy itself.

The second economic loss of "DPWYCP" is that if v_ss >> v_r then unnecessary
expense was incurred by Bob, who paid for the development of the
system. (This is assuming that higher v_ss values cost more to develop.  In
those cases where you can create a higher v_ss at no higher cost, then go
right ahead, but that is not a generally applicable strategy.)

The third economic loss of "DPWYCP" is that if Alice finds it impossible to
make v_ss greater than some vague "very high" value, then she will refuse on
principle to create the system for Bob, even if Bob intends to entrust a
value v_r << v_ss for some v_ss that Alice is actually capable of
delivering. This means that the benefit that Bob could gain from using the
system, v_g, is not gained.  Alternately, It means that Bob goes to a less
principled or more economically minded supplier -- perhaps he gives up on the
"Object-Capability security engineer" Alice and buys from an "ACL security
engineer"...


This brings us to the sticky issues of "prohibiting delegation" which has for
decades bedeviled capability researchers.  I will describe how including
costs in the analysis of the "prohibiting delegation" question changed my
mind about the wisdom of "prohibiting delegation" in certain cases, and how
it also led me to think of new ways to deter delegation.

The standard argument against prohibiting delegation is the "proxying
attack".  It is obvious that regardless of which access control scheme is
used, if Carol is unconfined (and if Carol is human, then we certainly hope
that she is), and Carol has access to a resource, then Carol can share the
resource with her co-conspirator Dan via proxying.  On the other hand, if the
access control scheme is an object-capability access control scheme, and it
offers no mechanism which attempts to prohibit delegation, then Carol could
accomplish the same goal by giving Dan a copy of her capability to the
resource.

Now clearly the cost to Carol of doing the proxying attack is different than
the cost of sharing her capability.  To estimate these costs could be
complicated.  For example, the passage of time might come into play -- Carol
may have access to the resource at time T, and might be able at that time to
proxy it to Dan, but then later at time T+k she may lose access to it while
her capability to it remains valid.  For example, she may suffer a network
failure which makes it impossible for her to proxy the resource even though
her capability to the resource remains valid.  If this case occurred it would
be impossible for her to proxy the resource at that time.  Perhaps the risk
of this situation occurring could be quantified as a cost to Carol.
Schneier's "Attack Trees" [2] are a formalism for quantifying this kind of
security case analysis.

Another attack against the prohibition of delegation is "sharing your
authenticator".  Access control systems are often based on secrets held
uniquely by a user, such as Carol's password, her private key, or a shared
secret key. Carol could give this secret to Dan, enabling him to access the
resource against the wishes of Bob.  This attack obviously imposes some other
kinds of costs on Carol: she cannot conveniently revoke Dan's access after
she has given him the secret, and the secret might provide access to more of
her resources than she wishes to share.

If Alice and Bob can analyze and estimate the cost that Carol would incur in
order to violate a prohibition of delegation -- v_ss -- then it might be
advantageous to build and use a system that prohibits such delegation, as
long as v_r and v_a are sufficiently small and v_g is sufficiently large.


Thinking about these two attacks against prohibition of delegation suggested
to me a novel (as far as I know) technique for deterring delegation: The
capability that Carol holds gives access not only to the resource under
consideration, but also to some resource which Carol does not wish to share
with Dan.  For example, Carol could be required to post a bond of a
substantial sum of money which is held in escrow.  The capability that Carol
is given offers two functions: first it offers access to the resource under
consideration, second it offers the ability to give the bond money to the
holder of the capability, which usage also atomically revokes the capability
so that it can no longer be used to access the resource.

Now Carol may be deterred from sharing this capability with Dan, if she
doesn't want to risk that he will take the money (and also revoke her access
to the resource).  This does not, of course, deter her from proxying the
resource to Dan.

An interesting thing about this technique is that it is effectively what ACLs
use in order to deter the "sharing your authenticator" attack.  A typical ACL
system which offers prohibition of delegation would succumb to a "sharing
your authenticator" attack, but there may be many different resources keyed
by that same authenticator, such as full access to your Unix account, or to
your e-mail, etc., which Carol may be loathe to share.

Another interesting thing about this technique is that it is more general
than the equivalent ACL technique.  One could choose to bind all of the same
resources to the capability as would happen in an ACL system, such as full
access to your Unix account or to your e-mail, or one could choose to bind
less resources, such as partial access to your e-mail, or one could choose to
bind something of even greater value, such as a large monetary bond.

A final interesting thing about this technique is that it is exactly the
opposite of the Principle of Least Authority.  That principle dictates that
security engineers should bind the least authority to each cap as is needed
to make that cap useful.  This minimizes the damage caused by misuse of that
cap. My technique for deterring delegation consists precisely of binding
unnecessary authority to the cap in order to increase the damage it can do
(to the one who originally posted the bond).


NOTE: the notion that "Security is Economics" is obviously not my invention.
It has always been appreciated by security researchers and engineers, not
excluding the capability security theorists, such as Mark Miller who was a
pioneer of economic computation -- "agorics" -- fifteen years, and Chip
Morningstar, the apocryphal author of the maxim "Don't Prohibit What You 
Can't Prevent".

Recently a field of research has sprung up where security and economics
overlap.  See Ross Anderson's "Economics and Security Resource Page" [3] for
starters.

NOTE: I showed this message to Darius Bacon before posting it, and he
suggested that Carol could work around the "bundled bond" by running a facet
on the same host as the proxied resource.  This is a good trick -- if Carol
can run arbitrary code on the same host as the resource under control then
she can "proxy" by running a facet on that host, rather then by proxying the
messages through her own host.  This reduces v_a both in the general case and
in the case of the bundled-bond defense.


Regards,

Zooko

[1] http://www.cap-lore.com/CapTheory/Prevent.html
[2] http://www.schneier.com/paper-attacktrees-ddj-ft.html
[3] http://www.cl.cam.ac.uk/~rja14/econsec.html