[cap-talk] The Limits of POLA's Utility - Social Engineering

[Toby Murray]

Hi cap-talk,

for anyone interested in the utility of POLA in the face of social 
engineering.

The Computing Lab at Cambridge published Tech Report 666 yesterday 
(6/6/06).
"A Pact With The Devil"
Mike Bond, George Danezis
http://www.cl.cam.ac.uk/TechReports/UCAM-CL-TR-666.pdf

(gotta love that style)

I haven't yet finished reading this paper but the following quote seems 
to summarise its direction:

> Our key contribution, following recent work on the economics of 
> information secu-
> rity [1], is to demonstrate that malware can provide enough incentives 
> to users for them
> to willingly maintain it on their systems, and can again provide in 
> the medium-term
> enough disincentives to them removing it. Users can therefore enter in 
> “a pact with the
> devil” that confers on them some powers, that the virus shares with 
> them, but as they
> soon realise, some heavy responsibilities too. Not surprisingly, it is 
> the darker human
> traits that such malware seeks to foster and exploit – greed, 
> curiosity, need for power,
> fear, shame, lust, to name but a few. 

This appears to clash with an idea that I heard on this list once, from 
Marc Stiegler. (Quoted below). When I read it, it struck me quite 
strongly (enough that I wrote it down at the time). I think its very 
true but that the work in this paper seems to present a good counter 
example.

>In fact, humans embody a wild mix of things they do well (fine grain
>distinctions of who to trust with what little piece of authority) and
>what they do poorly (memorize long random strings of characters). We
>will not achieve any real secure cooperation until we start relying on
>human beings to do the things they do well, and stop demanding people do
>the things they do poorly. Trying to programmatically do what people do
>more flexibly and more insightfully on their own kills cooperation;
>trying to depend on people to do what they do poorly kills security.
>
In the Cambridge paper (that like all social engineering attacks) 
they're actively exploiting vulnerabilities in the user. In this case 
the exploitation of these vulnerabilities undermines the user's ability 
to make "fine grain distionctions of who to trust with what authority". 
In the case of this virus, its also written such that it requires very 
little authority to be effective. (I explain further below). As a 
result, I think this sort of thing might severely lmit the potential of 
full POLA (eg. as provided by any capability system) to defend against 
this sort of virus. I've long held to the view that POLA would largely 
eliminiate the threat of viruses/trojans etc. However, this paper is 
making we wonder whether this view applies to viruses (or any attack) 
that employs social engineering. I'm not yet sure of its implications so 
I thought I'd see what others think here.

Bond and Danezis talk about a Satan virus that uses a carrot and stick 
approach with the user, roughly: "I'll give you something good if you 
install me". "I'll do something bad to you if you try to remove me".

An example is given, which I've quoted (part of) below:

> For this instantiation, we will use access to another user’s files as the
> carrot, and revelation of this access to the party spied upon as the 
> stick. Assume there
> are three parties: Alice, Bob and Charlie. Alice is already infected 
> with the virus, and
> Bob and Charlie are related to her (employees, colleagues, friends or 
> family). The virus
> propagates in the following manner:

> 1. Temptation. 

> The virus sends an email from Alice to Bob, offering access to all of
> Alice’s emails and documents. To make the offer more enticing, 
> extracts from these
> documents containing Bob’s name, or other interesting keywords can be 
> included.
> Bob can chose to accept this offer, by downloading the virus (that can 
> be hosted on
> Alice’s computer or bundled in the email) and executing it. As a 
> result he should
> have full access to Alice’s documents, with a search interface to help 
> locate files of
> interest.

> 2. Monitoring. 

> As soon as the virus has installed itself, it starts recording everything
> that Bob does, and in particular the accesses to Alice’s information. 
> Crucially, this
> includes the search queries performed as well as logs of the documents 
> retrieved.
> This information is sent back to Alice or another infected third party 
> (that can be
> known through Alice) for safekeeping, but it is not revealed. The key 
> intuition is
> that the virus avoids the hard problem of automatic detection of 
> ‘blackmail’ material
> on Bob’s computer, by collecting evidence on the unsavoury act of 
> spying that it
> has tempted Bob to commit. The unauthorised access to Alice’s 
> computer, both in
> the files Bob views, and the search terms he uses (revealing his 
> suspicions of Alice)
> should in most cases be incriminating material.

> 3. Blackmail. 

> When a critical mass of incriminating evidence of unauthorised accesses
> from Bob to Alice’s machine has been gathered, the virus emails Bob 
> with a warning.
> The warning specifies that if an attempt is made to remove the virus 
> the information
> gathered will be revealed. A snippet of the information can also be 
> provided to
> substantiate the threat. To safeguard the virus against retaliation, 
> it sets up a
> life-line between Bob and Alice’s machine (or a compromised third 
> party holding
> the incriminating evidence), to monitor Bob’s computer, and ensure 
> that it remains
> infected. If Bob’s computer does not appropriately respond, the 
> evidence is released.


I think this would succeed even if Bob's machine implements full POLA 
(ala CapDesk). Bob accepts the offer of the virus, giving it access to 
the network to contact Alice's machine, and thereby supply him access to 
Alice's files. The virus provides Bob access to Alice's files via a 
proxy that records Bob's accesses. POLA has not been violated. The virus 
need not have access to any of Bob's authority except the authority to 
access the network (which Bob *wants* to give it because Bob wants to 
get access to Alice's files).

The virus has exploited Bob's human weaknesses, thereby corrupting his 
ability to make good trust decisions. The provision of POLA hasn't 
helped Bob.
Of course, it had to infect Alice originally, which required her to give 
it a lot of authority. This could perhaps only occur if Alice is running 
an ambient authority system. The virus also has other stages (that I 
haven't described) that might not work at all since Bob is running a 
full POLA environment, but these stages appear like they'd still work 
regardless.

Is this sort of thing the limits of what POLA can achieve for virus 
prevention?

One way to guard against this would be to affect Bob's assessment of the 
virus. If the virus hasn't been introduced to Bob from someone he 
trusts, then he would be less likely to trust it. The fact that it's 
come from Alice (who he might trust) might make him trust it more. Of 
course, it is acting on its own behalf, not Alice's, since Bob knows 
Alice doesn't want him to access her files. Hence, the trustworthiness 
of the introducer seems irrelevant, since Bob can infer that it's not 
Alice introducing him to this virus, but the virus introducing itself to 
Bob using Alice as a condiut.

I certainly don't expect this to be the limit of countermeasures we 
could deploy against this sort of thing though. There must be more ways 
to try to defend against this thing. Anyone with more experience with 
full POLA environments (such as cap systems) like to comment on how 
effective they think this sort of thing would be in those environments?

much thanks, as always
Toby

-- 

Toby Murray
Advanced Computer Capabilities Group
Information Networks Division
DSTO, Australia

IMPORTANT: This e-mail remains the property of the Australian Defence
Organisation and is subject to the jurisdiction of section 70 of the
Crimes Act 1914. If you have received this e-mail in error, you are
requested to contact the sender and delete the e-mail.