[cap-talk] Scholarship of P-1935

[Jed Donnelley]

On 2/4/2008 11:07 AM, Jonathan S. Shapiro wrote:
> On Sat, 2008-02-02 at 15:56 -0800, Jed Donnelley wrote:
>> Regarding P-1935 and more generally the TCSEC, I wonder
>> why that process wasn't more 'scholarly' and include
>> wider review.  I wasn't involved, but seeing so many
>> names on it I expect the authors believed those people
>> constituted a sort of "review".
> 
> They were not scholarly because neither was produced by a scholarly
> process

You say that P-1935 was not produced by a "scholarly" process.
That is what I'm getting at.  What more did it need to be considered
"scholarly".  They certainly got wide review by what were considered
to be experts in the field.  One issue I think is that it was
only a draft that got such wide review?  Just from my reading
of P-1935 itself, I believe the authors intended to hold it to
a "scholarly" standard.  How could they have done better?

I ask because I believe such writing is continuing to this
day.  Perhaps we can provide some input to such writing.

> or a scholarly institution. IDA is a think tank whose job is to
> produce position papers.

Hmmm.  There I think you go a bit far.  Just because a paper
is published through something other than a university doesn't
to me mean that it can't be "scholarly".

Independent of the above point, I'm also not sure the term
"think tank" applies to the Institute for Defense Analysis.
Just as with the NSA or the DOE labs or NASA sites or other
government funded institutions, the IDA has a job to do.
They aren't simple a "think tank" as compared to institutions
like (choose your favorites) like RAND or ISI or Hoover
or SRI or Xerox PARC or the old Bell Labs or ...  The IDA
definitely does more than produce position papers.

It seems rather odd to me that the IDA would be chosen to
write this particular report.  I'd be quite interested to
know how IDA and the authors were chosen to write this
report.

Some of what I consider "think tank" organizations I think
should qualify as more "scholarly" than the purpose funded
governmental institutions.  E.g. ISI is closely affiliated
with USC.  Berkeley Lab (where I work) is affiliated with
UC Berkeley.

> TCSEC (and CC) resulted from a process of
> political and technical compromise, and both were explicitly framed as
> attempting to consolidate the existing, well-established state of the
> art.

I think there is certainly a lot of truth in the above statement.

That said, I believe the authors of P-1935 were sincere
in their belief that capability systems as they had been
developed to that point (what they refer to as "traditional"
capability systems) simply couldn't do the job that the
TCSEC required.  In that regard I agree with them.  I know
of no capability system that provides the sort of logging,
auditing, and control over who has access to what that the
TCSEC laid out.  That was their basic issue.  I believe they
were sincere in this and not just trying to "consolidate
an existing, well established state of the art."

On this same topic and reflecting some of the concerns
that Jonathan noted above, here below is a message from a
former colleague of mine at LLNL who was cited as one
of the reviewers of P-1935.  Dan tried to reply to the list
but I believe was blocked because he isn't on the list.
He doesn't want to join, but he indicated that he would
entertain comments on what he wrote if people wanted to
communicate with him directly (e.g. my cc with this
message).

On 2/4/2008 1:33 PM, Dan Nessett <dnessett at comcast.net> wrote:
 > Hello,
 >
 > Jed keeps including me on the CC line of the emails he is sending to
 > cap-talk. So, against my better judgment I will comment on the one
 > quoted below. (NB: I retired in 2004 and now enjoy studying physics and
 > composing music. I am loath to get back into security discussions,
 > simply because in my experience they more closely resemble
 > self-flagellation than progressive discovery. Nevertheless, since my
 > name was used in vain in the acknowledgments given in P-1935, I suppose
 > I have a responsibility to provide some perspective on the processes
 > that generated it).
 >
 > Perhaps it is best to give those who don't want to read much of what I
 > have to say an executive summary of what follows. EXECUTIVE SUMMARY: All
 > resistance is futile.
 >
 > To keep things as short as possible, here are some facts:
 >
 > + P1935 was a technical report from the Institute for Defense Analysis.
 > Technical reports are not peer-reviewed.
 >
 > + While there were a lot of papers published about TCSEC and related
 > topics, the TCSEC standardization process itself was not a scholarly
 > activity. It was driven by a well-funded and politically influential
 > bureaucracy that had decided well before hand what the standard should
 > look like.
 >
 > + The TCSEC (and its progeny, like the Common Criteria [CC]) was
 > interesting to a very small (some might say microscopically small)
 > community - specifically the DoD. It has had almost no effect on
 > products, other than those developed specifically for the DoD.
 >
 > It may be illuminating to expand on the last point by relating my
 > experience with commercial operating systems development. I left LLNL in
 > May of 1994 and went to work for Sun Microsystems on Solaris. My role
 > was to help improve its security. At that time Sun produced a version of
 > Solaris (Trusted Solaris) that targeted the TCSEC. However, the part of
 > the company (the Federal Systems Division) that worked on Trusted
 > Solaris had no (or at least very little) influence on the development of
 > Solaris. The workflow for Trusted Solaris consisted of getting the
 > latest version of commercial Solaris and modifying it so that it met
 > TCSEC standards (I don't remember exactly, but I believe trusted Solaris
 > was evaluated at the C1 or C2 level at that time).
 >
 > Since I left Sun in 1996, I cannot say with any authority whether the
 > workflow for trusted Solaris follows the same model, but I suspect it does.
 >
 > If you make an honest appraisal of reality, you will discover that the
 > TCSEC has had almost no influence on commercial off-the-shelf products.
 > That is because it imposes requirements that arise from a
 > inwardly-focused bureaucracy, not from commercial customers. You can
 > argue all you want that commercial customers should have these
 > requirements, but if you do you are trying to convince the tide that it
 > should not come in. A fundamental rule of selling is to avoid telling
 > the customer he is stupid, you are smarter than he is, and therefore he
 > should listen to you.
 >
 > This email is already longer than I intended when I first thought to
 > write it, so again to save space, here is a summary of some other
 > observations:
 >
 > + Capabilities will never emerge as an access control mechanism in
 > products until the existing mechanisms fail catastrophically and
 > capabilities are required to fix this. The best technical solutions are
 > rarely commercially successful, since it is usually the technology that
 > first makes it to market that wins. When I worked for 3Com after I left
 > Sun, I was exasperated by various academics who kept pushing OCaml as
 > the mobile programming language of choice, rather than Java. It really
 > didn't matter whether OCaml was technically better (and frankly, I
 > doubted it), Java had won the contest. No one was going to pay the
 > retraining, infrastructure and development costs to switch to OCaml when
 > Java basically did everything that customers wanted.
 >
 > + There is a huge existing security ecosystem that has been built around
 > traditional access control mechanisms. In order to even introduce
 > capabilities into existing environments, it will be necessary to figure
 > out how they affect things like firewalls, SSL, IPSec, intrusion
 > detection systems, Kerberos, RADIUS, etc. (this is by no means a
 > comprehensive list).
 >
 > So, while I think capabilities are cool, I am enough of a realist to
 > realize that it is unlikely that they will become common (unless, as I
 > stated above, some catastrophic problem arises that current access
 > control mechanisms cannot solve and it is shown that capabilities can).
 >
 > Sorry for the length of this message.
 >
 > Regards,
 >
 > Dan Nessett

One of my comments in response is that I believe the access control
problem of the Web (e.g. the "mash-up" problem) is exactly the sort
of "catastrophe" that may be able to induce a capability solution
(e.g. Web-keys) into the marketplace.  However, to do so I believe it
needs (or will certainly benefit by) all the security properties
(logging, auditing, and management of who has access to what) that
security professional are used to.  All the things that P-1935 rightly
pointed out that capability systems then and today do not have.  This
motivates my work on this list.

--Jed  http://www.webstart.com/jed/