[cap-talk] Security by safe language processing

[Jed Donnelley]

At 05:06 AM 9/5/2009, Ben Kloosterman wrote:

<snip>

> >I know of very few systems that depend on language processing for
> >security.  One example that I'm aware of was the Burrough B* systems
> >(B5500, B5700, B6500, B6700).  These systems depended on their
> >compliers for security.  One problem with this approach is that if
> >there is a bug in a complier that generates unsafe code at any
> >time and one can save that code, then it is unsafe for as long as
> >it can be saved.  I "owned" a B6700 system while I was in college
> >because it briefly had a bad compiler that I used to generate unsafe
> >code that I was able to save.
>
>The same can be said with non type safe protected memory OS's.

I don't agree, so perhaps some discussion of this topic is worthwhile.

>I worked
>with one around 1990 and there was a compiler bug in an obscure piece of
>code. However under conditions of heavy load and the user using the floppy
>drive ( pretty rare in a multi user system) the system would get a stack
>corruption to a sys call lose a field on the stack and would write the
>floppy FAT on top of the HD fat. Took like 12 months to find and lots of
>angry customers with trashed drives.

In the above when you say "there was a compiler bug in an obscure piece of
code" I take it you mean the bug was in an obscure piece of the security
kernel?  Of course such problems can be difficult to find and fix.  However,
once found and fixed then the system is at least safe from that bug.

Short story: I'm currently working for a .com.  We run 8 load balanced Linux
Web servers.  About two weeks ago one of them panic'ed.  This problem was
traced to an exploitable kernel bug that had been announced just the week
before.  A patch was made available.  We updated our kernels.  Now we
are presumably safe from that bug/exploit.

Now the hypothetical.  Suppose there is a bug in a compiler that's considered
part of the security perimeter of a system.  Or if you prefer there is a
procedural lapse that allows an unsafe library routine to be stored on
the system (the situation I'm familiar with).  Somebody exploits the lapse and
generates an unsafe library routine that they save - say on disk and on
other systems.

The first time the bug is exploited maybe somebody could get ahold of
a system dump and find the user code that has the problem.  At that point
what do they do?

In today's systems it's assumed that binary code (loadable libraries
or full executables) can come from anywhere.  Even if some binary or
library on the system is fixed, anybody could file transfer a binary
into the system with the exploit and exploit it again.

What I don't understand is the approach that will be used to avoid
this problem.  Do you have to scan all binaries (possibly byte code)
to insure that they are safe?  Perhaps every executable that is run
on the system must be compiled there?  This of course has it's own overhead
that can be significant.  Is there another approach?  Please explain
how systems that depend on language processing for their security
can deal with this issue.

>Agree it is a pretty new field , everything since the 70s has been Multics
>based but there were some interesting systems earlier there was an ALGOL
>based system I think.

The ALGOL based systems you refer to were the Burroughs systems that I
mentioned in my previous message and is copied above.  I still consider
myself an expert on those systems, so I would be happy to discuss their
shortcomings with you if you're interested.  As I mentioned, with a
simple library routine (essentially "execute this array") that I was
able to generate and save when an unsafe compiler was temporarily
made available by mistake (so-called "Communications Algol" that was
used for systems programming) I effectively came to own the system
that I was using and any other such system that I could transfer that
binary routine to.

At least in that context I considered this to be a fatal flaw in the
security architecture of the Burroughs systems.  To accept security by
language processing I would need to at least understand an approach
that protects against what I view as an obvious flaw in that
architecture.

> >If you use this approach of protection by "safe" language processors,
> >how do you save and later execute such processed (e.g. binary/byte) code?
> >Is code that's considered safe at one time still considered safe
> >at a later time?  If it's considered safe at one time but mistakenly
> >isn't, how does this problem get corrected?
>
>The compile is done as part of the install process of the application. There
>is like no research but I was planning on something like if its compiled it
>gets written to a non writable file that can only be altered ( deleted) by
>the uninstaller.

Ah.  Now we may be getting down to it.  Do you assume then that applications
can't be imported in an executable form (e.g. over the network)?  You
understand I'm sure that this approach would be considered to at least
provide a less flexible execution environment than "traditional" kernel
based systems with hardware protected context switches.

>A file capability execute can be used for this file ( and
>file w undelete).    Sure you can have compiler bugs but these are MUCH
>rarer these days , and 99% lead to the application not executing/crashing
>,one of the compilers under development has 5000 unit tests so far.
>Traditional systems also have some of the same issues with compiler bugs (
>especially for the Kernel) .

As you can see from my example above, one issue that I'd want to understand
is whether there is a need for some code to be generated that is otherwise
"unsafe" - e.g. to execute protected system operations (e.g. build
capabilities, context switch between methods - pick your favorite
terminology).  If so, how is such code compiled?  If not, how are such
security sensitive operations performed?

>It is also worth stating that the compilers tend to be simpler than language
>compilers  eg the developer uses the existing C# compilers  (Mono or ms) to
>generate the CIL ( MISL) the actual compile we are talking about just
>converts the Assembler like intermediate code into x86  .Its trivial for
>unoptimized code there is some complexity for optimized code  but you don't
>get all those weird issues you get with say a c compiler. It is also worth
>noting in 9 years working with C# I have never heard of a compiler bug , the
>chances of such a big existing and leading to a security breach are
>exceptionally low.

My.  "in 9 years working with C# I have never heard of a compiler bug"
That's a pretty amazing record.  Still, as you can see from my example,
even with no bugs in the "safe" compiler, security problems with binary
(or I expect byte) code are possible, unless such code is somehow scanned
before being executed.

>It's worth comparing a bug with the compiled code in a traditional system
>like I mentioned above , the solution is to rebuild the files affected and
>redistribute it . With a compiler bug in a managed system you would  patch
>the compiler ( the same as the OS)  , uninstall the app and re-install.
>Your probably should add another state to this eg   Install -> Install and
>Compiled ( Install state)  ->  Patch Compiler -> Install  ( by the
>de-installer just deleing the compiled versions) ->Install and Compiled ( by
>recompiling with the new compiler from the original installed assembly/byte
>code ). As you can see programs are much more tightly managed as well with
>meta data describing requirements .

I think once I better understand the limitations on how executable applications
can be generated and introduced into the system for execution I will have a
better idea of where the security boundary is.

 From the above I assume that applications must be compiled on the system?
Presumably they can only be compiled and loaded from safe executables on
the system?  If somehow an unsafe library routine (e.g. my "execute this"
routine) gets on the system, I guess that it will be unnoticed until
exploited.  Are you arguing that once it is exploited that analysts would
have about the same chance to find the problem as with a kernel bug in
protected context switch system?  That is, they would dig into what happened,
ultimately finding the unsafe code - perhaps loaded into an executable
application.  From there they would track the code (good accounting for
loaded executables would be needed here) until the library routine at
fault was found.

At that point the library routine at fault would be removed and any
executable applications that had been (statically) linked to it would
also be removed (?).  Such a procedure could well be a bit traumatic,
but I can imagine it solving the problem.

Fundamentally I think what you are depending on is that there is no
way to import already compiled and vetted code into your system that
is made safe by language processing.  Is that right?

--Jed  http://www.webstart.com/jed-signature.html