[cap-talk] Building a bridge: library API's and file descriptors?

Marcus Brinkmann marcus.brinkmann at ruhr-uni-bochum.de
Wed Feb 4 09:36:25 EST 2009

Rob Meijer wrote:
> The solution seems simple, get the library vendors to extend their API to
> accept file descriptors. File descriptors can be communicated between
> unconfined and confined processes like capabilities, allowing applications
> to confine their untrusted library usage if these libraries have an API
> that accepts file descriptors instead of or next to paths, network
> addresses and URL's.

Ulrich Drepper did some work towards this by extending the Linux kernel API
to add file-descriptor based interfaces for some POSIX calls which didn't
have them, for other reasons.

In GPGME, we offer a filename based interface and a file descriptor based
interface, plus some more (memory, FILE *, callbacks, glib channel, qt channel etc).
So I have some experience with that.  Let me give you a bit of a head start here,
and list some potential problems in following this approach.

A file descriptor is a complex object.  There are different types of
descriptors (file, pipe), with different guarantees.  It comes with important
state, for example if operations should block or not, and if the descriptor
is inherited or not.  So, in addition to accepting a file descriptor, it may
also be necessary to inspect and verify it.  Lots of potential for subtle bugs,
for example if the user of the library shares the file descriptor or otherwise
mishandles it.

In addition, on systems with a LARGEFILE interface, there are two types of
file descriptors, 32-bit and 64-bit file descriptors.  This can lead to very
subtle errors when dealing with files over 4 GB large.

On POSIX, there are two file interfaces, one based on file descriptors and
one based on FILE * pointers.  They can be converted into each other, but not
without some effort and additional hazards (see GNU C library manual, 13.5
"Dangers of Mixing Streams and Descriptors").

When porting the POSIX application to Windows you face the problem that there
is no standard file descriptor abstraction in Windows.  The porting environment
like MingW offers a C library file descriptor, while the  standard descriptor
type in Windows is in fact yet another type, a HANDLE.  The real fun starts when
you want to share those across process boundaries, for example to a child process,
in a controlled manner.

Luckily, a programming environment like glib will always provide its own I/O channel
abstraction, so that you can replace "file descriptor" with "glib channel" or
some other suitable object, and get some approximation to a Unix file descriptor
on other platforms as well.

So, if you try to convince library authors to extend their interface, be prepared to
get some resistance based on the above problems and surely others...

> File descriptor passing to these processes turns least privilege int least
> authority. Changes to libraries seem trivial, but how do we convince
> library vendors of the large merits of extending their API's in this
> relatively trivial way?

As a library author, I could be convinced if you show me a relevant use case where
the interface is necessary, but only if the support cost is acceptable.  The support
cost for exposing system data types in the library interface is relatively high, if
the library needs to be portable.  A use case is relevant if it is based on some significant
real world application, as opposed to be based on abstract reasoning or a research project,
and if it is within the scope of the library.


More information about the cap-talk mailing list