[cap-talk] Capabilities and Freedom vs. Safety

[Marc Stiegler]

> Many people on the Hurd list want to maximize "freedom". Until recently,
> I failed to appreciate that they were using "freedom" as a technical
> term. Freedom means "the exemption from control by some other person, or
> from arbitrary restriction of specific defined rights like Worship or
> Speech".
>
> In consequence, many of the Hurd group believe that the user should be
> in total control of their systems. In this view, it should not be
> possible for a vendor to hide their code from the machine owner, nor
> should it be possible for any program to guard itself from inspection by
> the owner. They argue that it is stupid to hide things from someone who
> can scan the drive. I disagree, but their position is consistent.
>
> Unfortunately, the desire for freedom is not perfectly aligned with the
> need for safety. By safety, I mean "the need to preserve and maintain an
> environment that preserves the ability to *exercise* freedom
> consistently and effectively in practice".
>
> The problem is that in a computing system which is totally free, it is
> very easy to take actions whose consequence is that freedom is
> unintentionally lost and/or compromise is assured. Most users are not
> competent to determine which of these actions are which, and some are
> surprisingly counter-intuitive.
>
> If users lived in non-networked worlds, it might be reasonable to
> declare "caveat emptor" and let them hang themselves. We would sell
> fewer systems that way, but there is no ethical problem with that
> position.
>
> In the real world, systems are networked. One consequence is that your
> mistakes have a negative impact on me, in the sense that your
> compromised system becomes a basis for attacking mine. The interaction
> of your freedoms and my freedoms must therefore be considered. In
> consequence, some of the actions you might take on your machine cease to
> be exercises of freedom and begin to be exercises of license: actions
> that undermine the freedom of others or the liberties of society.
>
> Determining where the line lies is a constant balancing act, but I
> believe that it is a responsibility of knowledgeable designers to assist
> in maintaining the balance of freedoms for all users. In consequence, I
> believe that it is ethically obligatory to design limitations into our
> systems to prevent inadvertent or intentional abuse where we know how to
> do so in a way that (on balance) respects the need for legitimate uses.
> A key element of this is creating systems in which naive users can be
> "obliviously safe", and even more so, in which naive users cannot
> undermine the oblivious safety of other, connected users.

The phrase "design limitations into our systems to prevent inadvertent
or intentional abuse" is both where we agree and disagree. Of course,
we should design our systems so that "inadvertent abuse" is unlikely,
even impossible. But one man's intentional abuse is another man's
needed flexibility. As for protecting me from a box on the network
that has been subverted, or is intentionally used to attack me, with a
billion boxes in the world someone is going to do it no matter what
the system designers say, my defense of my box has to be something I
can do without depending on a system designer to modify what other
people can do with their boxes. There is no basis for security in
attempting to control billions of boxes, there is only security in
making my own box secure against the threat. So build the boxes so
they can defend themselves, and don't worry about the other boxes.
Enable me to prevent plan interference from others, don't try to
control the others :-) Your mistakes can't be allowed to have an
affect on me anyway :-)

I would be more sympathetic to the view that we are all our brothers'
keepers, and that the USA needs to keep hit teams on standby for
tracking down Romanian teenage cybercrackers (the eventual necessary
logical deduction of the above philosophy),  if we in this community
didn't already know how to do better than that :-)

--marcs