Scenario: OpenSSL in D language, pros/cons

[Jonathan M Davis via Digitalmars-d]

On Sun, 04 May 2014 21:18:22 +0000
"Daniele M. via Digitalmars-d" <digitalmars-d at puremagic.com> wrote:

> On Sunday, 4 May 2014 at 10:23:38 UTC, Jonathan M Davis via
> Digitalmars-d wrote:

> And then comes my next question: except for that malloc-hack,
> would it have been possible to write it in @safe D? I guess that
> if not, module(s) could have been made un- at safe. Not saying that
> a similar separation of concerns was not possible in OpenSSL
> itself, but that D could have made it less development-expensive
> in my opinion.

I don't know what all OpenSSL is/was doing, and I haven't looked into it in
great detail. I'm familiar with what caused heartbleed, and I'm somewhat
familiar with OpenSSL's API from having dealt with it at work, but most of
what I know about OpenSSL, I know from co-workers who have had to deal with it
and other stuff that I've read about it, and in general, from what I
understand, it's just plain badly designed and badly written, and it's a
miracle that it works as well as it does. Most of the problems seem to stem
from how the project is managed (including having horrible coding style and
generally not liking to merge patches), but it's also certain that a number of
the choices that they've made make it easier for security problems to creep in
(e.g. using their own malloc in an attempt to gain some speed on some OSes).

>From what I know of SSL itself (and I've read some of the spec, but not all of
it), very little of it (and probably none of it save for the actual operations
on the sockets) actually requires anything that's @system. The problem is when
you go to great lengths to optimize the code, which the OpenSSL guys seem to
have done. When you do that, you do things like turn off array bounds
checking and generally try and avoid many of the safety features that a
language like D provides, since many of them do incur at least some overhead.

Actually implementing SSL itself wouldn't take all that long from what I
understand. The main problem is in maintenance - probably in particular with
regards to the fact that you'd have to keep adding support for more encryption
methods as they come out (which technically aren't part of SSL itself), but
I'm not familiar enough with the details to know all of the nooks and crannies
that would cause maintenance nightmares. The base spec is less than 70 pages
long though. The fellow who answered the question here seems to think that
implementing SSL itself is actually fairly easy and that he's done it several
times for companies already:

http://security.stackexchange.com/questions/55465

I fully expect that if someone were to implement it in D, it would be safer
out of the box than a C implementation would be. But if you had to start
playing tricks to get it faster, that would increase the security risk, and in
order for folks to trust it, you'd have to get the code audited, which is a
whole other order of pain (and one that potentially costs money, depending on
who does the auditing).

> > If I had more time, I'd actually be tempted to write an SSL
> > implementation in
> > D, but even if I were to do an excellent job of it, it would
> > still need to be
> > vetted by security experts to make sure that it didn't have
> > horrible security
> > bugs in it (much as it would be likely that there would be
> > fewer thanks to
> > the fact that it would be writen in D), and I suspect that it's
> > the kind of
> > thing that many people aren't likely to trust because of how
> > critical it is.
> Nobody would expect/trust a single person to do this job :P
> Working in an open source project would be best.

If someone around here implemented SSL in D, I fully expect that it would be
open source, and I fully expect that one person could do it. It's just a
question of how long it would take them - though obviously sharing the work
among multiple people could make it faster. Where it's definitely required
that more people get involved is when you want the code audited to ensure that
it's actually safe and secure. And that's where implementing an SSL library is
fundamentally different from implementing most libraries - it's so integral to
security that it doesn't really cut it to just throw an implementation
together and toss it out there for folks to use it if they're interested.

> > Unfortunately, even if
> > something better _were_ written in D, it's probably only the D
> > folks who
> > would benefit, since it's not terribly likely at this point
> > that very many
> > folks are going to wrap a D library in order to use it another
> > language.
> >
> Here I don't completely agree: if we can have a binary-compatible
> implementation done in D, then we would be able to modify
> software to eventually use it as a dependency. I don't see the
> necessary D dependencies as prohibitive here.

If D were to be part of a typical Linux distro, a D compiler would have to be
part of a typical Linux distro. We're getting there with gdc as it's getting
merged into gcc, but I don't think that it's ended up on many distros yet, and
you then end up with the problem of compatabitily, because while D is far, far
more stable than it used to be, it's still pretty common that small things
change or break over time (sometimes by necessity, sometimes due to
regressions), so writing a D library that could be used on a Linx system
with the same compiler version for several years is probably still a bit
unrealistic (though we're getting closer). And without that, even with a solid
C wrapper around it, nothing that's normally part of a Linux distro is going
to use it. At most, you'd get the occasional, smaller project that used it.

And as for Windows, that's not terribly likely to work at this point, because
we only use the MS linker for 64-bit programs, and it's still frequently the
case that Windows programs are build for 32-bit, because the developrs don't
want to have to require 64-bit due to all of the 32-bit copies of Windows that
are still sold.

So, while it's theoretically feasible to have a D library wrapped in C and
then used in C and C++ programs by folks outside the D community, I don't
think that it's actually realistic at this point.

> Thanks for your post Jonathan, however I believe that attempts
> like Go's are what makes a language more real and less
> theoretical. The field test of D on system-level software would
> be in my opinion useful and sometimes I think the sooner the
> better, to eventually modify the evolution of this beautiful
> language (if there's still a viable window for this).

It's mostly an issue of time and man power. I have quite a few projects that
I'd like to work on for myself and for the D community at large, and I only
have the time to work on a fraction of them, if that. And the same goes for
many of the rest of the folks around here. We tend to try and direct our time
and energy to projects that are of greater benefit (which often means not
reimplementing stuff that exists in C and can easily be wrapped - especially
if reimplementing it in D would be time consuming), but regardless of where we
spend our time and energy, we just plain need more people to pitch in if we
want more stuff implemented in D.

Something like SSL is only going to be implemented in D if a member of the
community decides that it's worth it to them to spend the time doing it, and
they have the knowledge and expertise to pull it off. And given all of the
other things that need to be done, I'm not sure that it even makes sense to
spend time on implementing SSL in D as great as it would be to have such an
implementation. We can't do everything, much as we might like to.

- Jonathan M Davis