CTFE and DI: The Crossroads of D

[Adam Wilson]

On Wed, 09 May 2012 19:24:48 -0700, Nick Sabalausky  
<SeeWebsiteToContactMe at semitwist.com> wrote:

> "H. S. Teoh" <hsteoh at quickfur.ath.cx> wrote in message
> news:mailman.510.1336610145.24740.digitalmars-d at puremagic.com...
>> On Wed, May 09, 2012 at 08:06:24PM -0400, Nick Sabalausky wrote:
>>> There's no need for all that.
>>>
>>> The whole point here is "Compile to some obfuscated form" right? So
>>> just make/use a good code obfuscator. Done. Problem solved.
>>>
>>> Inventing an AST storage format just to obfuscate some code is
>>> unnecesary overkill (although maybe it might have some other use).
>>> This "just use an obfuscator" approach even makes the whole DI system
>>> become totally redundant (except for binding to C code, of course).
>> [...]
>>
>> This is an interesting idea. Probably more feasible than mine.
>>
>> You don't necessarily have to throw away the DI system; some people may
>> sleep better at night if their proprietary algorithms are in binary
>> executable form only (though personally I think that's just self
>> delusion, but who am I to judge?).
>
> An ambassador of sanity, that's who ;)
>
> It's not just your personal opinion, it's hard fact: From a
> reverse-engineering standpoint, executable binary form *is* nothing more
> than obfuscation (except perhaps if it's an *encrypted* binary form, but
> I've never heard of a lib that did that, heck it would require special
> toolchain support anyway). Believing binary libs are more secure than  
> that
> is just simply incorrect, period, opinion doesn't enter into it. 2+2  
> *is* 4
> whether you believe it or not. Life isn't looney tunes, you don't walk on
> air just because nobody taught you gravity. Etc.
>
> Fuck if you want to steal something, you don't even need any source -
> obfuscated or not. Commercial games, for example, never release *any*
> source. *Only* the final binaries are distributed, and even *those* are
> usually encrypted. And yet they *still* get pirated like crazy. So the
> source is locked away - fat lot of fucking good THAT did!  (Ok, so it's
> harder to make an unauthorized modification, who the hell cares - the
> *original* is *already* out there getting ripped off, plus why would
> deviants wanna waste time modding when they can just sell bootlegs?)
>
> So source vs binary doesn't make a damn bit of difference, period - if  
> all
> you have is the binary, well, to use it you just *run* it! You don't need
> *any* sources to use it. You just use it. The only thing that can even  
> make
> any difference is encryption (which still isn't truly "secure").
>
> And for that matter, nobody's algorithms are proprietary. Code is
> proprietary. 99.9999999...9999999% of algorithms are not. For example,
> wrapping some action in a foreach to make a batch processor and adding an
> option box to enable it is not a fucking proprietary algorithm no matter
> what the suits and the subhuman USPTO fuckwads think. Real-world example:
> There isn't a fucking thing proprietary in Marmalade's MKB build system
> (it's a stinking *build system* for fucks sake!).
>
> And even for any algos that are proprietary, if such algos even
> exist...well, why bother trying to get the source? If you've got the  
> binary
> already, just *USE* it! Who cares about the damn source? If I wanted to  
> give
> someone access to Marmalade's MKB build system, the fact that half of  
> it's
> distributed in pyc-only does would do jack shit to stop me. And obviously
> there's no proprietary algos in there, again, it's just a fucking build
> system. So there's no algorithms to steal. *Only* thing it does is make  
> it
> impractical for me to work around any problems I encounter. Oh yea, and  
> it
> gives Marmalade's suit-department a big collective stiffy because their
> mini-monkey brains are telling them they're actually *earning* their
> paychecks. (Corona's 50x worse though, FWIW. You don't even *have* their
> software, you just rent the right to send your project to them and have
> *them* build it for you.)
>
> Excess offtopic ranting aside, *some* things are opinion: "Red is the  
> best
> color" <-- That's an opinion. "It is/isn't worthwhile to keep the source
> locked up." <- Even *that's* an opinion, too, note the vauge  
> "worthwhile".
> But merely having different viewpoints doesn't make something opinion.
> Either it's opinion or it isn't. You're not stating mere opinion here -
> you're stating honest-to-goodness FACT: Considering well-obfuscated  
> source
> less secure than compiled binary form *is* delusion, period.

I actually agree with you, im just telling you what I hear from PHB's.

>> So you just take the existing .di
>> files, complete with all their warts and function bodies and whatever,
>> and run an obfuscator on them. Ship the .di and your shared library as
>> usual.  Problem solved.
>>
>
> Or just skip the di entirely. It'll all just get obfuscated one way or  
> the
> other, so there's not much point.

We need some way to export the symbols without the underlying code, it  
makes for faster compile times and having the API handy can be useful to  
development tools.
However, my experience with PHB's is that as long as you don't send out  
the actual source files but some form of sanitized header, the PHB's don't  
really care beyond that.
That'd why I think embedding a version of the source D files that has been  
semantically analyzed could be helpful, you can pull in the source for  
CTFE as needed, but the only thing you have to actually ship out is the  
library file itself, it just happens to have source files inside. In my  
experience in the .NET world, this is good enough for the PHB's. Out of  
sight, out of mind as they say. So what if it's trickery, we developers  
get a benefit to, we don't have to wrangle include files.

>> Plus, all of this is already possible with the current system, except
>> for the only missing piece of a D obfuscator.
>>
>
> DustMite has some obfuscation capability, although I don't know how
> extensive it is or whether it would be enough make the pointy-haired  
> suits
> happy. (Then again, *anything* can be enough to make a suit happy - you  
> just
> have to present it in the right salesmany (read: "convoluted and full of
> shit") way. They'll swallow any amount of bullshit you give them, you  
> just
> have to make it *sound* good. Suits don't know the difference. Fuck,  
> most of
> them don't even know there *is* a difference. That's why salesmen exist -
> because bullshitting WORKS on suits (and on many others), in fact, most  
> of
> the time, it's the *only* thing that works on suits. Bullshit is the only
> language those fuckers speak and understand.)
>
>> (Wait, I hear you cry. But what about the library API? How would users
>> know how to use the library if the .di is incomprehensible?  As somebody
>> pointed out in another thread: just ship the ddocs generated from the
>> unobfuscated source with the library. Users don't need to read the .di.
>> Problem solved.)
>>
>
> Yea, the signatures (not even the definitions) of the symbols which make  
> up
> the public interface are the only parts that must remain non-obfuscated.  
> But
> of course, those *still* need to be non-obfuscated even under the
> old-fashioned C-style "lib + headers" approach. Don't even need any  
> markup
> to signal these "don't touch" symbols to the obfuscator - just make a  
> series
> of wrappers in separate files for the public API and tell the obfuscator
> "don't obfuscate the signatures in files x, y and z."
>
>


-- 
Adam Wilson
IRC: LightBender
Project Coordinator
The Horizon Project
http://www.thehorizonproject.org/