Conspiracy Theory #1

[retard]

Thu, 19 Nov 2009 22:27:34 -0700, Travis Boucher wrote:

> Sean Kelly wrote:
>> Performance per watt is a huge issue for server farms, and until all
>> this talk of low power, short pipeline, massively parallel computing is
>> realized (ie. true "cloud computing"), systems languages will have a
>> very definite place in this arena.  I know of large-scale Java projects
>> that go to extreme lengths to avoid garbage collection cycles because
>> they take upwards of 30 seconds to complete, even on top-of-the-line
>> hardware.  Using a language like C remains a huge win in these
>> situations.
> 
> This I agree with to a certain degree.  This really only applies to
> colocated systems.  Shared hosting situations, users are often too
> stupid to understand the effects of crap code, and shared hosting
> providers tend to over commit machines.
> 
> Then comes in the virtualization providers, Amazon EC2 being a perfect
> example.  As long as income is greater then costs, EC2 users rarely get
> their code running as well as it could, even tho they'd see the most
> direct cost savings from doing so.  With today's web languages, the cost
> to make something efficient and fast (and maintain, debug, etc) is
> higher then the cost to run slow crappy code.
> 
> This is amplified by the loss of money in an emerging market where
> coming out even a month after your competitors could mean your death.

If you're not developing web applications for the global audience, 
performance rarely matters. And it's a bit hard to compete with huge 
companies like Google or Amazon anyways so there's no point in trying to 
do that. The target audience size is usually something between 1 and 
100.000 here and most companies are smaller startups. In larger companies 
you typically write proprietary intraweb enterprise apps for commercial 
users (with usually less than 10000 clients).

Analyzing gene expression data etc. are really small niche markets. 
Usually the application users are experts of that domain within the same 
company (so the amount of concurrent users is low). Most web programming 
deals with simple pages with CRUD functionality, suboptimal database 
access and lots of hype. The site structure is becoming so standardized 
that soon you don't even need real programming languages to build one.

> Yes, the days of multi-cpu, multi-core, multi-thread hardware is here. I
> recently got a chance to do some work on a 32 hardware thread sun
> machine.  Very interesting design concepts.

What makes programming these machines rather simple at the moment is that 
they're mostly good at task parallelism. Very fine grained parallel 
algorithms aren't that useful in general in commercial use.