Allocatoin policy in Phobos - Was: Vote for std.process

[Dmitry Olshansky]

12-Apr-2013 18:40, Manu пишет:
> On 13 April 2013 00:18, Dmitry Olshansky <dmitry.olsh at gmail.com
> <mailto:dmitry.olsh at gmail.com>> wrote:
>         Believe it or not, I'm not actually a fan of over-complexity.
>         And I'm
>         focusing here on totally unnecessary allocations.
>         Isn't using the stack where applicable just a whole low easier?
>         That's
>         what it's there for.
>
>
>     'cause nobody can tell you how big it is. This knowledge is only
>     available to end user and there is still no easy way to "tell" that
>     to the library. The end result is utterly useless as library can't
>     reliably use stack space.
>
>
> Filenames, strings, etc have a fairly predictable size. Allocate
> slightly above that, and fallback to the heap upon overflow.
> I think it's safe to assume the stack is 'big enough' if you apply some
> common sense.

Up to 32K on WinNT. Hence the 32K reference.

> It measures in the megabytes on PC's, I'm used to working on machines
> with ~32k stack, I use it aggressively, and I don't tend to run out.

Just a moment ago you were arguing for some quite different platform :)

Regardless - think fibers on say servers. These get no more then around 
64K of stack. In fact, current D fibers have something like 16K or 32K.
There even was report that writeln triggered stack overflow with these, 
so the size was extended a bit.

In general simple non-GUI threads on Windows get 64K by default (IRC).

>
>     In the end if one library thinks it's fine to burn say 32K of stack
>     with alloca but then it calls into another one that burns another
>     chunk with alloca and so on untill happily stack overflowing
>     (sometimes, on some systems at the right time!). Call graphs to
>     calculate this is only available for the final app.
>
>     Maybe just adding a separate thread-local growable stack for data
>     would work - at least it wouldn't depend on sheer luck and
>     particular OS settings.
>
>
> If you're saying the stack is a limited resource, therefore it's unsafe
> to use it, then you might as well argue that calling any function is an
> unsafe process.

s/limited/unpredictably limited/

"calling any function is an  unsafe process" - indeed in principle you 
don't know how much of stack these use unless you measure them or 
analyze otherwise.

It's awful that on 32 bit system you can't expect stack to be 
arbitrarily long (as much as you'd use of it) due to threads quickly 
using up all of virtual memory. On 64-bit something to that effect is 
achievable.

> Some common sense is required.

Exactly except that in the library there is no knowledge to get a 
measure of "common sense". It can't tell how somebody intends to use it, 
especially the standard library.

 > I wouldn't personally burn more than 1k
 > in a single function, unless I knew it was close to a leaf by design
 > (which many library calls are).

And you trust that nobody will build a ton of wrappers on top of your 
function (even close to leaf one)? Come on, "they" all do it.

> 90% of what we're dealing with here are strings, and they tend to
> measure in the 10s of bytes.
???
Not at all, paths could easily get longer the 256, much to legacy apps 
chagrin.

-- 
Dmitry Olshansky