Allocatoin policy in Phobos - Was: Vote for std.process

[Manu]

On 12 April 2013 23:32, Vladimir Panteleev <vladimir at thecybershadow.net>wrote:

> On Friday, 12 April 2013 at 12:52:39 UTC, Manu wrote:
>
>> If it's somehow hard to put a string on the stack, then there may be a
>> hole
>> in phobos. I'm not suggesting changes that are somehow hard to implement,
>> or obscure in some way... they should be utterly trivial.
>>
>
> Well, ironically or not, it is not something utterly trivial.
>
> The main issue is that the stack can't hold a lot of data. This is not a
> problem with the heap, which is limited by the amount of memory and address
> space; these (usually abundant) limits are usually the user's concern, not
> the programmer's.
>
> Did you know that Linux does not impose a limit on the size of the
> environment? The default stack size seems to be 8MB... Now, what would
> happen if on certain machines that, for one reason or another, have an
> environment larger than that, and std.process did not account for it?
>
> So, to perform the task correctly, std.process would need to perform most
> allocations on the stack if they are up to a certain size, and on the heap
> otherwise.
>
> What would be a good limit for stack allocations? You may want to choose a
> value based on whatever's the default stack size on today's Linux versions
> (after all, std.process is near the "leaf" parts of call stacks). However,
> certain applications create a lot of stacks, for example for use in
> lightweight threads (fibers). When restricted by a small address space
> (32-bit architecture), the stacks need to be much smaller than usual...


Yes, you're right, there's an 'if' required here to catch unreasonably
large environment blocks, but I still consider that within the realm of
'trivial'.
This is processed in an appending loop, just check the next bit fits, and
if it overflows 1kb or so of stack string, revert to the heap and continue.

I reckon helpers could be written to assist with common cases of this
(which would have to be mixin template based I guess?)...
And I really like the variable-length static array idea!

 I highlighted, and suggested trivial changes that
>> would make a big difference and don't hurt anyone.
>>
>
> Well, why do you think they would make a big difference in std.process?
>
> I don't think any of the Phobos developers are against improving
> performance when the cost is low. So, it's not that I think you're wrong in
> general, but that the std.process scapegoat (for lack of better word) was
> not the best choice.
>

Fuck, I've repeated myself so many times now. The point I make is a general
issue I have with phobos, I consider it an issue that should be made policy
(irrespective of module being considered), and std.process came into
question right at the moment I thought to make the point. It may not be the
strongest case for the principle, it's just the one that appeared.

I suggest that you file enhancement requests on Bugzilla for each specific
> component of Phobos / Druntime, improving the allocation behavior of which
> would result in a real-world benefit for you.
>

I'll start doing it myself, but I also suggest it be made a policy, and
carefully considered when considering acceptance of ANY new module. That
way, new code that suffers the unpredictable/"surprise!" allocation
problems won't be introduced.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.puremagic.com/pipermail/digitalmars-d/attachments/20130413/fdc67965/attachment-0001.html>