On the D Blog: Language Plasticity is More Important Than Ever
monkyyy
crazymonkyyy at gmail.com
Tue Aug 11 01:16:16 UTC 2026
On Monday, 10 August 2026 at 22:35:46 UTC, Adam Wilson wrote:
> On Monday, 10 August 2026 at 13:41:59 UTC, monkyyy wrote:
>> If that where the numbers come from, they are suggesting
>> deleting whitespace; grouping 4-spaces as 1 token is a "gpt-2"
>> level technology.
>
> There has been a lot of work done at the harness and ingestion
> layers to reduce input token usage precisely because
> whitespace-sensitive languages are so token hungry. So yes,
> they've gotten the input token usage down, but by improving
> tooling, not because the language itself is less token
> expensive. And there is evidence, albeit in early stages, that
> such schemes are increasing the amount of mistakes that the LLM
> makes on output and thus creating more iterations over the
> problem. Which increases total token usage.
Python fucking sucks and ai labs are stupid for putting do much
python context into their magic strings. But your comparing d to
a strawman, 4 token per indent python as far as I know never
existed as 2019 chatgpt would`nt`ve; and they could go further
and encode it with the exactly the same tokens `new line; tab++`
`new line; tab--` they havnt yet, but they can at which point it
would compete with `{` with removed white space
If they put in "allot" of work adding in their `8 space` token
taking years, they are shit at it(they are shit engineers so
totally possible); what they put allot of work is changing the
names every year and marketing a new way to concat strings.
This however is not nearly good enough to make d competitive;
with every possible language. *Ive never seen a good line of
template code from an ai*, the main reason to use d; purely
hypothetically someone could be well positioned to take my
algorithms and data structure code fragments and spend tokens
making them cohesive into a nice neat package maybe someone could
figure out how to get good template code out of ai with such an
auto research task.
More information about the Digitalmars-d-announce
mailing list