std.fileformats?

[Steven Schveighoffer]

On 1/6/20 10:48 PM, Laeeth Isharc wrote:
> On Tuesday, 7 January 2020 at 01:45:21 UTC, Steven Schveighoffer wrote:
>>
>> Except you do need to depend on it to do basic things. Have you tried 
>> using std.xml?
> 
> Yes, and I didn't persist for long.
> 
> I wonder if sometimes there can be a tendency to let the best be the 
> enemy of the considerably better.  I think you made an argument for 
> replacing std.xml rather than XML not being in the standard library.  
> Though maybe that's a better argument than for getting rid of JSON and CSV.

Yes, for sure you can have a "basic" implementation for something, and 
then go elsewhere for more fancy implementations. Unit testing is a good 
example of this, the default works, and runs unit tests, but if you want 
fancy outputs, you go with one of the 3rd-party tools. The nice thing is 
that the runtime library allows you to swap out it's dull minimal 
implementation with your fancy implementation.

xml and other file formats are not like that. First, there isn't a 
requirement in the library to have xml anywhere -- it's its own project, 
and simply lives in Phobos, nothing else in Phobos depends on it. 
Because it is of poor quality, we should replace or remove it. Of 
course, there are many ways to do xml, and it's hard to agree on the 
"right" way, which is why it sits there basically untouched for over a 
decade. It's also hard to make a similar mechanism like the previously 
mentioned unittests where you can "swap out" the implementation with 
what you really want.

It fits better as an add-on library. It doesn't have to be part of 
Phobos to be maintained by a quality team. It's just kind of tacked on, 
mostly because other language libraries (C#, Java, etc) at the time had 
an xml package, and we were doing what they did. Note that the bare 
minimum has been done to it, just to keep it building. I don't think 
that's desirable, and it's not a good look for the language.

Not only that, but there is a cost to having a poor library being the 
"default" one. People do not go looking right away for something else, 
so they waste time on it, and finally go elsewhere anyway (perhaps away 
from D not even knowing there is something better for it). I'd say we 
are better off NOT having it in there.

>> Because something lives on code.dlang.org does not mean it is 
>> unreliable. The reliability depends on the developer, not the platform.
> 
> Yes, I agree, but people are wired different ways and learnt to program 
> in different eras and the cognitive cost of figuring out if a 
> code.dlang.org library is any good is much greater for some people than 
> others.  You don't even know if the project will build with the current 
> release of DMD.  That mostly doesn't bother me personally, but it 
> definitely does bother others.

I definitely think we should have a repository system that separates 
"blessed" and maintained projects from all the ones that are added by 
random people. Maybe like a certified project list, and in order to be 
on it, the project has to be maintained by the core team (and tested 
along with the core CI), or there has to be a promise to fix issues 
within a certain time period or whatnot.

Look at boost for example -- not part of the standard library, but might 
as well be. It lives in a space where innovation needs to move faster 
than standards committees.

D isn't in the same boat exactly, but we have much less leeway to make 
significant changes to Phobos packages than we would to make changes on 
external ones. It's also difficult to say "this is the one way you 
should use xml", when nobody is passionate enough about it.

> 
> "The reliability depends on the developer, not the
>> platform."
> 
> I don't agree with this.  Phobos will probably work on BSD or smartos or 
> whatever.  And quite well-written libraries may require tweaking to 
> build even on 32 bit Windows.

This is due to the extensive CI system we have on there. This doesn't 
stop other projects from doing the same.

What I meant was that code outside of Phobos can be just as reliable.

> And when you have a reasonable number of dependencies there is always 
> something breaking with new releases so the cost may not be trivial for 
> larger projects.

If we have the dependencies all covered, then there is not an issue. In 
other words, we could maintain a list of projects where all the 
dependencies are up to date.

> 
>> I imagine vibe.d is going to be super-reliable for years, and it lives 
>> on code.dlang.org.
> 
> Maybe.   I am very grateful to Sonke for his contribution but vibe is 
> one of the projects that breaks frequently and you can get into a mess 
> where you need newer versions of the compiler for other reasons and vibe 
> doesn't compile.  Plus dub often can bring in dependencies spuriously, 
> though that's a different problem.

I haven't had this experience. When I've updated for my project, the 
vibe framework seems to be very good about warning me with deprecations 
rather than breaking.

>> There's something to be said about having things outside the standard 
>> library not for the reasons of stability but for maneuverability. One 
>> can easily add/improve/change projects that are outside the standard 
>> library, but poor decisions in the standard library are sometimes 
>> stuck there (see std.xml).
> 
> Yes I agree but isn't that an argument for adopting a better one as part 
> of the standard library?

But which one? Nobody can agree on what to put in there, so it stays. 
XML is not easy. JSON is easy (I wrote jsoniopipe in 1 week I think), 
and it is still in there despite other better systems existing.

> Undead could also be more widely publicised and dub could be educated to 
> ask if you want to fix old code by updating references.

You mean move it to undead? I'm fine with that, and I'm also fine with 
dub being proactive about projects that depend on it. But that doesn't 
mean we have to replace it.

>> For file formats, or parsers, or really any kind of system that has 
>> innumerable ways of solving the problem, I really think the standard 
>> library is not where they should be. The benefits are cohesion with 
>> the library, and usage by the library. But there are many ways to skin 
>> the JSON cat. Especially when there's little reason for JSON to be 
>> included in Phobos other than a place for it to be.
> 
> A sensible default doesn't stop you from using something suited to your 
> particular needs.  It also saves time and cognitive effort to know what 
> to expect when reading code.

The standard library should contain a minimal or best implementation of 
things that are core to the language. I don't think it needs a default 
implementation of everything.

At one point we were looking at putting all sorts of things in there 
(std.database, std.io, std.kitchensink). I just think the advent of a 
build system that can pull in dependencies mitigates a lot of that need. 
It brings with it other problems, but if we can mow that grass down so 
you aren't wading through a forest, maybe it's not so bad.

-Steve