std.fileformats?

[Jonathan M Davis]

On Monday, January 6, 2020 12:40:10 PM MST berni44 via Digitalmars-d wrote:
> In Phobos there are several toplevel modules which each are about
> one special file format. Wouldn't it be better to put them all
> into std.fileformats (or some similar place)?
>
> std.base64
> std.csv
> std.json
> std.xml
> std.zip
>
> If yes, there are some more questions poping up:
>
> a) Should we use the change to replace some of the modules? At
> least for std.xml I found std-experimental-xml [1], which seems
> to be thought to replace xml eventually. And there are lot's of
> packages which address json. I don't know if there is one, which
> is *the* candidate for replacement.
>
> b) Should we instead remove some of these? Probably std.zip is
> here the first candidate. (I put some work in it in the last few
> weeks, but it would be fine for me throwing this away.)
>
> It's on a gut level - just wondering, what you think about this.
>
> [1] https://code.dlang.org/packages/std-experimental-xml

Are you talking about putting them all in a single module or all in a single
package? Putting them in a single module would be the opposite of the
direction that we've been going with Phobos over the past few years. We've
generally been breaking up larger modules, not merging modules. So, if we do
rearrange these modules, it would definitely not be by putting them all in
one module. However, even if we moved these modules into a sub-package,
moving them around at this point would arguably just be unnecessary churn.
It would break existing code for minimal benefit. If we replace any of them
with new implementations (e.g. there's been talk of replacing std.xml and
std.json for years now), then maybe they should go in a deeper package
hierarchy, but I really don't think that it makes much sense to simply
rearrange modules. It's also the sort of thing that Walter tends to be
against.

Now, personally, I don't think that anything regarding file formats should
have been in the standard library in the first place. IMHO, that's not the
sort of thing that belongs in a standard library, and they really should
have been on code.dlang.org. However, when they were originally written, we
didn't have code.dlang.org, so they ended up in Phobos. Either way, based on
previous discussions on adding stuff to Phobos, I think that it's pretty
clear that any new, major additions would have to be on code.dlang.org and
battle-tested there before being moved into Phobos, and I don't see any of
these modules being replaced any time soon even if we want to replace them.

std.experimental.xml was a GSoC project that was not completed and is
basically dead. It seems like the original author got too busy with school
and never got back to it. For it to get anywhere, someone else would have to
finish it. It was started with the intention of replacing std.xml, but like
any other major additions, it would still have to go through the Phobs
review process and be voted in.

http://code.dlang.org/packages/dxml might end up in Phobos at some point,
but that's not a fight that I want to fight right now, and the only reason
that I think that it would make sense to put any XML parser/writer in Phobos
at this point is because we already have std.xml, and it really needs to
either be replaced or removed.

http://code.dlang.org/packages/std_data_json was a candidate for replacing
std.json, but IIRC, there was basically too much arguing over what the new
std.json should look like, so Sonke gave up on getting it into Phobos. No
one has tried to get a new JSON solution through the Phobos review process
since then, and I think that for the most part, people have been happy to
just put their stuff on code.dlang.org rather than trying to push anything
through the Phobos review process.

For the most part, I don't see any point in removing any of these modules,
since that would break existing code, and while I don't think that Phobos
should have been implementing parsers for standard file formats, that
doesn't necessarily mean that we should be breaking existing code to remove
them. The primary exception would probably be std.xml, since it arguably
does more harm than good, but there's never really been a consensus on
ripping it out without a replacement being added to Phobos.

BTW, base64 isn't really a file format. It's an encoding. So, even if we
were going to move all of these into a sub-package, std.base64 wouldn't
belong with them.

- Jonathan M Davis