How to open a compressed file in gz format ?

[Steven Schveighoffer]

On 3/14/21 9:36 PM, sharkloc wrote:
> I want to read the content(file.gz) line by line,the following code is 
> not friendly to large files of hundreds of Gb, and the memory overhead 
> is also very large.
> 
> 
> import std.stdio;
> import std.process;
> import std.string;
> 
> void main(string[] args){
> 
>      string fileName = args[1];
>      string command = "gzip -dc " ~ fileName ;
>      auto dmd = executeShell(command);
> 
>      if(dmd.status != 0){
>          writeln("Compilation failed:\n", dmd.output);
>      }
>      else{
>          auto all=chomp(dmd.output).split("\n");
>          writeln(typeid(all));
>          for(int i=0; i<all.length; i++){
>              writeln(all[i]);
>          }
>      }
> 
> }

It's not super-user-friendly, but iopipe excels at this kind of stuff 
(untested):

// dub dependencies: [iopipe, io]
import iopipe.bufpipe;
import iopipe.textpipe;
import iopipe.zip;
import iopipe.refc;
import std.io;

import std.stdio;

void main(string[] args) {
    string fileName = args[1];
    auto lineRange = File(fileName) // open file
        .refCounted // make it copyable
        .bufd // buffer it
        .unzip // unzip it
        .assumeText // assume the binary data is utf8 text
        .byLineRange!true; // true = discard newlines

    foreach(line; lineRange)
        writeln(line);
}

-Steve