Splitting up large dirty file

Dennis dkorpel at gmail.com
Wed May 16 07:06:45 UTC 2018


On Wednesday, 16 May 2018 at 02:47:50 UTC, Jon Degenhardt wrote:
> Can you show the program you are using that throws when using 
> byLine?

Here's a version that only outputs the first chunk:
```
import std.stdio;
import std.range;
import std.algorithm;
import std.file;
import std.exception;

void main(string[] args) {
	enforce(args.length == 2, "Pass one filename as argument");
	auto lineChunks = File(args[1], 
"r").byLine.drop(4).chunks(10_000_000/10);
	new File("output.txt", "w").write(lineChunks.front.joiner);
}
```

dmd splitFile -g
./splitFile.exe UTF-8-test.txt

std.utf.UTFException at C:\D\dmd2\windows\bin\..\..\src\phobos\std\utf.d(1380): Invalid UTF-8 sequence (at index 4)
----------------
0x004038D2 in pure dchar std.utf.decodeImpl!(true, 0, 
char[]).decodeImpl(ref char[], ref uint) at 
C:\D\dmd2\windows\bin\..\..\src\phobos\std\utf.d(1529)
0x00403677 in pure @trusted dchar std.utf.decode!(0, 
char[]).decode(ref char[], ref uint) at 
C:\D\dmd2\windows\bin\..\..\src\phobos\std\utf.d(1076)
0x00403575 in pure @property @safe dchar 
std.range.primitives.front!(char).front(char[]) at 
C:\D\dmd2\windows\bin\..\..\src\phobos\std\range\primitives.d(2333)
0x0040566D in pure @property dchar 
std.algorithm.iteration.joiner!(std.range.Chunks!(std.stdio.File.ByLineImpl!(char, char).ByLineImpl).Chunks.Chunk).joiner(std.range
.Chunks!(std.stdio.File.ByLineImpl!(char, 
char).ByLineImpl).Chunks.Chunk).Result.front() at 
C:\D\dmd2\windows\bin\..\..\src\phobos\std\algorithm\iteration.d(2491)


More information about the Digitalmars-d-learn mailing list