Read non-UTF8 file

Nrgyzer nrgyzer at gmail.com
Sun Feb 13 13:49:14 PST 2011


Hey guys,

I've the following source:

module filereader;

import std.file;
import std.stdio : writeln;

void main(string[] args) {
	File f = new File("myFile.ext", FileMode.In);
	while(!f.eof()) {
		writeln(convertToUTF8(f.readLine()));
	}
	f.close();
}

string convertToUTF8(char[] text) {
	string result;
	for (uint i=0; i<text.length; i++) {
		wchar ch = text[i];
		if (ch < 0x80) {
			result ~= ch;
		} else {
			result ~= 0xC0 | (ch >> 6);
			result ~= 0x80 | (ch & 0x3F);
		}
	}
	return result;
}

It compiles and works as long as the returned char-array/string of f.readLine() doesn't contain non-UTF8 character(s). If it contains such
chars, writeln() doesn't write anything to the console. Is there any chance to read such files?

Thanks a lot!


More information about the Digitalmars-d-learn mailing list