[Issue 1357] Cannot use FFFF and FFFE in Unicode escape sequences.

[d-bugmail at puremagic.com]

http://d.puremagic.com/issues/show_bug.cgi?id=1357





------- Comment #6 from smjg at iname.com  2007-09-30 16:31 -------
When that talks of "decoders", is it talking about:
(a) decoding Unicode text from files?
(b) translating data used internally by an application?

If (a), then obviously it should reject U+FFFE and U+FFFF.  If (b), then it
should allow them.  The std.utf.toUTF* functions accept these codepoints for
this reason, and the behaviour of isValidDchar is by the same design:

/*******************************
 * Test if c is a valid UTF-32 character.
 *
 * \uFFFE and \uFFFF are considered valid by this function,
 * as they are permitted for internal use by an application,
 * but they are not allowed for interchange by the Unicode standard.
 *
 * Returns: true if it is, false if not.
 */

So it's not a bug in Phobos.  Just an omission - of a function to check that a
Unicode string is valid for data interchange (i.e. contains no U+FFFE or U+FFFF
codepoints as well as being otherwise valid).

It certainly ought to be possible to include U+FFFE and U+FFFF in string
literals by some means or another, as such things are necessarily being put
there for internal use by the application being developed.


--