RTF problem with ZWNJ and ZWJ followed by Space

When entering text in a Word (2007 / 2003) document, and entering a Unicode
Zero Width Non-Joiner / ZWNJ (0x200C) or a Zero Width Joiner / ZWJ (0x200D)
followed by a space, the space disappears when saving the document to RTF,
and the re-opening it in Word.

Using these character is sometimes necessary in Complex script languages,
such as Persian (which uses the ZWNJ to achive the terminal form of a
character in the middle of a word) and Malayalam (which uses ZWJ in typing
the Chillus prior to Unicode 5.1).

This doesn't happen every time. If the ZWNJ / ZWJ is in it's own RTF
grouping ("{\keywords text}") then the space is retained. If the formatting
of the ZWNJ / ZWJ is the same as the surrounding text, then the space
following the ZWNJ / ZWJ will be lost.

The loss of the space happens apparently because of a bug in the way the RTF
is written. RTF keywords are terminated by means of (1) another \ character
(2) a space. So then the \zwnj and \zwj keywords are written into a string
where there is no space, the RTF is written as follows:
....\'ed\'ed\zwnj\'ed\'ed...
This is fine. However, when there is a space, it is written like this:
....\'ed\'ed\zwnj \'ed\'ed...
This is not fine, because the space after \zwnj simply indicates the end of
the keyword. To indicate the need for a space, we need an additional space in
the RTF string.

I am interested in getting this bug confirmed, and then reporting this to
the Word programming team.

How do I go about that?

Thanks,
--
Paul Willies

Hi Paul,

This sounds like one that came up once before, but to confirm that it's a problem, what are the steps to reproduce this, using one
of the MS Office supplied multilingual fonts and what languages are enabled with the Office Language Settings Tool.

If you have two scenarios, one where it works the way you're trying to go and one that doesn't please provide both repro steps.
Also do you have a page/blog, etc that you can provide a link to for good/bad example documents?

===================
<<"PaulWill" <pwillies.RemoveThis@jw.org.za.donotspam> wrote in message news:DB45EAC8-6BCB-4ECC-843C-2013FD9A150C@microsoft.com...
When entering text in a Word (2007 / 2003) document, and entering a Unicode
Zero Width Non-Joiner / ZWNJ (0x200C) or a Zero Width Joiner / ZWJ (0x200D)
followed by a space, the space disappears when saving the document to RTF,
and the re-opening it in Word.

Using these character is sometimes necessary in Complex script languages,
such as Persian (which uses the ZWNJ to achive the terminal form of a
character in the middle of a word) and Malayalam (which uses ZWJ in typing
the Chillus prior to Unicode 5.1).

This doesn't happen every time. If the ZWNJ / ZWJ is in it's own RTF
grouping ("{\keywords text}") then the space is retained. If the formatting
of the ZWNJ / ZWJ is the same as the surrounding text, then the space
following the ZWNJ / ZWJ will be lost.

The loss of the space happens apparently because of a bug in the way the RTF
is written. RTF keywords are terminated by means of (1) another \ character
(2) a space. So then the \zwnj and \zwj keywords are written into a string
where there is no space, the RTF is written as follows:
....\'ed\'ed\zwnj\'ed\'ed...
This is fine. However, when there is a space, it is written like this:
....\'ed\'ed\zwnj \'ed\'ed...
This is not fine, because the space after \zwnj simply indicates the end of
the keyword. To indicate the need for a space, we need an additional space in
the RTF string.

I am interested in getting this bug confirmed, and then reporting this to
the Word programming team.

How do I go about that?

Thanks,
--
Paul Willies >>
--

Bob Buckland ?
MS Office System Products MVP

*Courtesy is not expensive and can pay big dividends*