Thread: NSIS 3.0b2
View Single Post
Old 26th October 2015, 22:38   #10
Anders
Moderator
 
Anders's Avatar
 
Join Date: Jun 2002
Location: ${NSISDIR}
Posts: 5,534
Yes, it was clear that we had to do something about this but deciding on the behavior was not that easy.

I have now changed FileReadUTF16LE so that it will always skip the BOM at the start of a file. (If you need to detect the BOM you must use FileReadByte or FileReadWord.

FileWriteUTF16LE can now optionally add a BOM if you specify the /BOM switch. The thinking here was that not all apps can handle the BOM so you have to be specific about it. This is especially important if we decide to add UTF8 support as well since it is very common for *nix based tools to choke on the BOM.

This last issue is of course open for discussion while we are still in Beta but kichik and I have already spent all day discussing this and we came down on the side of no BOM by default vs BOM by default and a /NoBOM switch... (Inno Setup does not seem to support UTF16, the _wfopen MS CRT function wants a Unicode flag and finally "cmd.exe /U /C echo.Foo > Bar.txt" does not write a BOM)

Quote:
Originally Posted by aerDNA View Post
You are technically correct about everything but I think users will tend to assume FileReadUTF16LE returns the same thing they see in text editor (or MessageBox or DetailPrint).
You cannot see the BOM because it has zero-width , converting to Ansi however will fail and you get a '?'.

IntOp $PostCount $PostCount + 1
Anders is offline   Reply With Quote