如何比较 utf8 字符串,例如 c++ 中的波斯语单词?
我想比较波斯语(utf8)的字符串。我知道我必须使用像 L"带" 这样的东西,并且它必须保存在 wchar_t * 或 wstring 中。问题是当我通过函数compare()字符串进行比较时,我没有得到正确的结果。
I want to compare strings in Persian (utf8). I know I must use some thing like L"گل" and it must be saved in wchar_t * or wstring. the question is when I compare by the function compare() strings I dont get the right result.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(3)
wchar_t
不适用于 UTF-8,但(取决于平台)通常适用于 UTF-16 或 UCS-32。如果您想使用 UTF-8,请使用普通的旧式char *
或string
以及它们的相等比较函数。如果你想要人类有意义的排序,它就会变得更加复杂(无论你使用哪种编码)。wchar_t
is not for UTF-8, but (depending on the platform) typically either UTF-16 or UCS-32. If you want to work on UTF-8, use plain oldchar *
orstring
, and their comparison functions for equality. If you want human-meaingful sorting, it gets much more involved (no matter which encoding you use).Unicode 是出了名的难以比较。
请注意,任何 Unicode 编码(包括 UTF-8、16 或 32)都不能按字节进行比较,除非字节相等。显示可能是相同的,但所使用的字节(例如R→L标记、代理对、显示修饰符以及非英语语言(例如波斯语)中使用的类似字节)将不同。
一般来说,如果文本的含义具有任何意义,则需要先对 Unicode 进行规范化,然后才能进行实际比较:
http://userguide.icu-project.org/transforms/normalization
Unicode is notoriously difficult to compare.
Note that any Unicode encoding, including UTF-8, 16 or 32 cannot be compared byte-wise for anything other than byte-equality. The display may be identical, but the bytes used (such as R->L markers, surrogate pairs, display modifiers, and similar used in non-English languages such as Persian) will not be.
Generally, you need to normalize Unicode before you can make a realistic comparison if the meaning of the text has any significance:
http://userguide.icu-project.org/transforms/normalization
如果您要比较的字符串已经采用特定的、明确的编码,则不要使用
wchar_t
也不要使用L""
文字 - 这些是不适用于 Unicode,仅适用于实现定义的不透明编码。如果您的字符串采用 UTF-8 格式,请使用
char
字符串。如果您想将它们转换为原始 Unicode 代码点 (UCS-4/UTF-32),或者您已经拥有该形式的它们,请将它们存储在uint32_t
字符串中,或char32_t
s(如果您有现代编译器)。如果您使用的是 C++11,则文字可以是 char str8[] = u8"带"; 或 char32_t str32[] = U"带";。 请参阅此主题了解更多相关信息。
如果您想与命令行参数或环境交互,请使用 iconv() 将 WCHAR 转换为 UTF-32 或 UTF-8。
If the strings that you want to compare are in a specific, definite encoding already, then don't use
wchar_t
and don't useL""
literals -- those are not for Unicode, but for implementation-defined, opaque encodings only.If your strings are in UTF-8, use a string of
char
s. If you want to convert them to raw Unicode codepoints (UCS-4/UTF-32), or if you already have them in that form, store them in a string ofuint32_t
s, orchar32_t
s if you have a modern compiler.If you have C++11, your literal can be
char str8[] = u8"گل";
orchar32_t str32[] = U"گل";
. See this topic for some more on this.If you want to interact with command line arguments or the environment, use
iconv()
to convert from WCHAR to UTF-32 or UTF-8.