如何使用nodejs-iconv模块（或其他解决方案）在nodejs javascript中将字符编码从CP932转换为UTF-8

发布于 2024-11-16 07:00:33 字数 1014 浏览 0 评论 0原文

我正在尝试在 JavaScript 中将字符串从 CP932（又名 Windows-31J）转换为 utf8。基本上，我正在抓取一个忽略请求标头中的 utf-8 请求并返回 cp932 编码文本的网站（即使 html 元标记表明该页面是 shift_jis）。

无论如何，我将整个页面存储在一个名为“html”的字符串变量中。从那里我尝试使用以下代码将其转换为 utf8：

var Iconv = require('iconv').Iconv;
var conv = new Iconv('CP932', 'UTF-8//TRANSLIT//IGNORE');

var myBuffer = new Buffer(html.length * 3);
myBuffer.write(html, 0, 'utf8')
var utf8html = (conv.convert(myBuffer)).toString('utf8');

结果不是它应该的样子。例如，字符串“稿投者さんの稚内全日空ホテルのクチコミ (感想·情报)”的结果为“ｿｽｿｽｿｽeｿｽﾒゑｿｽｿｽｿｽｿｽｿｽｿｽtｿｽｿｽｿｽSｿｽｿｽｿｽｿｽｿｽzｿｽeｿｽｿｽｿｽﾌクｿｽ`ｿｽRｿｽ~ (ｿｽｿｽｿｽzｿｽEｿｽｿｽｿｽｿｽ)"

如果我删除 //TRANSLIT//IGNORE （这应该会导致它返回丢失字符的相似字符，并且如果失败则忽略不可转码的字符），我收到此错误：错误：EILSEQ，非法字符序列。

我愿意使用任何可以在 nodejs 中实现的解决方案，但我的搜索结果没有在 nodejs-iconv 模块之外产生很多选项。

nodejs-iconv 参考： https://github.com/bnoordhuis/node-iconv

谢谢！

2011 年 6 月 24 日编辑：我已经用 Java 实现了一个解决方案。不过，如果有人能解决这个问题，我仍然对这个问题的 javascript 解决方案感兴趣。

原文

I'm attempting to convert a string from CP932 (aka Windows-31J) to utf8 in javascript. Basically I'm crawling a site that ignores the utf-8 request in the request header and returns cp932 encoded text (even though the html metatag indicates that the page is shift_jis).

Anyway, I have the entire page stored in a string variable called "html". From there I'm attempting to convert it to utf8 using this code:

var Iconv = require('iconv').Iconv;
var conv = new Iconv('CP932', 'UTF-8//TRANSLIT//IGNORE');

var myBuffer = new Buffer(html.length * 3);
myBuffer.write(html, 0, 'utf8')
var utf8html = (conv.convert(myBuffer)).toString('utf8');

The result is not what it's supposed to be. For example, the string: "投稿者さんの稚内全日空ホテルのクチコミ (感想・情報)" comes out as "ｿｽｿｽｿｽeｿｽﾒゑｿｽｿｽｿｽｿｽｿｽｿｽtｿｽｿｽｿｽSｿｽｿｽｿｽｿｽｿｽzｿｽeｿｽｿｽｿｽﾌクｿｽ`ｿｽRｿｽ~ (ｿｽｿｽｿｽzｿｽEｿｽｿｽｿｽｿｽ)"

If I remove //TRANSLIT//IGNORE (Which should cause it to return similar characters for missing characters, and failing that omit non-transcode-able characters), I get this error:
Error: EILSEQ, Illegal character sequence.

I'm open to using any solution that can be implemented in nodejs, but my search results haven't yielded many options outside of the nodejs-iconv module.

nodejs-iconv ref: https://github.com/bnoordhuis/node-iconv

Thanks!

Edit 24.06.2011:
I've gone ahead and implemented a solution in Java. However I'd still be interested in a javascript solution to this problem if somebody can solve it.

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

北城挽邺 2024-11-23 07:00:33

我今天遇到了同样的麻烦:)
这取决于 libiconv。您需要 libiconv-1.13-ja-1.patch。
请检查以下内容。

或者您可以使用 iconv-jp 尝试避免问题

npm install iconv-jp

I got same trouble today :)
It depends libiconv. You need libiconv-1.13-ja-1.patch.
Please check followings.

or you can avoid problem using iconv-jp try

npm install iconv-jp

回复收藏 0 原文

许一世地老天荒 2024-11-23 07:00:33

我也有同样的问题，但是是用 CP1250。我到处寻找问题，一切都很好，除了请求调用 - 我必须添加 encoding: 'binary'。

request = require('request')
Iconv  = require('iconv').Iconv

request({uri: url, encoding: 'binary'}, function(err, response, body) {
    body = new Buffer(body, 'binary')
    iconv = new Iconv('CP1250', 'UTF8')
    body = iconv.convert(body).toString()
    // ...
})

I had same problem, but with CP1250. I was looking for problem everywhere and everything was OK, except call of request – I had to add encoding: 'binary'.

request = require('request')
Iconv  = require('iconv').Iconv

request({uri: url, encoding: 'binary'}, function(err, response, body) {
    body = new Buffer(body, 'binary')
    iconv = new Iconv('CP1250', 'UTF8')
    body = iconv.convert(body).toString()
    // ...
})

回复收藏 0 原文