Rails 3、mysql/mysql2 将某些检索到的字符串误解为 ASCII-8BIT

发布于 2024-10-05 01:16:59 字数 1399 浏览 3 评论 0原文

这个问题最初是常见的“不兼容的字符编码：ASCII-8BIT 和 UTF-8”问题，但这不是我要问的。相反，我发现发生此问题是因为数据库的某些字段在检索时被标记为 ASCII-8BIT，而大多数字段正确显示为 UTF-8。

例如，在包含 country 和 nationality 列的表中，第 16 行中的两列具有相同的值（复制并粘贴），我得到

irb(main):003:0> c = Country.find(16)
irb(main):004:0> puts "#{c.name}, #{c.name.encoding}, #{c.name.bytes.to_a}"
�land Islands, UTF-8, [195, 133, 108, 97, 110, 100, 32, 73, 115, 108, 97, 110, 100, 115]
irb(main):005:0> puts "#{c.nationality}, #{c.nationality.encoding}, #{c.nationality.bytes.to_a}"
�land Islands, ASCII-8BIT, [195, 133, 108, 97, 110, 100, 32, 73, 115, 108, 97, 110, 100, 115]

同样，一个简单的 < code>puts name 给出 �landIslands 而对于 nationality 它给出 "\xC3\x85landIslands" -- 相同的字节，不同的演示。

无论字符串是否包含非 ascii 字符，给定列的编码似乎都是恒定的，因此这不仅仅是字符串的问题。也就是说，nationality 中的所有值都被解释为 ascii，name 中的所有值都被解释为 UTF-8。

问题不仅限于单个表，而且我还没有发现任何列被错误识别的模式。

以下是设置和环境：

Windows 7 64 位上的 Rails 3.0.0
数据库适配器：mysql2 和 mysql 都显示相同的行为
Database.yml 包含 encoding: utf8
application.rb 包含 config。 coding = "utf-8"
MySQL 数据库、表和两列都定义为 utf8
MySQL 中的两列都是 varchar, 255, allowed null
我可以通过全新安装 Rails 重现该问题，除了国家/地区之外什么都没有定义模型，访问数据库。我还没有尝试过使用新的单行数据库。

有人知道这是怎么回事吗？

原文

This problem began as the commonly-seen “incompatible character encodings: ASCII-8BIT and UTF-8” problem, but that is not what I'm asking. Rather, I discovered that this problem was happening because certain fields of my database are being tagged as ASCII-8BIT when they're retrieved, while most are correctly shown as UTF-8.

For example, in a table with columns country and nationality, where both columns in row 16 have identical values (copied-and-pasted), I get

irb(main):003:0> c = Country.find(16)
irb(main):004:0> puts "#{c.name}, #{c.name.encoding}, #{c.name.bytes.to_a}"
�land Islands, UTF-8, [195, 133, 108, 97, 110, 100, 32, 73, 115, 108, 97, 110, 100, 115]
irb(main):005:0> puts "#{c.nationality}, #{c.nationality.encoding}, #{c.nationality.bytes.to_a}"
�land Islands, ASCII-8BIT, [195, 133, 108, 97, 110, 100, 32, 73, 115, 108, 97, 110, 100, 115]

Likewise, a simple puts name gives �land Islands while for nationality it gives "\xC3\x85land Islands" -- same bytes, different presentation.

The encoding for a given column appears to be constant regardless of whether the string has non-ascii characters, so it is not simply a problem with the string. That is, all the values in nationality are interpreted as ascii and all those in name as UTF-8.

The problem is not limited to a single table, and I have not found any pattern to which columns are mis-recognized.

Here are the settings and environment:

Rails 3.0.0 on Windows 7 64-bit
Database adapter: mysql2 and mysql both show same behavior
Database.yml includes encoding: utf8
application.rb includes config.encoding = "utf-8"
MySQL database, table, and both columns are defined as utf8
Both columns in MySQL are varchar, 255, allow null
I can reproduce the problem with a fresh installation of Rails and nothing except the Country model defined, to access the database. I have not yet tried with a fresh, one-line database.

Anyone know what's going on here?

分享到QQ

分享到微博