最大化存储的信息（熵？）
所以我不确定这个问题是否属于这里，或者可能是数学溢出。无论如何，我的问题是关于信息论的。
假设我有一个 16 位字。该数字中有 65,536 种 1 和 0 的独特配置。这些配置中的每一种代表什么并不重要，因为根据您的符号（2 的补码与带符号的大小等）相同的配置可能意味着不同的事情。
我想知道是否有任何技术可以存储比 16 位字更多的信息？
我最初的想法是奇/偶奇偶校验或其他什么，但后来我意识到这已经由配置决定了...即其中没有编码额外的信息。我开始怀疑这样的事情是否存在。
编辑 例如，假设某个神奇的计算机（思考量子或这里的其他东西）可以理解 0,1,a。那么显然我们有 3^16 种配置，现在可以存储的数量超过了数字 [0 - 65,536]。为了在比特流中编码额外信息，您是否可以修改 16 位字的任何其他属性？
EDIT2 我真的很难用语言表达这一点。现在，当我查看计算机中的 16 位字时，该属性向我传达单个 1 和 0 的相对顺序信息。是否有另一种属性或方式来查看 16 位字，它允许超过 2^16 个唯一的“配置”？ （注意，它不再是一个配置，而是 2^16 xxxx，其中 xxxx 是描述该属性实例的名词）。我真正能想到的唯一一件事是，我们是否查看 1 到 0 转换的数量或其他内容，而不是查看每个位实际上是 1 还是 0？现在，转换不会产生超过 2^16 种组合，因为它最终仅取决于 1 和 0 的配置。我正在寻找从 1 和 0 的配置派生出来的属性AND其他东西，从而导致更多超过 2^16。有谁知道如果它确实存在的话会叫什么？
EDIT3 好的，我明白了。我的问题归结为：我们如何证明一个词中的 1 和 0 的配置完全定义了它？ IE 我们如何证明除了位图之外不需要其他信息来显示两个 16 位字之间的相等性？
最终编辑
我有一个例子...如果我们不查看 1 和 0 的存在，而是查看位之间的转换，我们就可以存储 2^16 个字母字符。如果左边的位相同，则将其视为 1，如果发生转换，则将其视为 0。使用 16 位字作为循环链接列表类型结构，其中每个链接代表 0/1 我们基本上对于 16 位字出位之间的转换。这正是我正在寻找的示例，但结果是 2^16，没有更好的了。我相信你不能做得更好，并标记正确答案 =(
So I'm not sure if this question belongs here or maybe Math overflow. In any case, my question is about information theory.
Let's say I have a 16 bit word. There are 65,536 unique configurations of 1's and 0's in that number. What each one of those configurations represents is unimportant as depending on your notation (2's complement vs signed magnitude etc.) the same configuration can mean different things.
What I'm wondering is are there any techniques to store more information than that in a 16 bit word?
My original ideas were like odd/even parity or something but then I realized that's already determined by the configuration... i.e. there is no extra information encoded in that. I'm beginning to wonder if no such thing exists.
EDIT For example, let's say some magical computer (thinking quantum or something here) could understand 0,1,a. Then obviously we have 3^16 configurations and can now store more than the numbers [0 - 65,536]. Are there any other properties of a 16 bit word that you can mess with in order to encode extra information in your bit stream?
EDIT2 I am really struggling to put this into words. Right now when I look at a 16 bit word in the computer, the property which conveys information to me the relative ordering of individual 1's and 0's. Is there another property or way of looking at a 16 bit word which would allow more than 2^16 unique "configurations"? (Note it would no longer be a configuration, but 2^16 xxxx's where xxxx is a noun describing an instance of that property). The only thing I can really think of is something like if we looked at the number of 1 to 0 transitions or something rather than whether each bit was actually a 1 or 0? Now transitions does not yield more than 2^16 combinations because it is ultimately solely dependent on the configuration of 1's and 0's. I'm looking for properties that would derive from the configuration of 1's and 0's AND something else thus resulting in MORE than 2^16. Does anyone even know what this would be called if it did exist?
EDIT3 Ok I got it. My question boils down to this: How do we prove that the configuration of 1's and 0's in a word completely defines it? I.E. How do we prove that you need no other information besides the bitmap to show equality between two 16 bit words?
FINAL EDIT
I have an example... If instead of looking at the presence of 1's and 0's we look at transition between bits we can store 2^16 alphabet characters. If the bit to left is the same, treat it as a 1, if it transitions, treat it as a 0. Using the 16 bit word as a circularly linked list type structure where each link represent 0/1 we basically for a 16 bit word out of the transition between bits. That is an exact example of what I was looking for but that results in 2^16, nothing better. I am convinced that you cannot do better and am marking the correct answer =(
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
16 0/1s 的特定配置中的信息量由该配置的概率决定（这称为自信息）。如果配置的可能性小于 1/(2^16)，则该值可能大于 16 位，但这意味着其他一些配置的可能性大于 1/(2^16)，因此包含的信息将少于 16 位。
为了考虑所有可能的配置，您必须使用各个配置的自信息（称为熵）的预期值。当所有配置的概率相等（即 1/(2^16)）时，该值将达到最大值，然后它正好是 16 位。
所以答案是否定的，你不能在 16 0/1 秒内存储超过 16 位的信息。
请参阅
编辑 这很重要意识到这一点确实不代表0或1，但它是一个信息单位，即-log_2 P(w)，其中P(w)是特定配置的概率。
The amount of information in a particular configuration of 16 0/1s is determined by the probability of this configuration (this is called self-information). This can be bigger than 16 bits if the configuration is less likely than 1/(2^16), but that means that some other configurations are more likely than 1/(2^16) and so will contain less information than 16 bits.
To take into account all the possible configurations, you have to use the expected value of self-information (called entropy) of individual configurations. This value will reach its maximum when the probabilities of all configurations are equal (that is 1/(2^16)) and then it will be exactly 16 bits.
So the answer is no, you cannot store more than 16 bits of information in 16 0/1s.
See
EDIT It is important to realize that bit does not stand for 0 or 1, but it is a unit of information, that is -log_2 P(w) where P(w) is the probability of a particular configuration.
半导体器件的一位数字中不能存储超过 2 个状态。你自己回答了。 16 位数字可以容纳更多信息的唯一方法是每个数字都有许多可能的值。
You cannot store more than 2 states in one digit of a semiconductor device. You answered it yourself. The only way more information can be fitted into 16 digits is if each digit were to have many possible values.