我要问的是众所周知的“结构的最后一个成员具有可变长度”技巧。它是这样的:
struct T {
int len;
char s[1];
};
struct T *p = malloc(sizeof(struct T) + 100);
p->len = 100;
strcpy(p->s, "hello world");
由于结构在内存中的布局方式,我们能够将结构覆盖在大于必要的块上,并将最后一个成员视为大于 1 char< /代码>指定。
所以问题是:这种技术在技术上是未定义的行为吗?。我希望是这样,但很好奇标准对此有何规定。
PS:我知道 C99 对此的方法,我希望答案专门针对上面列出的技巧版本。
What I am asking about is the well known "last member of a struct has variable length" trick. It goes something like this:
struct T {
int len;
char s[1];
};
struct T *p = malloc(sizeof(struct T) + 100);
p->len = 100;
strcpy(p->s, "hello world");
Because of the way that the struct is laid out in memory, we are able to overlay the struct over a larger than necessary block and treat the last member as if it were larger than the 1 char
specified.
So the question is: Is this technique technically undefined behavior?. I would expect that it is, but was curious what the standard says about this.
PS: I am aware of the C99 approach to this, I would like the answers to stick specifically to the version of the trick as listed above.
发布评论
评论(8)
正如 C FAQ 所说:
和:
“严格符合”位背后的基本原理位于规范的 J.2 未定义行为 部分,其中包括未定义行为的列表:
6.5.6 加法运算符第 8 段再次提到,超出定义的数组边界的访问是未定义的:
As the C FAQ says:
and:
The rationale behind the 'strictly conforming' bit is in the spec, section J.2 Undefined behavior, which includes in the list of undefined behavior:
Paragraph 8 of Section 6.5.6 Additive operators has another mention that access beyond defined array bounds is undefined:
我相信从技术上讲这是未定义的行为。该标准(可以说)没有直接解决它,因此它属于“或通过省略任何明确的行为定义”。条款(C99 §4/2,C89 §3.16/2)表示这是未定义的行为。
上面的“可以说”取决于数组下标运算符的定义。具体来说,它说:“后缀表达式后跟方括号 [] 中的表达式是数组对象的下标名称。” (C89,第 6.3.2.1/2 节)。
您可以认为这里违反了“数组对象”(因为您在数组对象的定义范围之外下标),在这种情况下,行为(稍微多一点)显式未定义,而不仅仅是未定义没有什么可以完全定义它。
从理论上讲,我可以想象一个编译器会进行数组边界检查,并且(例如)当/如果您尝试使用超出范围的下标时会中止程序。事实上,我不知道这样的事情是否存在,并且鉴于这种代码风格的流行,即使编译器在某些情况下尝试强制执行下标,也很难想象有人会忍受它这样做这种情况。
I believe that technically it's undefined behavior. The standard (arguably) doesn't address it directly, so it falls under the "or by the omission of any explicit definition of behavior." clause (§4/2 of C99, §3.16/2 of C89) that says it's undefined behavior.
The "arguably" above depends on the definition of the array subscripting operator. Specifically, it says: "A postfix expression followed by an expression in square brackets [] is a subscripted designation of an array object." (C89, §6.3.2.1/2).
You can argue that the "of an array object" is being violated here (since you're subscripting outside the defined range of the array object), in which case the behavior is (a tiny bit more) explicitly undefined, instead of just undefined courtesy of nothing quite defining it.
In theory, I can imagine a compiler that does array bounds checking and (for example) would abort the program when/if you attempted to use an out of range subscript. In fact, I don't know of such a thing existing, and given the popularity of this style of code, even if a compiler tried to enforce subscripts under some circumstances, it's hard to imagine that anybody would put up with its doing so in this situation.
是的,这是未定义的行为。
C 语言缺陷报告#051 给出了这个问题的明确答案:
http:// /www.open-std.org/jtc1/sc22/wg14/www/docs/dr_051.html
在 C99 基本原理文档中,C 委员会添加了:
Yes, it is undefined behavior.
C Language Defect Report #051 gives a definitive answer to this question:
http://www.open-std.org/jtc1/sc22/wg14/www/docs/dr_051.html
In the C99 Rationale document the C Committee adds:
这种特殊的实现方法并未在任何 C 标准中明确定义,但 C99 确实将“struct hack”作为该语言的一部分。在 C99 中,结构的最后一个成员可能是“灵活数组成员”,声明为 char foo[](使用您想要的任何类型来代替 char)。
That particular way of doing it is not explicitly defined in any C standard, but C99 does include the "struct hack" as part of the language. In C99, the last member of a struct may be a "flexible array member", declared as
char foo[]
(with whatever type you desire in place ofchar
).这不是未定义的行为,无论任何人,官方或其他人怎么说,因为它是由标准定义的。
p->s
,除非用作左值,否则计算结果为与(char *)p + offsetof(struct T, s)
相同的指针。特别是,这是 malloc 对象内的有效char
指针,并且紧随其后有 100 个(或更多,取决于对齐考虑)连续地址,这些地址也作为char 有效
分配对象内的对象。事实上,该指针是通过使用->
派生的,而不是显式地将偏移量添加到malloc
返回的指针上,并强制转换为char *
,是无关紧要的。从技术上讲,
p->s[0]
是结构体中char
数组的单个元素,接下来的几个元素(例如p->s [1]
到p->s[3]
) 可能是结构体内部的填充字节,如果您对整个结构体执行赋值,则可能会被损坏,但如果您仅访问单个成员,其余元素是分配的对象中的附加空间,您可以随意使用,只要您遵守对齐要求(并且char
没有对齐要求) 。如果您担心结构中与填充字节重叠的可能性可能会以某种方式引发鼻恶魔,您可以通过将
[1]
中的1
替换为值来避免这种情况这确保了结构末尾没有填充。一种简单但浪费的方法是创建一个具有相同成员的结构,除了末尾没有数组,并使用s[sizeof struct that_other_struct];
作为数组。然后,p->s[i]
被明确定义为i 的结构中的数组元素,以及地址处的 char 对象在
i>=sizeof struct that_other_struct
的结构末尾之后。编辑:实际上,在上面获得正确大小的技巧中,您可能还需要在数组之前放置一个包含每个简单类型的并集,以确保数组本身以最大对齐方式开始,而不是以最大对齐方式开始其他一些元素的填充的中间。再说一遍,我认为这些都没有必要,但我将其提供给最偏执的语言律师。
编辑2:由于标准的另一部分,与填充字节的重叠绝对不是问题。 C 要求如果两个结构体其元素的初始子序列一致,则可以通过指向任一类型的指针访问公共初始元素。因此,如果声明了与
struct T
相同但具有更大的最终数组的结构,则元素s[0]
必须与元素一致struct T
中的 s[0],并且这些附加元素的存在不会影响使用指向 struct T< 的指针访问较大结构的公共元素,也不会受到这些元素的影响/代码>。It is not undefined behavior, regardless of what anyone, official or otherwise, says, because it is defined by the standard.
p->s
, except when used as an lvalue, evaluates to a pointer identical to(char *)p + offsetof(struct T, s)
. In particular, this is a validchar
pointer inside the malloc'd object, and there are 100 (or more, dependign on alignment considerations) successive addresses immediately following it which are also valid aschar
objects inside the allocated object. The fact that the pointer was derived by using->
instead of explicitly adding the offset to the pointer returned bymalloc
, cast tochar *
, is irrelevant.Technically,
p->s[0]
is the single element of thechar
array inside the struct, the next few elements (e.g.p->s[1]
throughp->s[3]
) are likely padding bytes inside the struct, which could be corrupted if you perform assignment to the struct as a whole but not if you merely access individual members, and the rest of the elements are additional space in the allocated object which you are free to use however you like, as long as you obey alignment requirements (andchar
has no alignment requirements).If you are worried that the possibility of overlapping with padding bytes in the struct might somehow invoke nasal demons, you could avoid this by replacing the
1
in[1]
with a value which ensures that there is no padding at the end of the struct. A simple but wasteful way to do this would be to make a struct with identical members except no array at the end, and uses[sizeof struct that_other_struct];
for the array. Then,p->s[i]
is clearly defined as an element of the array in the struct fori<sizeof struct that_other_struct
and as a char object at an address following the end of the struct fori>=sizeof struct that_other_struct
.Edit: Actually, in the above trick for getting the right size, you might also need to put a union containing every simple type before the array, to ensure that the array itself begins with maximal alignment rather than in the middle of some other element's padding. Again, I don't believe any of this is necessary, but I'm offering it up for the most paranoid of the language-lawyers out there.
Edit 2: The overlap with padding bytes is definitely not an issue, due to another part of the standard. C requires that if two structs agree in an initial subsequence of their elements, the common initial elements can be accessed via a pointer to either type. As a consequence, if a struct identical to
struct T
but with a larger final array were declared, the elements[0]
would have to coincide with the elements[0]
instruct T
, and the presence of these additional elements could not affect or be affected by accessing common elements of the larger struct using a pointer tostruct T
.是的,这在技术上是未定义的行为。
请注意,至少有三种方法可以实现“struct hack”:
(1) 声明大小为 0 的尾随数组(遗留代码中最“流行”的方式)。这显然是 UB,因为零大小数组声明在 C 中始终是非法的。即使它能够编译,该语言也不保证任何违反约束的代码的行为。
(2) 声明具有最小合法大小的数组 - 1 (您的情况)。在这种情况下,任何尝试获取指向
p->s[0]
的指针并将其用于超出p->s[1]
范围的指针算术都是未定义的行为。例如,调试实现可以生成一个带有嵌入范围信息的特殊指针,每次您尝试创建超出p->s[1]
的指针时,该指针都会被捕获。(3) 声明“非常大”的数组,例如 10000。这个想法是,声明的大小应该大于实际操作中可能需要的任何大小。该方法在数组访问范围方面没有 UB。然而,在实践中,我们当然总是会分配较小的内存量(仅分配真正需要的内存量)。我不确定这样做的合法性,即我想知道为对象分配比对象声明的大小更少的内存有多合法(假设我们从不访问“未分配”成员)。
Yes, it is technically undefined behavior.
Note, that there are at least three ways to implement the "struct hack":
(1) Declaring the trailing array with size 0 (the most "popular" way in legacy code). This is obviously UB, since the zero size array declarations are always illegal in C. Even if it does compile, the language makes no guarantees about the behavior of any constraint-violating code.
(2) Declaring the array with minimal legal size - 1 (your case). In this case any attempts to take pointer to
p->s[0]
and use it for pointer arithmetic that goes beyondp->s[1]
is undefined behavior. For example, a debugging implementation is allowed to produce a special pointer with embedded range information, which will trap every time you attempt to create a pointer beyondp->s[1]
.(3) Declaring the array with "very large" size like 10000, for example. The idea is that the declared size is supposed to be larger than anything you might need in actual practice. This method is free of UB with regard to array access range. However, in practice, of course, we will always allocate smaller amount of memory (only as much as really needed). I'm not sure about the legality of this, i.e. I wonder how legal it is to allocate less memory for the object than the declared size of the object (assuming we never access the "non-allocated" members).
标准非常明确,您无法访问数组末尾之外的内容。 (并且通过指针没有帮助,因为您甚至不允许在数组结束后将指针增加到超过一)。
以及“在实践中工作”。我见过 gcc/g++ 优化器使用标准的这一部分,因此在遇到这个无效的 C 时会生成错误的代码。
The standard is quite clear that you cannot access things beside the end of an array. (and going via pointers does not help, as you are not allowed to even increment pointers past one after array end).
And for "working in practise". I've seen gcc/g++ optimizer using this part of the standard thus generating wrong code when meeting this invalid C.
如果编译器接受类似的内容,
我认为很明显它必须准备好接受“dat”上超出其长度的下标。另一方面,如果有人编写如下代码:
然后稍后访问 somestruct->dat[x];我不认为编译器有任何义务使用可以处理较大 x 值的地址计算代码。我认为如果想要真正安全,正确的范例更像是:
然后执行 (sizeof(MYSTRUCT)-LARGEST_DAT_SIZE +desired_array_length) 字节的 malloc(请记住,如果desired_array_length大于LARGEST_DAT_SIZE,结果可能未定义)。
顺便说一句,我认为禁止零长度数组的决定是一个不幸的决定(Turbo C 等一些较旧的方言支持它),因为零长度数组可以被视为编译器必须生成可使用较大索引的代码的标志。
If a compiler accepts something like
I think it's pretty clear that it must be ready to accept a subscript on 'dat' beyond its length. On the other hand, if someone codes something like:
and then later accesses somestruct->dat[x]; I would not think the compiler is under any obligation to use address-computation code which will work with large values of x. I think if one wanted to be really safe, the proper paradigm would be more like:
and then do a malloc of (sizeof(MYSTRUCT)-LARGEST_DAT_SIZE + desired_array_length) bytes (bearing in mind that if desired_array_length is larger than LARGEST_DAT_SIZE, the results may be undefined).
Incidentally, I think the decision to forbid zero-length arrays was an unfortunate one (some older dialects like Turbo C support it) since a zero-length array could be regarded as a sign that the compiler must generate code that will work with larger indices.