是“struct hack”吗?技术上未定义的行为?

发布于 2024-09-19 10:58:43 字数 411 浏览 5 评论 0 原文

我要问的是众所周知的“结构的最后一个成员具有可变长度”技巧。它是这样的:

struct T {
    int len;
    char s[1];
};

struct T *p = malloc(sizeof(struct T) + 100);
p->len = 100;
strcpy(p->s, "hello world");

由于结构在内存中的布局方式,我们能够将结构覆盖在大于必要的块上,并将最后一个成员视为大于 1 char< /代码>指定。

所以问题是:这种技术在技术上是未定义的行为吗?。我希望是这样,但很好奇标准对此有何规定。

PS:我知道 C99 对此的方法,我希望答案专门针对上面列出的技巧版本。

What I am asking about is the well known "last member of a struct has variable length" trick. It goes something like this:

struct T {
    int len;
    char s[1];
};

struct T *p = malloc(sizeof(struct T) + 100);
p->len = 100;
strcpy(p->s, "hello world");

Because of the way that the struct is laid out in memory, we are able to overlay the struct over a larger than necessary block and treat the last member as if it were larger than the 1 char specified.

So the question is: Is this technique technically undefined behavior?. I would expect that it is, but was curious what the standard says about this.

PS: I am aware of the C99 approach to this, I would like the answers to stick specifically to the version of the trick as listed above.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(8

过度放纵 2024-09-26 10:58:43

正如 C FAQ 所说:

尚不清楚它是否合法或可移植,但它相当受欢迎。

和:

...官方解释认为它并不严格符合 C 标准,尽管它似乎确实可以在所有已知的实现下工作。 (仔细检查数组边界的编译器可能会发出警告。)

“严格符合”位背后的基本原理位于规范的 J.2 未定义行为 部分,其中包括未定义行为的列表:

  • 数组下标超出范围,即使对象显然可以通过给定下标访问(如左值表达式 a[1][7] 中给出的声明 int a [4][5]) (6.5.6).

6.5.6 加法运算符第 8 段再次提到,超出定义的数组边界的访问是未定义的:

如果指针操作数和结果都指向同一个数组对象的元素,或者超过数组对象的最后一个元素,则求值不会产生溢出;否则,行为未定义。

As the C FAQ says:

It's not clear if it's legal or portable, but it is rather popular.

and:

... an official interpretation has deemed that it is not strictly conforming with the C Standard, although it does seem to work under all known implementations. (Compilers which check array bounds carefully might issue warnings.)

The rationale behind the 'strictly conforming' bit is in the spec, section J.2 Undefined behavior, which includes in the list of undefined behavior:

  • An array subscript is out of range, even if an object is apparently accessible with the given subscript (as in the lvalue expression a[1][7] given the declaration int a[4][5]) (6.5.6).

Paragraph 8 of Section 6.5.6 Additive operators has another mention that access beyond defined array bounds is undefined:

If both the pointer operand and the result point to elements of the same array object, or one past the last element of the array object, the evaluation shall not produce an overflow; otherwise, the behavior is undefined.

淡忘如思 2024-09-26 10:58:43

我相信从技术上讲这是未定义的行为。该标准(可以说)没有直接解决它,因此它属于“或通过省略任何明确的行为定义”。条款(C99 §4/2,C89 §3.16/2)表示这是未定义的行为。

上面的“可以说”取决于数组下标运算符的定义。具体来说,它说:“后缀表达式后跟方括号 [] 中的表达式是数组对象的下标名称。” (C89,第 6.3.2.1/2 节)。

您可以认为这里违反了“数组对象”(因为您在数组对象的定义范围之外下标),在这种情况下,行为(稍微多一点)显式未定义,而不仅仅是未定义没有什么可以完全定义它。

从理论上讲,我可以想象一个编译器会进行数组边界检查,并且(例如)当/如果您尝试使用超出范围的下标时会中止程序。事实上,我不知道这样的事情是否存在,并且鉴于这种代码风格的流行,即使编译器在某些情况下尝试强制执行下标,也很难想象有人会忍受它这样做这种情况。

I believe that technically it's undefined behavior. The standard (arguably) doesn't address it directly, so it falls under the "or by the omission of any explicit definition of behavior." clause (§4/2 of C99, §3.16/2 of C89) that says it's undefined behavior.

The "arguably" above depends on the definition of the array subscripting operator. Specifically, it says: "A postfix expression followed by an expression in square brackets [] is a subscripted designation of an array object." (C89, §6.3.2.1/2).

You can argue that the "of an array object" is being violated here (since you're subscripting outside the defined range of the array object), in which case the behavior is (a tiny bit more) explicitly undefined, instead of just undefined courtesy of nothing quite defining it.

In theory, I can imagine a compiler that does array bounds checking and (for example) would abort the program when/if you attempted to use an out of range subscript. In fact, I don't know of such a thing existing, and given the popularity of this style of code, even if a compiler tried to enforce subscripts under some circumstances, it's hard to imagine that anybody would put up with its doing so in this situation.

旧城空念 2024-09-26 10:58:43

是的,这是未定义的行为。

C 语言缺陷报告#051 给出了这个问题的明确答案:

这个习语虽然常见,但并不严格遵循

http:// /www.open-std.org/jtc1/sc22/wg14/www/docs/dr_051.html

在 C99 基本原理文档中,C 委员会添加了:

这种结构的有效性一直受到质疑。在对一个缺陷的响应中
报告,委员会决定这是未定义的行为,因为数组 p->items
只包含一项,无论空间是否存在。

Yes, it is undefined behavior.

C Language Defect Report #051 gives a definitive answer to this question:

The idiom, while common, is not strictly conforming

http://www.open-std.org/jtc1/sc22/wg14/www/docs/dr_051.html

In the C99 Rationale document the C Committee adds:

The validity of this construct has always been questionable. In the response to one Defect
Report, the Committee decided that it was undefined behavior because the array p->items
contains only one item, irrespective of whether the space exists.

梦断已成空 2024-09-26 10:58:43

这种特殊的实现方法并未在任何 C 标准中明确定义,但 C99 确实将“struct hack”作为该语言的一部分。在 C99 中,结构的最后一个成员可能是“灵活数组成员”,声明为 char foo[](使用您想要的任何类型来代替 char)。

That particular way of doing it is not explicitly defined in any C standard, but C99 does include the "struct hack" as part of the language. In C99, the last member of a struct may be a "flexible array member", declared as char foo[] (with whatever type you desire in place of char).

緦唸λ蓇 2024-09-26 10:58:43

这不是未定义的行为,无论任何人,官方或其他人怎么说,因为它是由标准定义的。 p->s,除非用作左值,否则计算结果为与 (char *)p + offsetof(struct T, s) 相同的指针。特别是,这是 malloc 对象内的有效 char 指针,并且紧随其后有 100 个(或更多,取决于对齐考虑)连续地址,这些地址也作为 char 有效 分配对象内的对象。事实上,该指针是通过使用 -> 派生的,而不是显式地将偏移量添加到 malloc 返回的指针上,并强制转换为 char * ,是无关紧要的。

从技术上讲,p->s[0] 是结构体中 char 数组的单个元素,接下来的几个元素(例如 p->s [1]p->s[3]) 可能是结构体内部的填充字节,如果您对整个结构体执行赋值,则可能会被损坏,但如果您仅访问单个成员,其余元素是分配的对象中的附加空间,您可以随意使用,只要您遵守对齐要求(并且 char 没有对齐要求) 。

如果您担心结构中与填充字节重叠的可能性可能会以某种方式引发鼻恶魔,您可以通过将 [1] 中的 1 替换为值来避免这种情况这确保了结构末尾没有填充。一种简单但浪费的方法是创建一个具有相同成员的结构,除了末尾没有数组,并使用 s[sizeof struct that_other_struct]; 作为数组。然后,p->s[i] 被明确定义为 i 的结构中的数组元素,以及地址处的 char 对象在 i>=sizeof struct that_other_struct 的结构末尾之后。

编辑:实际上,在上面获得正确大小的技巧中,您可能还需要在数组之前放置一个包含每个简单类型的并集,以确保数组本身以最大对齐方式开始,而不是以最大对齐方式开始其他一些元素的填充的中间。再说一遍,我认为这些都没有必要,但我将其提供给最偏执的语言律师。

编辑2:由于标准的另一部分,与填充字节的重叠绝对不是问题。 C 要求如果两个结构体其元素的初始子序列一致,则可以通过指向任一类型的指针访问公共初始元素。因此,如果声明了与 struct T 相同但具有更大的最终数组的结构,则元素 s[0] 必须与元素 一致struct T 中的 s[0],并且这些附加元素的存在不会影响使用指向 struct T< 的指针访问较大结构的公共元素,也不会受到这些元素的影响/代码>。

It is not undefined behavior, regardless of what anyone, official or otherwise, says, because it is defined by the standard. p->s, except when used as an lvalue, evaluates to a pointer identical to (char *)p + offsetof(struct T, s). In particular, this is a valid char pointer inside the malloc'd object, and there are 100 (or more, dependign on alignment considerations) successive addresses immediately following it which are also valid as char objects inside the allocated object. The fact that the pointer was derived by using -> instead of explicitly adding the offset to the pointer returned by malloc, cast to char *, is irrelevant.

Technically, p->s[0] is the single element of the char array inside the struct, the next few elements (e.g. p->s[1] through p->s[3]) are likely padding bytes inside the struct, which could be corrupted if you perform assignment to the struct as a whole but not if you merely access individual members, and the rest of the elements are additional space in the allocated object which you are free to use however you like, as long as you obey alignment requirements (and char has no alignment requirements).

If you are worried that the possibility of overlapping with padding bytes in the struct might somehow invoke nasal demons, you could avoid this by replacing the 1 in [1] with a value which ensures that there is no padding at the end of the struct. A simple but wasteful way to do this would be to make a struct with identical members except no array at the end, and use s[sizeof struct that_other_struct]; for the array. Then, p->s[i] is clearly defined as an element of the array in the struct for i<sizeof struct that_other_struct and as a char object at an address following the end of the struct for i>=sizeof struct that_other_struct.

Edit: Actually, in the above trick for getting the right size, you might also need to put a union containing every simple type before the array, to ensure that the array itself begins with maximal alignment rather than in the middle of some other element's padding. Again, I don't believe any of this is necessary, but I'm offering it up for the most paranoid of the language-lawyers out there.

Edit 2: The overlap with padding bytes is definitely not an issue, due to another part of the standard. C requires that if two structs agree in an initial subsequence of their elements, the common initial elements can be accessed via a pointer to either type. As a consequence, if a struct identical to struct T but with a larger final array were declared, the element s[0] would have to coincide with the element s[0] in struct T, and the presence of these additional elements could not affect or be affected by accessing common elements of the larger struct using a pointer to struct T.

谁许谁一生繁华 2024-09-26 10:58:43

是的,这在技术上是未定义的行为。

请注意,至少有三种方法可以实现“struct hack”:

(1) 声明大小为 0 的尾随数组(遗留代码中最“流行”的方式)。这显然是 UB,因为零大小数组声明在 C 中始终是非法的。即使它能够编译,该语言也不保证任何违反约束的代码的行为。

(2) 声明具有最小合法大小的数组 - 1 (您的情况)。在这种情况下,任何尝试获取指向 p->s[0] 的指针并将其用于超出 p->s[1] 范围的指针算术都是未定义的行为。例如,调试实现可以​​生成一个带有嵌入范围信息的特殊指针,每次您尝试创建超出 p->s[1] 的指针时,该指针都会被捕获。

(3) 声明“非常大”的数组,例如 10000。这个想法是,声明的大小应该大于实际操作中可能需要的任何大小。该方法在数组访问范围方面没有 UB。然而,在实践中,我们当然总是会分配较小的内存量(仅分配真正需要的内存量)。我不确定这样做的合法性,即我想知道为对象分配比对象声明的大小更少的内存有多合法(假设我们从不访问“未分配”成员)。

Yes, it is technically undefined behavior.

Note, that there are at least three ways to implement the "struct hack":

(1) Declaring the trailing array with size 0 (the most "popular" way in legacy code). This is obviously UB, since the zero size array declarations are always illegal in C. Even if it does compile, the language makes no guarantees about the behavior of any constraint-violating code.

(2) Declaring the array with minimal legal size - 1 (your case). In this case any attempts to take pointer to p->s[0] and use it for pointer arithmetic that goes beyond p->s[1] is undefined behavior. For example, a debugging implementation is allowed to produce a special pointer with embedded range information, which will trap every time you attempt to create a pointer beyond p->s[1].

(3) Declaring the array with "very large" size like 10000, for example. The idea is that the declared size is supposed to be larger than anything you might need in actual practice. This method is free of UB with regard to array access range. However, in practice, of course, we will always allocate smaller amount of memory (only as much as really needed). I'm not sure about the legality of this, i.e. I wonder how legal it is to allocate less memory for the object than the declared size of the object (assuming we never access the "non-allocated" members).

一指流沙 2024-09-26 10:58:43

标准非常明确,您无法访问数组末尾之外的内容。 (并且通过指针没有帮助,因为您甚至不允许在数组结束后将指针增加到超过一)。

以及“在实践中工作”。我见过 gcc/g++ 优化器使用标准的这一部分,因此在遇到这个无效的 C 时会生成错误的代码。

The standard is quite clear that you cannot access things beside the end of an array. (and going via pointers does not help, as you are not allowed to even increment pointers past one after array end).

And for "working in practise". I've seen gcc/g++ optimizer using this part of the standard thus generating wrong code when meeting this invalid C.

酷炫老祖宗 2024-09-26 10:58:43

如果编译器接受类似的内容,

typedef struct {
  int len;
  char dat[];
};

我认为很明显它必须准备好接受“dat”上超出其长度的下标。另一方面,如果有人编写如下代码:

typedef struct {
  int whatever;
  char dat[1];
} MY_STRUCT;

然后稍后访问 somestruct->dat[x];我不认为编译器有任何义务使用可以处理较大 x 值的地址计算代码。我认为如果想要真正安全,正确的范例更像是:

#define LARGEST_DAT_SIZE 0xF000
typedef struct {
  int whatever;
  char dat[LARGEST_DAT_SIZE];
} MY_STRUCT;

然后执行 (sizeof(MYSTRUCT)-LARGEST_DAT_SIZE +desired_array_length) 字节的 malloc(请记住,如果desired_array_length大于LARGEST_DAT_SIZE,结果可能未定义)。

顺便说一句,我认为禁止零长度数组的决定是一个不幸的决定(Turbo C 等一些较旧的方言支持它),因为零长度数组可以被视为编译器必须生成可使用较大索引的代码的标志。

If a compiler accepts something like

typedef struct {
  int len;
  char dat[];
};

I think it's pretty clear that it must be ready to accept a subscript on 'dat' beyond its length. On the other hand, if someone codes something like:

typedef struct {
  int whatever;
  char dat[1];
} MY_STRUCT;

and then later accesses somestruct->dat[x]; I would not think the compiler is under any obligation to use address-computation code which will work with large values of x. I think if one wanted to be really safe, the proper paradigm would be more like:

#define LARGEST_DAT_SIZE 0xF000
typedef struct {
  int whatever;
  char dat[LARGEST_DAT_SIZE];
} MY_STRUCT;

and then do a malloc of (sizeof(MYSTRUCT)-LARGEST_DAT_SIZE + desired_array_length) bytes (bearing in mind that if desired_array_length is larger than LARGEST_DAT_SIZE, the results may be undefined).

Incidentally, I think the decision to forbid zero-length arrays was an unfortunate one (some older dialects like Turbo C support it) since a zero-length array could be regarded as a sign that the compiler must generate code that will work with larger indices.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文