C - 无法释放分配的内存
我当前正在开发的应用程序有问题。在这个程序中,我必须从文本文件中读取大量(数十亿)数据并对其进行管理,但由于这是一个两个学生的项目,因此阅读部分将由我的伙伴开发。出于测试原因,我编写了一个小程序来生成伪随机结构来代替我的伴侣将要做的事情。
问题如下:为了释放内存,可以丢弃大量生成的数据(由于冗余)。但即使调用 free() 函数,内存使用量也会不断增加。因此,我尝试开发一个调试应用程序,它可以简单地生成一大块数据并立即释放它。并重复数千次。 好吧,我无法理解原因,但分配给进程的内存增长到 ~1.8 GB ram,然后崩溃。为什么?最奇怪的事情,让我觉得有很多我不太理解的是,当进程崩溃时,malloc 不会返回 NULL 指针,因为当 readCycles == 6008 时进程总是崩溃并绕过 NULL 检查。
我已经在 StackOverflow 上阅读了其他相关主题,并且理解了为什么 free() 不会减少分配给我的进程的内存。没关系。但为什么内存使用量不断增长呢? malloc 不应该分配以前释放的内存而不是不断请求新的内存吗?
这是我的代码中最相关的部分:
#define NREAD 1000
#define READCYCLES 10000
#define N_ALPHA_ILLUMINA 7
#define N_ALPHA_SOLID 5
#define SEQLEN 76
typedef struct{
char* leftDNA;
char* leftQuality;
unsigned long int leftRow;
char* rightDNA;
char* rightQuality;
unsigned long int rightRow;
} MatePair;
unsigned long int readCycles = 0;
MatePair* readStream(MatePair* inputStream, short* eof, unsigned long int* inputSize){
double r;
unsigned long int i, j;
unsigned long int leftRow;
int alphabet[] = {'A', 'C', 'G', 'T', 'N'};
inputStream = (MatePair*) malloc (sizeof(MatePair) * (NREAD + 1));
printf("%d\n", readCycles);
if (inputStream == NULL){
(*eof) = 1;
return;
}
for (i = 0; i < NREAD; i++){
leftRow = readCycles * NREAD + i;
inputStream[i].leftDNA = (char*) malloc (SEQLEN);
inputStream[i].rightDNA = (char*) malloc (SEQLEN);
inputStream[i].leftQuality = (char*) malloc (SEQLEN);
inputStream[i].rightQuality = (char*) malloc (SEQLEN);
for (j = 0; j < SEQLEN; j++){
r = rand() / (RAND_MAX + 1);
inputStream[i].leftDNA[j] = alphabet[(int)(r * 5)];
inputStream[i].rightDNA[j] = alphabet[(int)(r * 5)];
inputStream[i].leftQuality[j] = (char) 64 + (int)(r * 60);
inputStream[i].rightQuality[j] = (char) 64 + (int)(r * 60);
}
inputStream[i].leftDNA[SEQLEN - 1] = '\0';
inputStream[i].rightDNA[SEQLEN - 1] = '\0';
inputStream[i].leftQuality[SEQLEN - 1] = '\0';
inputStream[i].rightQuality[SEQLEN - 1] = '\0';
inputStream[i].leftRow = leftRow;
inputStream[i].rightRow = leftRow;
}
inputStream[i].leftRow = -1;
readCycles++;
(*inputSize) = NREAD;
(*eof) = readCycles > READCYCLES;
return inputStream;
}
int main(int argc, char* argv[]){
short eof = 0;
unsigned long int inputSize = 0;
MatePair* inputStream = NULL;
while (!eof){
inputStream = readStream(inputStream, &eof, &inputSize);
free(inputStream);
inputStream = NULL;
}
return 0;
}
我忘了提及这一点,但在此处发布之前,我没有调用 free(inputStream),而是尝试调用 freeMemory(inputStream)。但不确定这是否是正确的做法。
void freeMemory(MatePair* memblock){
for ( ; memblock->leftRow != 1; memblock++){
free(memblock -> leftDNA);
free(memblock -> leftQuality);
free(memblock -> rightDNA);
free(memblock -> rightQuality);
}
}
I have a problem with an application I'm currently developing. In this program I have to read huge amounts (billions) of data from text files and manage them consequently, but since it's a two students project, the reading part will be developed by my mate. For testing reason I wrote a small procedures that generates pseudo-random structures to replace what my mate will do.
The problem is the following: a big amount of the generated data (due to redundancy) can be discarded in order to free its memory. But even invoking the free() function the memory usage keeps growing. So I tried to develop a debug application that simply generates a chunk of data and immediately frees it. And repeats that for thousands of times. Well, I can't grasp the reason, but the memory allocated to the process grows to ~1.8 GB ram and then crashes. Why? The strangest thing, that makes me thing there's a lot I'm not understanding well, is that when the process crashes the malloc does NOT return a NULL pointer, because the process always crashes when readCycles == 6008 and bypasses the NULL check.
I already read other related topics here on StackOverflow and I understood why free() doesn't reduce the memory allocated to my process. That's fine. But why the memory usage keeps growing? Shouldn't the malloc allocate previously freed memory instead of constantly requesting new one?
This is the most relevant part of my code:
#define NREAD 1000
#define READCYCLES 10000
#define N_ALPHA_ILLUMINA 7
#define N_ALPHA_SOLID 5
#define SEQLEN 76
typedef struct{
char* leftDNA;
char* leftQuality;
unsigned long int leftRow;
char* rightDNA;
char* rightQuality;
unsigned long int rightRow;
} MatePair;
unsigned long int readCycles = 0;
MatePair* readStream(MatePair* inputStream, short* eof, unsigned long int* inputSize){
double r;
unsigned long int i, j;
unsigned long int leftRow;
int alphabet[] = {'A', 'C', 'G', 'T', 'N'};
inputStream = (MatePair*) malloc (sizeof(MatePair) * (NREAD + 1));
printf("%d\n", readCycles);
if (inputStream == NULL){
(*eof) = 1;
return;
}
for (i = 0; i < NREAD; i++){
leftRow = readCycles * NREAD + i;
inputStream[i].leftDNA = (char*) malloc (SEQLEN);
inputStream[i].rightDNA = (char*) malloc (SEQLEN);
inputStream[i].leftQuality = (char*) malloc (SEQLEN);
inputStream[i].rightQuality = (char*) malloc (SEQLEN);
for (j = 0; j < SEQLEN; j++){
r = rand() / (RAND_MAX + 1);
inputStream[i].leftDNA[j] = alphabet[(int)(r * 5)];
inputStream[i].rightDNA[j] = alphabet[(int)(r * 5)];
inputStream[i].leftQuality[j] = (char) 64 + (int)(r * 60);
inputStream[i].rightQuality[j] = (char) 64 + (int)(r * 60);
}
inputStream[i].leftDNA[SEQLEN - 1] = '\0';
inputStream[i].rightDNA[SEQLEN - 1] = '\0';
inputStream[i].leftQuality[SEQLEN - 1] = '\0';
inputStream[i].rightQuality[SEQLEN - 1] = '\0';
inputStream[i].leftRow = leftRow;
inputStream[i].rightRow = leftRow;
}
inputStream[i].leftRow = -1;
readCycles++;
(*inputSize) = NREAD;
(*eof) = readCycles > READCYCLES;
return inputStream;
}
int main(int argc, char* argv[]){
short eof = 0;
unsigned long int inputSize = 0;
MatePair* inputStream = NULL;
while (!eof){
inputStream = readStream(inputStream, &eof, &inputSize);
free(inputStream);
inputStream = NULL;
}
return 0;
}
I forgot to mention that, but before posting here, instead of calling free(inputStream), I tried invoking freeMemory(inputStream). Not sure if it's the correct way of doing it, though.
void freeMemory(MatePair* memblock){
for ( ; memblock->leftRow != 1; memblock++){
free(memblock -> leftDNA);
free(memblock -> leftQuality);
free(memblock -> rightDNA);
free(memblock -> rightQuality);
}
}
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(2)
内存泄漏。您调用了多少个“malloc()”,必须使用多少个“free()”来释放堆上所有分配的内存。
因此,
这些“malloc()”函数必须与 free() 配对。
Memory leaks. How many 'malloc()' you have called, how many 'free()' you must use to free all allocated memory on the heap.
Thus,
these 'malloc()' functions must be paired with free().
您没有释放读取循环中分配的所有成员,因此您每次都会丢失内存。请记住,您必须释放使用 malloc 分配的所有内容,而不仅仅是数组。
好吧,看看你的编辑,你的 freeMemory 仍然是错误的。试试这个;
你的 free(memblock) 处于循环中,这是不应该的,而且我倾向于在释放时使用与 mallocing 相同的迭代序列。您还需要在每次 malloc 之后进行错误检查,并决定此时如何处理 NULL。
You're not freeing all members allocated within the read loop, hence you're losing memory eahc time. Remember, you have to free everything you allocate with a malloc, not just your array.
Ok, Just look at your edit, and your freeMemory is still wrong. Try this;
Your free(memblock) was in the loop, which it shouldn't have been, and I'd tend to use the same iteration sequence on freeing as mallocing. You also need to error check after each malloc, and decide what to do with a NULL at that point.