C: 使用 opendir 和 open 打开的文件太多

发布于 2024-12-01 04:49:45 字数 1258 浏览 2 评论 0原文

我正在循环中使用以下代码将大约 6000 个文本文件读入内存：

void readDocs(const char *dir, char **array){
DIR *dp = opendir(dir);;
struct dirent *ep;
struct stat st;
static uint count = 0;
if (dp != NULL){
  while (ep = readdir(dp)){    // crawl through directory
    char name[strlen(dir) + strlen(ep->d_name) + 2];
    sprintf(name, "%s/%s", dir, ep->d_name);

    if(ep->d_type == DT_REG){  // regular file
      stat(name, &st);
      array[count] = (char*) malloc(st.st_size);
      int f;
      if((f = open(name, O_RDONLY)) < 0) perror("open:  ");
      read(f, array[count], st.st_size));
      if(close(f) < 0) perror("close: ");
      ++count;
    }    

    else if(ep->d_type == DT_DIR && strcmp(ep->d_name, "..") && strcmp(ep->d_name, "."))
      // go recursive through sub directories
      readDocs(name, array);
  }
}
}

在迭代 2826 中，打开第 2826 个文件时出现“打开文件过多”错误。到目前为止，关闭操作没有发生错误。
由于它总是在第 2826 次迭代中挂起，我不认为我应该等到调用 close();
后文件真正关闭我在使用 fopen、fread 和 fclose 时遇到了同样的问题。
我认为这与此片段的上下文无关，但如果您这样做，我会提供它。
感谢您抽出时间！

编辑：
我让程序进入睡眠状态并检查/proc//fd/（感谢 nos）。就像你怀疑的那样，文件描述符恰好有 1024 个，我发现这是一个通常的限制。
+ 我给了你从目录和所有子目录中读取文档的整个函数
+ 该程序在 Linux 上运行！抱歉忘记了！

原文

I am reading about 6000 text-files into memory with the following code in a loop:

void readDocs(const char *dir, char **array){
DIR *dp = opendir(dir);;
struct dirent *ep;
struct stat st;
static uint count = 0;
if (dp != NULL){
  while (ep = readdir(dp)){    // crawl through directory
    char name[strlen(dir) + strlen(ep->d_name) + 2];
    sprintf(name, "%s/%s", dir, ep->d_name);

    if(ep->d_type == DT_REG){  // regular file
      stat(name, &st);
      array[count] = (char*) malloc(st.st_size);
      int f;
      if((f = open(name, O_RDONLY)) < 0) perror("open:  ");
      read(f, array[count], st.st_size));
      if(close(f) < 0) perror("close: ");
      ++count;
    }    

    else if(ep->d_type == DT_DIR && strcmp(ep->d_name, "..") && strcmp(ep->d_name, "."))
      // go recursive through sub directories
      readDocs(name, array);
  }
}
}

In iteration 2826 i get an "Too many open files" error when opening the 2826th file.
No error occured in the close operation until this point.
Since it always hangs in the 2826th iteration i do not believe that i should wait until a file is realy closed after calling close();
I had the same issue using fopen, fread and fclose.
I don't think it has to do with the context of this snippet but if you do i will provide it.
Thanks for your time!

EDIT:
I put the program to sleep and checked /proc//fd/ (thanks to nos). Like you suspected there were exactly 1024 file descriptors which i found to be a usual limit.
+ i gave you the whole function which reads documents out of a directory and all subdirectories
+ the program runs on Linux! Sorry for forgetting that!

分享到QQ

分享到微博