使用 Mongo 和 Morphia 时发生内存泄漏

发布于 2025-01-06 12:46:46 字数 2716 浏览 0 评论 0原文

我发现我的应用程序中存在非常严重的内存泄漏。当尝试使用 Morphia 批量写入 Mongo 时,性能迅速下降,并且持续下降。 (即写入初始 100 条记录需要不到 1 秒,记录 8899 到 9000 需要 310 秒。)

使用 Yourkit,我可以看到 com.mongodb.BasicDBObject、com.mongodb.DBRef、com.mongodb.BasicDBList 具有非常高的保留大小(41-81%) 并持续报告为待定案。

此外,如果我将 datastore.save(...) 更改为写入磁盘,我不会看到性能下降。

我目前没有使用 MongoOptions。我应该这样吗?可以进行哪些调整来提高应用程序的性能?

我正在使用 mongo java 驱动程序版本 2.7.0 和 morphia 0.99.1-SNAPSHOT。

我在一个简单的 for 循环中调用我的控制器方法:

@RequestMapping(value = "/reports", method = RequestMethod.POST, headers = "Accept=application/json")
public ResponseEntity<Result> add(@RequestParam("title") String title,
                                                   @RequestParam("uri") String uri,
                                                   @RequestParam(value = "imageUri", required = false) String imageUri,
                                                   @RequestParam(value = "categoryIds", required = false) List<String> categoryIds,
                                                   @RequestParam(value = "expertsIds", required = false) List<String> expertIds,
                                                   @RequestParam(value = "userId", required = false) String userIdString,
                                                   @RequestParam(value = "description", required = false) String description) {

    if (imageUri == null) {
        imageUri = "";
    }

    List<Category> categories = new ArrayList<Category>();
    if (categoryIds != null && !categoryIds.isEmpty()) {
        categories = categoryService.getCategoriesById(categoryIds);
    }

    Report report = new Report(StringEscapeUtils.escapeHtml(title), uri, imageUri, categories, expertIds, description);

    UUID userID = userService.convertUserId(userIdString);
    Result result = userService.addReport(userID, report);

    return new ResponseEntity<Result>(result);
}

 public Result addReport(UUID userID, Report report) {
    Result result = reportService.add(report);
    if (!result.getErrorMessages().isEmpty()) {
        return result;
    }
    if (!doesUserExistInDb(userID)) {
        createUserWithId(userID);
    }
    updateUserWithReport(userID, report);
    return result;
}

public Result add(Report report) {
    Result result = new Result();
    try {
        datastore.save(report);
        result.addReport(report);
    } catch (ConstraintViolationException ex) {
        populateValidationErrors(result, ex);
    }
    return result;
}

public void updateUserWithReport(UUID userID, Report report) {
    User user = datastore.get(User.class, userID);
    user.getReports().add(report);
    datastore.save(user);
}

I have found a pretty severe memory leak in my application. When trying to bulk write into Mongo, using Morphia, performance dips swiftly, and continues to degrade. (i.e. initial 100 records take < 1 second to write, records 8899 to 9000 take 310 seconds.)

Using Yourkit I can see than com.mongodb.BasicDBObject, com.mongodb.DBRef, com.mongodb.BasicDBList have very high retained sizes (41-81%) and continually report as Pending Finalisation.

Additionally, if I change datastore.save(...) to write to disk instead, I do not see the performance degradation.

I am not using MongoOptions at the moment. Should I be? What tweaks are available to improve the performance of my application?

I am using mongo java driver version 2.7.0, and morphia 0.99.1-SNAPSHOT.

I call my controller method in a simple for loop:

@RequestMapping(value = "/reports", method = RequestMethod.POST, headers = "Accept=application/json")
public ResponseEntity<Result> add(@RequestParam("title") String title,
                                                   @RequestParam("uri") String uri,
                                                   @RequestParam(value = "imageUri", required = false) String imageUri,
                                                   @RequestParam(value = "categoryIds", required = false) List<String> categoryIds,
                                                   @RequestParam(value = "expertsIds", required = false) List<String> expertIds,
                                                   @RequestParam(value = "userId", required = false) String userIdString,
                                                   @RequestParam(value = "description", required = false) String description) {

    if (imageUri == null) {
        imageUri = "";
    }

    List<Category> categories = new ArrayList<Category>();
    if (categoryIds != null && !categoryIds.isEmpty()) {
        categories = categoryService.getCategoriesById(categoryIds);
    }

    Report report = new Report(StringEscapeUtils.escapeHtml(title), uri, imageUri, categories, expertIds, description);

    UUID userID = userService.convertUserId(userIdString);
    Result result = userService.addReport(userID, report);

    return new ResponseEntity<Result>(result);
}

 public Result addReport(UUID userID, Report report) {
    Result result = reportService.add(report);
    if (!result.getErrorMessages().isEmpty()) {
        return result;
    }
    if (!doesUserExistInDb(userID)) {
        createUserWithId(userID);
    }
    updateUserWithReport(userID, report);
    return result;
}

public Result add(Report report) {
    Result result = new Result();
    try {
        datastore.save(report);
        result.addReport(report);
    } catch (ConstraintViolationException ex) {
        populateValidationErrors(result, ex);
    }
    return result;
}

public void updateUserWithReport(UUID userID, Report report) {
    User user = datastore.get(User.class, userID);
    user.getReports().add(report);
    datastore.save(user);
}

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。
列表为空,暂无数据
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文