容器无法启动:用于继续的Java运行时环境的内存不足

发布于 2025-02-13 00:54:57 字数 2233 浏览 0 评论 0原文

我们有一个在Java 8上运行的企业应用程序。部署环境是构建&通过BitBucket管道更新。我有一个图形显示环境的高级体系结构。除了某些特定于应用程序的环境变量外,我们还有两个运行相同配置的应用程序服务器。

直到一周前,成功的管道运行后,这一切都很好,其中一台服务器上的两个应用程序实例停止处理以下错误:

There is insufficient memory for the Java Runtime Environment to continue.
Cannot create GC thread. Out of system resources.

两个实例在另一台服务器上都可以正常工作。相反,容器无法在此服务器上启动。

解决方案

尝试了以下信息:

可能的原因: 该系统不含物理RAM或交换空间 该过程启用了压缩机运行,Java堆可能会阻止本机堆的生长。

可能的解决方案:

  • 减少系统上的内存负载
  • 增加物理内存或交换空间
  • 检查交换备份商店是否完全
  • 降低Java堆大小(-XMX/-XMS)
  • 减少Java线程的数量
  • 减小Java线程堆栈尺寸(-XSS)
  • 设置较大的代码带有-xx的缓存:保留codecachesize =

我们尝试过:

  1. 添加更多交换内存。当我们尝试从4GB到9GB的交易时,该服务器具有8GB RAM。
  2. 播放了堆尺寸XMS& XMX从1.28m到4096m。
  3. 将此服务器上的RAM增加到16GB,而另一个工作人员仍在8GB上使用。

这是记忆的方式&交换消费看起来像:

free -mh
              total        used        free      shared  buff/cache   available
Mem:           15Gi       378Mi        12Gi       1.0Mi       2.9Gi        14Gi
Swap:           9Gi          0B         9Gi

我有几个相关文物的链接。其中包括完成docker logs输出docker信息的输出 on 失败的服务器操作服务器

这就是docker ps -a吸引我们的原因:

:~$ docker ps -a
CONTAINER ID   IMAGE                                                                                  COMMAND                  CREATED        STATUS                    PORTS                                       NAMES
d29747bf2ad3   :a7608a838625ae945bd0a06fea9451f8bf11ebe4   "catalina.sh run"        10 hours ago   Exited (1) 10 hours ago                                               jbbatch
0951b6eb5d42   :a7608a838625ae945bd0a06fea9451f8bf11ebe4   "catalina.sh run"        10 hours ago   Exited (1) 10 hours ago                                               jbapp

我们现在已经尝试了几乎所有解决方案堆栈溢出的解决方案。我们缺少什么?

We have an enterprise application running on Java 8. The deployment environment is built & updated through Bitbucket pipelines. I have a graphic showing the high-level architecture of the environment. We have two app servers running identical configurations apart from some application specific environment variables.

It was all working well until a week ago when after a successful pipeline run, the 2 app instances on one of the servers stopped working with the following error:

There is insufficient memory for the Java Runtime Environment to continue.
Cannot create GC thread. Out of system resources.

Both the instances are working fine on the other server. In contrast, the containers fail to start on this server.

Solutions Tried

The error accompanies the following information:

Possible reasons:
The system is out of physical RAM or swap space
The process is running with CompressedOops enabled, and the Java Heap may be blocking the growth of the native heap.

Possible solutions:

  • Reduce memory load on the system
  • Increase physical memory or swap space
  • Check if swap backing store is full
  • Decrease Java heap size (-Xmx/-Xms)
  • Decrease number of Java threads
  • Decrease Java thread stack sizes (-Xss)
  • Set larger code cache with -XX:ReservedCodeCacheSize=

We have tried:

  1. Adding more swap memory. The server has 8GB of RAM while we have tried the swap from 4GB to 9GB.
  2. Played with the heap sizes Xms & Xmx from 128m to 4096m.
  3. Increased the RAM on this server to 16GB while the other server that works still does on 8GB.

Here is how the memory & swap consumption looks like:

free -mh
              total        used        free      shared  buff/cache   available
Mem:           15Gi       378Mi        12Gi       1.0Mi       2.9Gi        14Gi
Swap:           9Gi          0B         9Gi

I have links to several related artifacts. These include the complete docker logs output and the output of docker info on the failing server and the operational server.

This is what docker ps -a gets us:

:~$ docker ps -a
CONTAINER ID   IMAGE                                                                                  COMMAND                  CREATED        STATUS                    PORTS                                       NAMES
d29747bf2ad3   :a7608a838625ae945bd0a06fea9451f8bf11ebe4   "catalina.sh run"        10 hours ago   Exited (1) 10 hours ago                                               jbbatch
0951b6eb5d42   :a7608a838625ae945bd0a06fea9451f8bf11ebe4   "catalina.sh run"        10 hours ago   Exited (1) 10 hours ago                                               jbapp

We are out of ideas right now as we have tried almost all the solutions on stack overflow. What are we missing?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(2

网白 2025-02-20 00:54:57

我看到您的Docker图像使用Ubuntu 22.04 LTS作为其基础。最近,基本的Java图像在此LTS版本之上进行了重建,这在较旧的Docker Runtimes上引起了很多问题。这很可能是您所经历的。它与内存无关,而是与docker不兼容与用作基本图像的较新的Linux版本。

您的操作服务器具有Docker Server版本20.10.10,而失败的服务器具有20.10.09版本。不兼容的问题完全解决了Docker 20.10.10。有关不兼容问题的更多技术详细信息,请在这里

解决方案将是将故障服务器升级到至少Docker 20.10.10。

I see that your Docker image uses Ubuntu 22.04 LTS as its base. Recently base Java images were rebuilt on top of this LTS version, which caused a lot of issues on older Docker runtimes. Most likely this is what you're experiencing. It has nothing to do with memory, but rather with Docker incompatibility with a newer Linux version used as a base image.

Your operational server has Docker server version 20.10.10, while the failing server has version 20.10.09. The incompatibility issue was fixed exactly in Docker 20.10.10. Some more technical details on the incompatibility issue are available here.

The solution would be to upgrade the failing server to at least Docker 20.10.10.

聽兲甴掵 2025-02-20 00:54:57

我有同样的错误。
WAS的输出

# docker info

....
Security Options:
 seccomp
  WARNING: You're not using the default seccomp profile
  Profile: /etc/docker/seccomp.json
....

解决了问题,也许可以在使用/ETC/Docker/seccompp.jsonn进行调整 - 我尝试并失败了。

  security_opt:
    - seccomp:unconfined

通过将docker -compose.yml放入服务以及删除和重新创建容器的

docker rm <container_name>
docker-compose up -d <service_name>

情况下

I had had the same error.
Output of

# docker info

was:

....
Security Options:
 seccomp
  WARNING: You're not using the default seccomp profile
  Profile: /etc/docker/seccomp.json
....

The issue was resolved by putting

  security_opt:
    - seccomp:unconfined

in the docker-compose.yml for the service and removing and recreating the container

docker rm <container_name>
docker-compose up -d <service_name>

Maybe the same result could be achieved having /etc/docker/seccomp.jsonn tweaked - I tried and failed.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文