当前位置： 文江博客话题详情

Postgres HA（基于 WAL 传送）失败

发布于 2024-08-18 04:09:24 字数 6028 浏览 14 评论 0原文

我希望有人能帮我解决 WAL 运输和热备问题。我的备用系统愉快地运行了数周，然后突然开始寻找不存在的 .history 文件。然后它就崩溃了，如果不重建备用数据库，我就无法成功重新启动它。

两个系统都运行 CentOS 4.5 和 postgres 8.4.1。他们使用 NFS 将生产中的 WAL 文件存储在备用数据库上。

日志的相关部分，以及我的评论：

[** Recovery is running normally **]

Trigger file            : /tmp/pgsql.trigger
Waiting for WAL file    : 00000001000000830000005B
WAL file path           : /var/tafkan_backup_from_db1/00000001000000830000005B
Restoring to            : pg_xlog/RECOVERYXLOG
Sleep interval          : 2 seconds
Max wait interval       : 0 forever
Command for restore     : cp "/var/tafkan_backup_from_db1/00000001000000830000005B" "pg_xlog/RECOVERYXLOG"
Keep archive history    : 00000001000000830000004D and later
WAL file not present yet. Checking for trigger file...
WAL file not present yet. Checking for trigger file...
WAL file not present yet. Checking for trigger file...
running restore         : OK

Trigger file            : /tmp/pgsql.trigger
Waiting for WAL file    : 00000001000000830000005B
WAL file path           : /var/tafkan_backup_from_db1/00000001000000830000005B
Restoring to            : pg_xlog/RECOVERYXLOG
Sleep interval          : 2 seconds
Max wait interval       : 0 forever
Command for restore     : cp "/var/tafkan_backup_from_db1/00000001000000830000005B" "pg_xlog/RECOVERYXLOG"
Keep archive history    : 000000000000000000000000 and later
running restore         : OK

[** All of a sudden it starts looks for .history files **]

Trigger file            : /tmp/pgsql.trigger
Waiting for WAL file    : 00000002.history
WAL file path           : /var/tafkan_backup_from_db1/00000002.history
Restoring to            : pg_xlog/RECOVERYHISTORY
Sleep interval          : 2 seconds
Max wait interval       : 0 forever
Command for restore     : cp "/var/tafkan_backup_from_db1/00000002.history" "pg_xlog/RECOVERYHISTORY"
Keep archive history    : 000000000000000000000000 and later
running restore         :cp: cannot stat `/var/tafkan_backup_from_db1/00000002.history': No such file or directory
cp: cannot stat `/var/tafkan_backup_from_db1/00000002.history': No such file or directory
cp: cannot stat `/var/tafkan_backup_from_db1/00000002.history': No such file or directory
cp: cannot stat `/var/tafkan_backup_from_db1/00000002.history': No such file or directory
not restored
history file not found
Trigger file            : /tmp/pgsql.trigger
Waiting for WAL file    : 00000001.history
WAL file path           : /var/tafkan_backup_from_db1/00000001.history
Restoring to            : pg_xlog/RECOVERYHISTORY
Sleep interval          : 2 seconds
Max wait interval       : 0 forever
Command for restore     : cp "/var/tafkan_backup_from_db1/00000001.history" "pg_xlog/RECOVERYHISTORY"
Keep archive history    : 000000000000000000000000 and later
running restore         :cp: cannot stat `/var/tafkan_backup_from_db1/00000001.history': No such file or directory
cp: cannot stat `/var/tafkan_backup_from_db1/00000001.history': No such file or directory
cp: cannot stat `/var/tafkan_backup_from_db1/00000001.history': No such file or directory
cp: cannot stat `/var/tafkan_backup_from_db1/00000001.history': No such file or directory
not restored
history file not found

[** I stopped Postgres, renamed recovery.done to recovery.conf, and restarted it. **]

Trigger file            : /tmp/pgsql.trigger
Waiting for WAL file    : 00000002.history
WAL file path           : /var/tafkan_backup_from_db1/00000002.history
Restoring to            : pg_xlog/RECOVERYHISTORY
Sleep interval          : 2 seconds
Max wait interval       : 0 forever
Command for restore     : cp "/var/tafkan_backup_from_db1/00000002.history" "pg_xlog/RECOVERYHISTORY"
Keep archive history    : 000000000000000000000000 and later
running restore         :cp: cannot stat `/var/tafkan_backup_from_db1/00000002.history': No such file or directory
cp: cannot stat `/var/tafkan_backup_from_db1/00000002.history': No such file or directory
cp: cannot stat `/var/tafkan_backup_from_db1/00000002.history': No such file or directory
cp: cannot stat `/var/tafkan_backup_from_db1/00000002.history': No such file or directory
not restored
history file not found
Trigger file            : /tmp/pgsql.trigger
Waiting for WAL file    : 0000000200000083000000A2
WAL file path           : /var/tafkan_backup_from_db1/0000000200000083000000A2
Restoring to            : pg_xlog/RECOVERYXLOG
Sleep interval          : 2 seconds
Max wait interval       : 0 forever
Command for restore     : cp "/var/tafkan_backup_from_db1/0000000200000083000000A2" "pg_xlog/RECOVERYXLOG"
Keep archive history    : 000000000000000000000000 and later
WAL file not present yet. Checking for trigger file...
WAL file not present yet. Checking for trigger file...
WAL file not present yet. Checking for trigger file...
WAL file not present yet. Checking for trigger file...

[** This file is not present. All WAL files start with 00000001. **]

有什么想法吗？我什至不知道 .history 文件是什么，并且（大多数优秀的）文档对此都不是很清楚。

附言。我希望我正在运行虚拟机，这样我就可以使用 链接文本 而不是不必担心任何应用程序级 HA 的废话:-)

更新：以下是此时备用服务器的一些日志。看起来好像有什么东西使服务器停止恢复并重新上线，但我不知道是什么。我非常确定没有任何东西可以创建触发文件。

2010-01-20 03:30:15 EST 4b3a5c63.401b LOG:  restored log file "00000001000000830000005A" from archive
2010-01-20 03:30:23 EST 4b3a5c63.401b LOG:  restored log file "00000001000000830000005B" from archive
2010-01-20 03:30:23 EST 4b3a5c63.401b LOG:  record with zero length at 83/5BFA2FF8
2010-01-20 03:30:23 EST 4b3a5c63.401b LOG:  redo done at 83/5BFA2FAC
2010-01-20 03:30:23 EST 4b3a5c63.401b LOG:  last completed transaction was at log time 2010-01-20 03:28:04.594399-05
2010-01-20 03:30:25 EST 4b3a5c63.401b LOG:  restored log file "00000001000000830000005B" from archive
2010-01-20 03:30:37 EST 4b3a5c63.401b LOG:  selected new timeline ID: 2
2010-01-20 03:30:49 EST 4b3a5c63.401b LOG:  archive recovery complete
2010-01-20 03:30:59 EST 4b3a5c62.4019 LOG:  database system is ready to accept connections
原文

I'm hoping someone can help me a WAL-shipping and warm standby issue. My standby system runs happily for weeks, then all of a sudden it starts looking for .history files that don't exist. It then craps out and I can't successfully restart it without rebuilding the standby.

Both systems are running CentOS 4.5 and postgres 8.4.1. They use NFS to store WAL files from production on the standby.

A relevant chunk of the log, with my comments:

[** Recovery is running normally **]

Trigger file            : /tmp/pgsql.trigger
Waiting for WAL file    : 00000001000000830000005B
WAL file path           : /var/tafkan_backup_from_db1/00000001000000830000005B
Restoring to            : pg_xlog/RECOVERYXLOG
Sleep interval          : 2 seconds
Max wait interval       : 0 forever
Command for restore     : cp "/var/tafkan_backup_from_db1/00000001000000830000005B" "pg_xlog/RECOVERYXLOG"
Keep archive history    : 00000001000000830000004D and later
WAL file not present yet. Checking for trigger file...
WAL file not present yet. Checking for trigger file...
WAL file not present yet. Checking for trigger file...
running restore         : OK

Trigger file            : /tmp/pgsql.trigger
Waiting for WAL file    : 00000001000000830000005B
WAL file path           : /var/tafkan_backup_from_db1/00000001000000830000005B
Restoring to            : pg_xlog/RECOVERYXLOG
Sleep interval          : 2 seconds
Max wait interval       : 0 forever
Command for restore     : cp "/var/tafkan_backup_from_db1/00000001000000830000005B" "pg_xlog/RECOVERYXLOG"
Keep archive history    : 000000000000000000000000 and later
running restore         : OK

[** All of a sudden it starts looks for .history files **]

Trigger file            : /tmp/pgsql.trigger
Waiting for WAL file    : 00000002.history
WAL file path           : /var/tafkan_backup_from_db1/00000002.history
Restoring to            : pg_xlog/RECOVERYHISTORY
Sleep interval          : 2 seconds
Max wait interval       : 0 forever
Command for restore     : cp "/var/tafkan_backup_from_db1/00000002.history" "pg_xlog/RECOVERYHISTORY"
Keep archive history    : 000000000000000000000000 and later
running restore         :cp: cannot stat `/var/tafkan_backup_from_db1/00000002.history': No such file or directory
cp: cannot stat `/var/tafkan_backup_from_db1/00000002.history': No such file or directory
cp: cannot stat `/var/tafkan_backup_from_db1/00000002.history': No such file or directory
cp: cannot stat `/var/tafkan_backup_from_db1/00000002.history': No such file or directory
not restored
history file not found
Trigger file            : /tmp/pgsql.trigger
Waiting for WAL file    : 00000001.history
WAL file path           : /var/tafkan_backup_from_db1/00000001.history
Restoring to            : pg_xlog/RECOVERYHISTORY
Sleep interval          : 2 seconds
Max wait interval       : 0 forever
Command for restore     : cp "/var/tafkan_backup_from_db1/00000001.history" "pg_xlog/RECOVERYHISTORY"
Keep archive history    : 000000000000000000000000 and later
running restore         :cp: cannot stat `/var/tafkan_backup_from_db1/00000001.history': No such file or directory
cp: cannot stat `/var/tafkan_backup_from_db1/00000001.history': No such file or directory
cp: cannot stat `/var/tafkan_backup_from_db1/00000001.history': No such file or directory
cp: cannot stat `/var/tafkan_backup_from_db1/00000001.history': No such file or directory
not restored
history file not found

[** I stopped Postgres, renamed recovery.done to recovery.conf, and restarted it. **]

Trigger file            : /tmp/pgsql.trigger
Waiting for WAL file    : 00000002.history
WAL file path           : /var/tafkan_backup_from_db1/00000002.history
Restoring to            : pg_xlog/RECOVERYHISTORY
Sleep interval          : 2 seconds
Max wait interval       : 0 forever
Command for restore     : cp "/var/tafkan_backup_from_db1/00000002.history" "pg_xlog/RECOVERYHISTORY"
Keep archive history    : 000000000000000000000000 and later
running restore         :cp: cannot stat `/var/tafkan_backup_from_db1/00000002.history': No such file or directory
cp: cannot stat `/var/tafkan_backup_from_db1/00000002.history': No such file or directory
cp: cannot stat `/var/tafkan_backup_from_db1/00000002.history': No such file or directory
cp: cannot stat `/var/tafkan_backup_from_db1/00000002.history': No such file or directory
not restored
history file not found
Trigger file            : /tmp/pgsql.trigger
Waiting for WAL file    : 0000000200000083000000A2
WAL file path           : /var/tafkan_backup_from_db1/0000000200000083000000A2
Restoring to            : pg_xlog/RECOVERYXLOG
Sleep interval          : 2 seconds
Max wait interval       : 0 forever
Command for restore     : cp "/var/tafkan_backup_from_db1/0000000200000083000000A2" "pg_xlog/RECOVERYXLOG"
Keep archive history    : 000000000000000000000000 and later
WAL file not present yet. Checking for trigger file...
WAL file not present yet. Checking for trigger file...
WAL file not present yet. Checking for trigger file...
WAL file not present yet. Checking for trigger file...

[** This file is not present. All WAL files start with 00000001. **]

Any ideas? I don't even know what .history files are, and the (mostly excellent) documentation is not very clear on any of this.

PS. I wish I was running VMs so I could use link text and not have to worry about any of this application-level HA nonsense :-)

Update: Here are some of the logs from the standby server at about this time. It looks like something made the server stop recovering and come online, but I don't know what. I'm pretty certain that nothing could have created the trigger file.

2010-01-20 03:30:15 EST 4b3a5c63.401b LOG:  restored log file "00000001000000830000005A" from archive
2010-01-20 03:30:23 EST 4b3a5c63.401b LOG:  restored log file "00000001000000830000005B" from archive
2010-01-20 03:30:23 EST 4b3a5c63.401b LOG:  record with zero length at 83/5BFA2FF8
2010-01-20 03:30:23 EST 4b3a5c63.401b LOG:  redo done at 83/5BFA2FAC
2010-01-20 03:30:23 EST 4b3a5c63.401b LOG:  last completed transaction was at log time 2010-01-20 03:28:04.594399-05
2010-01-20 03:30:25 EST 4b3a5c63.401b LOG:  restored log file "00000001000000830000005B" from archive
2010-01-20 03:30:37 EST 4b3a5c63.401b LOG:  selected new timeline ID: 2
2010-01-20 03:30:49 EST 4b3a5c63.401b LOG:  archive recovery complete
2010-01-20 03:30:59 EST 4b3a5c62.4019 LOG:  database system is ready to accept connections

如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论， 你可以免费 注册 一个本站的账号。

评论（4）

鸠书 2024-08-25 04:09:24

一种完全不同的 HA 方法可能是将 PG 数据库托管在两台计算机之间共享的 DRBD 设备上。

A totally different approach for HA might be to host the PG database on a DRBD device shared between the two machines.

回复 原文
〃安静 2024-08-25 04:09:24

您使用自己的恢复脚本/程序吗？如果是 - 请不要这样做。使用 PostgreSQL contrib 中的 pg_standby。

否则 - 只需忽略 .history 文件。

Did you use your own recovery script/program? If yes - please don't do it. Use pg_standby from PostgreSQL contrib.

Otherwise - just ignore the .history files.

回复 原文
汐鸠 2024-08-25 04:09:24

您的复制副本在某个时候上线了。 “00000002.history”正在查找时间线 00000002 的历史文件，而其余日志则以 00000001（原始时间线）开头。

在开始查找历史文件之前，我会检查您的 PostgreSQL 日志，看看是否有任何迹象表明数据库已上线，即使只是一会儿。

Your replicated copy came online at some point. "00000002.history" is looking for a history file for timeline 00000002, whereas the rest of your logs start with 00000001, which is the original timeline.

I would check your PostgreSQL logs right before it started looking for the history file to see if there's any indication the DB came online, even for a moment.

回复 原文
东走西顾 2024-08-25 04:09:24

我通过更新两台 PostgreSQL 服务器上的 CentOS 操作系统解决了这个问题。因此，我认为这是某种潜在网络错误的症状。

I was able to resolve this problem by updating the CentOS operating systems on my two PostgreSQL servers. Therefore, I think this was a symptom of an underlying networking bug of some sort.

回复 原文
~没有更多了~

关于作者

葵雨

暂无简介

文章
评论
27 人气
发私信

相关话题

更多

热门标签

操作系统 程序设计 IT运维 Linux系统管理 JavaScript 服务器应用 solaris C/C++ PHP Shell BSD Vue.js aix Oracle Python HTML 系统管理 HTML5 CSS 前端
更多

推荐作者

西西弗的石头怪

文章 0 评论 0

5397313

文章 0 评论 0

烟沫凡尘

文章 0 评论 0

一个破名字

文章 0 评论 0

萌︼了一个春

文章 0 评论 0

当爱已成负担

文章 0 评论 0

更多

友情链接

文江博客
    我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站，即表示您同意使用 Cookies 和您的相关数据。
    原文