换风扇后系统超慢，日志如下，请帮忙分析

Question

有台rp7410,换了PCI风扇后，重启系统，发现系统超慢，在event.log里边发现在了一些信息如下：
vent data from monitor:

Event Time..........: Tue Aug 2 19:01:33 2010
Severity............: INFORMATION
Monitor.............: dm_chassis
Event #.............: 1188               
System..............: XXX

Summary:
     Chassis Code incoming FIFO is full.

Description of Error:

     Chassis Code Keyword CCI_FULL.

     The buffer (FIFO) holding incoming reflected chassis codes is full. The
     cell utilities hardware signals the PDC the buffer is full by setting a
     bit in the Dillon Micro Status Register.
Probable Cause / Recommended Action:

     Cause:
        The PDC is hung and not reporting to the CCI work bit or the reflected
        chassis codes are coming in so fast the PDC can't respond fast enough.
     
     Action:
        Look for failure information within the partition as to why PDC would
        hang or not respond quickly. If this is the first occurrence, Reset.
        If more than once, replace cell board. The physical location in the
        chassis code should point to a root cell.

Additional Event Data:
     System IP Address...: XXXXX
     Event Id............: 0x4c73522f00000000
     Monitor Version.....: A.01.00
     Event Class.........: System
     Client Configuration File...........:
     /var/stm/config/tools/monitor/default_dm_chassis.clcfg
     Client Configuration File Version...: A.02.00
          Qualification criteria met.
               Number of events..: 1
     Associated OS error log entry id(s):
          None
     Additional System Data:
          System Model Number.............: 9000/800/rp7410  
          EMS Version.....................: A.03.20
          STM Version.....................: A.38.00
     Latest information on this event:
          http://docs.hp.com/hpux/content/hardware/ems/dm_chassis.htm

v-v-v-v-v-v-v-v-v-v-v-v-v    D  E  T  A  I  L  S    v-v-v-v-v-v-v-v-v-v-v-v-v

Chassis Log Timestamp : Tue Aug 2 19:10:33 2010
Chassis Code Encoded Field : 0x2060121035012fcf
   Alert Level             : 1 (Service processor alert, no failure detected,
   forward progress logs and timer requests only.)
   Source FRU              : 3 (PDH)
   Source FRU Detail       : 5 (platform registers)
   Source ID               : 1 (platform dependent)
   Event Detail            : 0 (no problem detail )
   Caller Activity         : 2 (operation)
   Caller Subactivity      : 252 (unknown)
   Activity Status         : 15 (implementation dependent)
   Reporting Entity Type   : 6 (PDH controller)
   Reporting Entity ID     : 1
   Data Type               : 4 (physical location)
   Message ID              : 2
FRU Physical Location:   0x00ffff01ff00ff91
   FRU Source          = 9 (cell)
   Source Detail       = 1 (cell controller)
   Cabinet Location    = 0
   Cell Location       = 1

>---------- End Event Monitoring Service Event Notification ----------<

>------------ Event Monitoring Service Event Notification ------------<

Notification Time:

sent Event Monitor notification information:

/system/events/chassis/chassis_log is >= 1.
Its current value is INFORMATION(1).

Event data from monitor:

Event Time..........:
Severity............: INFORMATION
Monitor.............: dm_chassis
Event #.............: 1029               
System..............:

Summary:
     PDC is finished booting..

Description of Error:

     Chassis Code Keyword BOOT_BOOT_COMPLETE.

     This chassis code is sent immediately before the point where PDC branches
     to ISL. It is the last chassis code sent by the monarch CPU in the core
     cell before ISL runs and tells the system utilities that ISL is now
     running in the partition.
Probable Cause / Recommended Action:

     Cause:
        Forward progress
     
     Action:
        None

Additional Event Data:
     System IP Address...: XXXX
     Event Id............: 0x4c73523d00000000
     Monitor Version.....: A.01.00
     Event Class.........: System
     Client Configuration File...........:
     /var/stm/config/tools/monitor/default_dm_chassis.clcfg
     Client Configuration File Version...: A.02.00
          Qualification criteria met.
               Number of events..: 1
     Associated OS error log entry id(s):
          None
     Additional System Data:
          System Model Number.............: 9000/800/rp7410  
          EMS Version.....................: A.03.20
          STM Version.....................: A.38.00
     Latest information on this event:
          http://docs.hp.com/hpux/content/hardware/ems/dm_chassis.htm

v-v-v-v-v-v-v-v-v-v-v-v-v    D  E  T  A  I  L  S    v-v-v-v-v-v-v-v-v-v-v-v-v

Chassis Log Timestamp :
Chassis Code Encoded Field : 0xa0004610af000065
   Alert Level             : 1 (Service processor alert, no failure detected,
   forward progress logs and timer requests only.)
   Source FRU              : 10 (protection domain)
   Source FRU Detail       : 15 (unknown)
   Source ID               : 0 (platform dependent)
   Event Detail            : 0 (unknown)
   Caller Activity         : 0 (unspecified)
   Caller Subactivity      : 6 (implementation dependent)
   Activity Status         : 5 (implementation dependent)
   Reporting Entity Type   : 0 (system firmware)
   Reporting Entity ID     : 4
   Data Type               : 20 (major changes in system state)
   Message ID              : 6
Chassis Code Data Field    : 0x0000000000000001
   System State            : 1 (boot complete)
   LED Run                 : 0 (off (default))
   LED Attention           : 0 (unknown)
   LED Stopped             : 0 (off (default))

>---------- End Event Monitoring Service Event Notification ----------<

另外，高手请指导下， Look for failure information within the partition as to why PDC would
        hang or not respond quickly. If this is the first occurrence, Reset.
        If more than once, replace cell board. The physical location in the
        chassis code should point to a root cell.
请问，这个PDC怎样reset,谢谢。

Answer 1

关注

Answer 2

本帖最后由 zqm871 于 2010-09-14 11:32 编辑

最近国网公司检查，所以没有批准不让动设备。风扇应该没有问题。二是reset PDC应该不在开机中断后，ser->reset .
好不容易找到个资料发现：
xd, ru - Reset MP
MP:CM> xd
    MP Diagnostics and Reboot menu
    ------------------------------
        1. Parameters checksum
        2. Ping
        3. Soft Reset the Master MP
        4. Soft Reset the Slave MP
        5. Hard Reset the Slave MP
    Enter your choice: 2
    Enter IP address: 15.41.32.1
    Test result is: PASS
MP:CM> ru
This command resets the selected MP bus device.
    B - BPS  (Bulk Power Supply)
    A - PACI (Partition Console Interface)
    G - MP   (Management Processor)
    H - PDHC (Cell Board Controller)
        Select device: h
    Enter cell number: 0
    Do you want to reset the Cell PDH Controller Slot 0? (Y/[N])y
 This command is typically used after loading firmware in order to reset an
    entity and start execution of the new code, as is the case with the PDCH code.
 Resetting PDCH will not reset the PDC on the cell board! Use the “rs”
    commnad to reset PDC.
Caution: Always follow firmware install procedures and only
          execute a reset when directed to by the procedures!
这个资料上显示， Use the “rs”   commnad to reset PDC. 准备动手做。

Answer 3

关注一下。

Answer 4

不好意思，由于机器在远程容灾机房，所以，去处理比较麻烦，提交的进入机房维修许可还没下来呢，不过，正在办理过程中，谢谢各位帮忙。

Answer 5

等着看LZ处理过程

Answer 6

1、进入BCH或PDC后，SER菜单里面有reset吗，这个倒没注意过，谢谢。我试一下。
2、楼上那位大哥，CELL报错我也看到了，只是，attention建议，第一次发生CCC_I full，可以先reset,如果发生不只一次，再建议换cell板，不过，这个信息不一定跟机器变慢有直接的关系。
3、michael1983 这位老兄也分析了，谢谢，风扇应该是兼容的，风扇型号是一样的，如果有一种可能就是没有插好，这个只能关机再插一次试试，顺便，然后进PDC，ser->reset一次。
谢谢楼上三位兄弟分析，望各位高手继续帮忙分析，我也将故障处理情况实时的记录下来给各位，谢谢。

Answer 7

怀疑风扇换的有问题
要么不兼容
要么插的有问题

Answer 8

本帖最后由 tyzhang3 于 2010-08-25 21:06 编辑

cell board报错啦，大哥看一下吧，cell 1报的

Answer 9

进BCH的SER菜单