【求助】cloudstack+kvm部署在一台机器上，系统虚拟机启动失败

发布于 2021-12-03 06:07:20 字数 13094 浏览 991 评论 3

linux版本：redhat6.2

cloudstack版本：2.2.14

libvirt版本：0.9.4

环境：cloudstack management+kvm+storage在同一台设备上

现象：

1.host添加成功，添加secondary storage时，系统虚拟机Console Proxy VM和Secondary Storage VM都处于starting状态

2.virsh list有时可以看到虚拟机状态为running

virsh # list

Id Name State

----------------------------------

26 v-2-VM running

virsh #

virsh # dominfo v-2-VM

Id: 26

Name: v-2-VM

UUID: 1422832d-be18-352a-a08a-9bbff40e0d14

OS Type: hvm

State: running

CPU(s): 1

CPU time: 283.5s

Max memory: 1048576 kB

Used memory: 1048576 kB

Persistent: yes

Autostart: disable

Managed save: no

virsh #

日志：

【agent日志：】

2013-07-21 08:13:42,519 WARN [resource.computing.LibvirtComputingResource] (agentRequest-Handler-5:null) Timed out waiting for domain v-2-VM to shutdown gracefully

libvir: QEMU error : Domain not found: no domain with matching uuid '4fb50e31-b032-3012-b70b-7e27c2acdfe2'

libvir: QEMU error : Domain not found: no domain with matching uuid '1422832d-be18-352a-a08a-9bbff40e0d14'

2013-07-21 08:32:13,089{GMT} WARN [resource.computing.LibvirtComputingResource] (agentRequest-Handler-1:) Timed out waiting for domain s-6-VM to shutdown gracefully

2013-07-21 08:32:13,089 WARN [resource.computing.LibvirtComputingResource] (agentRequest-Handler-1:null) Timed out waiting for domain s-6-VM to shutdown gracefully

2013-07-21 08:34:17,217{GMT} WARN [resource.computing.LibvirtComputingResource] (agentRequest-Handler-2:) Timed out waiting for domain v-2-VM to shutdown gracefully

2013-07-21 08:34:17,217 WARN [resource.computing.LibvirtComputingResource] (agentRequest-Handler-2:null) Timed out waiting for domain v-2-VM to shutdown gracefully

libvir: QEMU error : Domain not found: no domain with matching uuid '4fb50e31-b032-3012-b70b-7e27c2acdfe2'

libvir: QEMU error : Domain not found: no domain with matching uuid '1422832d-be18-352a-a08a-9bbff40e0d14'

2013-07-21 08:52:47,247{GMT} WARN [resource.computing.LibvirtComputingResource] (agentRequest-Handler-2:) Timed out waiting for domain s-6-VM to shutdown gracefully

2013-07-21 08:52:47,247 WARN [resource.computing.LibvirtComputingResource] (agentRequest-Handler-2:null) Timed out waiting for domain s-6-VM to shutdown gracefully

【management日志：】

2013-07-21 08:54:51,242 DEBUG [cloud.network.NetworkManagerImpl] (secstorage-1:null) Asking BareMetal to prepare for Nic[20-7-fc79f1d5-6066-4165-8d62-09ea025863a3-169.254.2.12]

2013-07-21 08:54:51,242 DEBUG [cloud.network.NetworkManagerImpl] (secstorage-1:null) Asking ElasticLoadBalancer to prepare for Nic[20-7-fc79f1d5-6066-4165-8d62-09ea025863a3-169.254.2.12]

2013-07-21 08:54:51,244 DEBUG [cloud.network.NetworkManagerImpl] (secstorage-1:null) Network id=201 is already implemented

2013-07-21 08:54:51,262 DEBUG [network.guru.PodBasedNetworkGuru] (secstorage-1:null) Allocated a nic NicProfile[21-7-fc79f1d5-6066-4165-8d62-09ea025863a3 for VM[SecondaryStorageVm|s-7-VM]

2013-07-21 08:54:51,266 DEBUG [cloud.network.NetworkManagerImpl] (secstorage-1:null) Asking ExternalFirewall to prepare for Nic[21-7-fc79f1d5-6066-4165-8d62-09ea025863a3-172.16.102.153]

2013-07-21 08:54:51,266 DEBUG [cloud.network.NetworkManagerImpl] (secstorage-1:null) Asking ExternalLoadBalancer to prepare for Nic[21-7-fc79f1d5-6066-4165-8d62-09ea025863a3-172.16.102.153]

2013-07-21 08:54:51,266 DEBUG [cloud.network.NetworkManagerImpl] (secstorage-1:null) Asking DomainRouter to prepare for Nic[21-7-fc79f1d5-6066-4165-8d62-09ea025863a3-172.16.102.153]

2013-07-21 08:54:51,267 DEBUG [cloud.network.NetworkManagerImpl] (secstorage-1:null) Asking Dhcp to prepare for Nic[21-7-fc79f1d5-6066-4165-8d62-09ea025863a3-172.16.102.153]

2013-07-21 08:54:51,267 DEBUG [cloud.network.NetworkManagerImpl] (secstorage-1:null) Asking Ovs to prepare for Nic[21-7-fc79f1d5-6066-4165-8d62-09ea025863a3-172.16.102.153]

2013-07-21 08:54:51,271 DEBUG [cloud.network.NetworkManagerImpl] (secstorage-1:null) Asking ExternalDhcp to prepare for Nic[21-7-fc79f1d5-6066-4165-8d62-09ea025863a3-172.16.102.153]

2013-07-21 08:54:51,271 DEBUG [cloud.network.NetworkManagerImpl] (secstorage-1:null) Asking BareMetal to prepare for Nic[21-7-fc79f1d5-6066-4165-8d62-09ea025863a3-172.16.102.153]

2013-07-21 08:54:51,271 DEBUG [cloud.network.NetworkManagerImpl] (secstorage-1:null) Asking ElasticLoadBalancer to prepare for Nic[21-7-fc79f1d5-6066-4165-8d62-09ea025863a3-172.16.102.153]

2013-07-21 08:54:51,273 DEBUG [cloud.storage.StorageManagerImpl] (secstorage-1:null) Checking if we need to prepare 1 volumes for VM[SecondaryStorageVm|s-7-VM]

2013-07-21 08:54:51,287 DEBUG [cloud.storage.StorageManagerImpl] (secstorage-1:null) Creating volume: Vol[7|vm=7|ROOT]

2013-07-21 08:54:51,287 DEBUG [cloud.storage.StorageManagerImpl] (secstorage-1:null) Trying to create in Pool[200|NetworkFilesystem]

2013-07-21 08:54:51,300 DEBUG [cloud.template.TemplateManagerImpl] (secstorage-1:null) Template 3 has already been downloaded to pool 200

2013-07-21 08:54:51,304 DEBUG [agent.transport.Request] (secstorage-1:null) Seq 1-139461193: Waiting for Seq 139461192 Scheduling: { Cmd , MgmtId: 345050611072, via: 1, Ver: v1, Flags: 100111, [{"storage.CreateCommand":{"volId":7,"pool":{"id":200,"uuid":"49b457c7-a241-3735-9104-ad92e417fc3d","host":"172.16.102.99","path":"/export/primary","port":2049,"type":"NetworkFilesystem"},"diskCharacteristics":{"size":0,"tags":[],"type":"ROOT","name":"ROOT-7","useLocalStorage":false,"recreatable":true,"diskOfferingId":6,"volumeId":7,"hyperType":"KVM"},"templateUrl":"/mnt/49b457c7-a241-3735-9104-ad92e417fc3d/5656e529-2c2d-4a6c-b480-ed34dae04e95","wait":0}}] }

2013-07-21 08:54:57,318 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:null) Found 0 routers.

2013-07-21 08:55:12,527 DEBUG [cloud.server.StatsCollector] (StatsCollector-2:null) VmStatsCollector is running...

2013-07-21 08:55:27,311 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:null) Found 0 routers.

2013-07-21 08:55:38,031 DEBUG [cloud.server.StatsCollector] (StatsCollector-2:null) HostStatsCollector is running...

2013-07-21 08:55:38,850 DEBUG [cloud.vm.VirtualMachineManagerImpl] (AgentManager-Handler-2:null) Cleanup succeeded. Details null

2013-07-21 08:55:38,850 DEBUG [agent.transport.Request] (StatsCollector-2:null) Seq 1-139461194: Received: { Ans: , MgmtId: 345050611072, via: 1, Ver: v1, Flags: 10, { GetHostStatsAnswer } }

2013-07-21 08:55:38,850 DEBUG [cloud.vm.VirtualMachineManagerImpl] (StatsCollector-2:null) Cleanup succeeded. Details null

2013-07-21 08:55:39,654 DEBUG [cloud.server.StatsCollector] (StatsCollector-2:null) StorageCollector is running...

2013-07-21 08:55:39,658 DEBUG [cloud.server.StatsCollector] (StatsCollector-2:null) There is no secondary storage VM for secondary storage host nfs://172.16.102.99/export/secondary

2013-07-21 08:55:39,744 DEBUG [cloud.vm.VirtualMachineManagerImpl] (AgentManager-Handler-3:null) Cleanup succeeded. Details null

2013-07-21 08:55:39,745 DEBUG [agent.transport.Request] (StatsCollector-2:null) Seq 1-139461195: Received: { Ans: , MgmtId: 345050611072, via: 1, Ver: v1, Flags: 10, { GetStorageStatsAnswer } }

2013-07-21 08:55:39,745 DEBUG [cloud.vm.VirtualMachineManagerImpl] (StatsCollector-2:null) Cleanup succeeded. Details null

2013-07-21 08:55:50,382 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-4:null) Ping from 1

2013-07-21 08:55:57,311 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:null) Found 0 routers.

【libvirt日志：】

08:09:12.840: 25527: error : virNetSocketReadWire:911 : End of file while reading data: Input/output error

08:13:53.948: 25532: warning : qemuDomainObjTaint:1122 : Domain id=22 name='s-6-VM' uuid=4fb50e31-b032-3012-b70b-7e27c2acdfe2 is tainted: high-privileges

08:22:07.369: 25528: warning : qemuDomainObjTaint:1122 : Domain id=23 name='v-2-VM' uuid=1422832d-be18-352a-a08a-9bbff40e0d14 is tainted: high-privileges

08:34:28.550: 25532: warning : qemuDomainObjTaint:1122 : Domain id=24 name='s-6-VM' uuid=4fb50e31-b032-3012-b70b-7e27c2acdfe2 is tainted: high-privileges

08:42:41.817: 25530: warning : qemuDomainObjTaint:1122 : Domain id=25 name='v-2-VM' uuid=1422832d-be18-352a-a08a-9bbff40e0d14 is tainted: high-privileges

08:55:02.188: 25531: warning : qemuDomainObjTaint:1122 : Domain id=26 name='v-2-VM' uuid=1422832d-be18-352a-a08a-9bbff40e0d14 is tainted: high-privileges

【虚拟机日志：】

2013-07-21 08:42:41.816: starting up

LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -S -M rhel6.2.0 -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -name v-2-VM -uuid 1422832d-be18-352a-a08a-9bbff40e0d14 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/v-2-VM.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -drive file=/mnt/49b457c7-a241-3735-9104-ad92e417fc3d/2efdefbc-47c4-4dc6-b35c-269874fd2b7e,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=2 -drive file=/mnt/49b457c7-a241-3735-9104-ad92e417fc3d/7190189b-e867-4bd2-829d-95da3cb7e70e,if=none,id=drive-virtio-disk1,format=raw,cache=none -device virtio-blk-pci,bus=pci.0,addr=0x7,drive=drive-virtio-disk1,id=virtio-disk1 -drive file=/usr/lib64/cloud/agent/vms/systemvm.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw,cache=none -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1 -netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=26 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=0e:00:a9:fe:01:5e,bus=pci.0,addr=0x3 -netdev tap,fd=27,id=hostnet1,vhost=on,vhostfd=28 -device virtio-net-pci,netdev=hostnet1,id=net1,mac=06:d3:1c:00:00:02,bus=pci.0,addr=0x4 -netdev tap,fd=29,id=hostnet2,vhost=on,vhostfd=30 -device virtio-net-pci,netdev=hostnet2,id=net2,mac=06:72:4c:00:00:07,bus=pci.0,addr=0x5 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -device usb-tablet,id=input0 -vnc 0.0.0.0:1 -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8

Domain id=25 is tainted: high-privileges

char device redirected to /dev/pts/2

qemu: terminating on signal 15 from pid 25527

2013-07-21 08:54:50.763: shutting down

2013-07-21 08:55:02.187: starting up

LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin QEMU_AUDIO_DRV=none /usr/libexec/qemu-kvm -S -M rhel6.2.0 -enable-kvm -m 1024 -smp 1,sockets=1,cores=1,threads=1 -name v-2-VM -uuid 1422832d-be18-352a-a08a-9bbff40e0d14 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/v-2-VM.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -drive file=/mnt/49b457c7-a241-3735-9104-ad92e417fc3d/2efdefbc-47c4-4dc6-b35c-269874fd2b7e,if=none,id=drive-virtio-disk0,format=qcow2,cache=none -device virtio-blk-pci,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=2 -drive file=/mnt/49b457c7-a241-3735-9104-ad92e417fc3d/a687cddc-f76c-4fb5-ba22-9fbc55761c88,if=none,id=drive-virtio-disk1,format=raw,cache=none -device virtio-blk-pci,bus=pci.0,addr=0x7,drive=drive-virtio-disk1,id=virtio-disk1 -drive file=/usr/lib64/cloud/agent/vms/systemvm.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw,cache=none -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0,bootindex=1 -netdev tap,fd=24,id=hostnet0,vhost=on,vhostfd=25 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=0e:00:a9:fe:03:5c,bus=pci.0,addr=0x3 -netdev tap,fd=26,id=hostnet1,vhost=on,vhostfd=27 -device virtio-net-pci,netdev=hostnet1,id=net1,mac=06:da:44:00:00:09,bus=pci.0,addr=0x4 -netdev tap,fd=28,id=hostnet2,vhost=on,vhostfd=29 -device virtio-net-pci,netdev=hostnet2,id=net2,mac=06:72:4c:00:00:07,bus=pci.0,addr=0x5 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -device usb-tablet,id=input0 -vnc 0.0.0.0:0 -vga cirrus -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8

Domain id=26 is tainted: high-privileges

char device redirected to /dev/pts/1