Xen vm crashes.

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Xen vm crashes.

Carlos Pinochet
Hello Everyone,

I'm using SLES 11 SP3 servers and upgraded it to SP4.
On it i had a vm that is crashing randomly after the upgrade.

I've installed "newer" xen rpm files on host and vm, version 4.4.4_14-51.1
From a Patch we found that suposingly prevent those crashes

On Xend.log i see these logs:

/var/log/xen/xend.log:[2017-08-04 13:00:01 6013] WARNING (XendDomainInfo:2194) Domain has crashed: name=ISS-PCI id=3.

On xm dmesg I get this kind of messages:

xm dmesg
 __  __            _  _   _  _   _  _     _ _  _     ____  _   _
 \ \/ /___ _ __   | || | | || | | || |   / | || |   | ___|/ | / |
  \  // _ \ '_ \  | || |_| || |_| || |_  | | || |_ _|___ \| | | |
  /  \  __/ | | | |__   _|__   _|__   _| | |__   _|__|__) | |_| |
 /_/\_\___|_| |_|    |_|(_) |_|(_) |_|___|_|  |_|   |____/|_(_)_|
                                    |_____|                     
(XEN) Xen version 4.4.4_14-51.1 (abuild@) (gcc (SUSE Linux) 4.3.4 [gcc-4_3-branch revision 152973]) debug=n Fri Feb 17 15:34:11 UTC 2017
(XEN) Latest ChangeSet:
(XEN) Bootloader: GNU GRUB 0.97
(XEN) Command line: vga=mode-0x31a dom0_mem=1024M
(XEN) Video information:
(XEN)  VGA is graphics mode 1280x1024, 16 bpp
(XEN)  VBE/DDC methods: none; EDID transfer time: 0 seconds
(XEN)  EDID info not retrieved because no DDC retrieval method detected
(XEN) Disc information:
(XEN)  Found 1 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 000000000009c000 (usable)
(XEN)  000000000009c000 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 000000007a289000 (usable)
(XEN)  000000007a289000 - 000000007af0b000 (reserved)
(XEN)  000000007af0b000 - 000000007b93b000 (ACPI NVS)
(XEN)  000000007b93b000 - 000000007bab8000 (ACPI data)
(XEN)  000000007bab8000 - 000000007bb00000 (usable)
(XEN)  000000007bb00000 - 0000000090000000 (reserved)
(XEN)  00000000feda8000 - 00000000fedac000 (reserved)
(XEN)  00000000ff310000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 0000000480000000 (usable)
(XEN) ACPI: RSDP 000FE320, 0024 (r2 DELL  )
(XEN) ACPI: XSDT 7BAB60E8, 00BC (r1 DELL   PE_SC3          0       1000013)
(XEN) ACPI: FACP 7BAB2000, 00F4 (r4 DELL   PE_SC3          0 DELL        1)
(XEN) ACPI: DSDT 7BA9C000, E3A2 (r2 DELL   PE_SC3          3 DELL        1)
(XEN) ACPI: FACS 7B91B000, 0040
(XEN) ACPI: MCEJ 7BAB5000, 0130 (r1 INTEL                  1 INTL  100000D)
(XEN) ACPI: WD__ 7BAB4000, 0134 (r1 DELL   PE_SC3          1 DELL        1)
(XEN) ACPI: SLIC 7BAB3000, 0024 (r1 DELL   PE_SC3          1 DELL        1)
(XEN) ACPI: HPET 7BAB1000, 0038 (r1 DELL   PE_SC3          1 DELL        1)
(XEN) ACPI: APIC 7BAB0000, 0AFC (r2 DELL   PE_SC3          0 DELL        1)
(XEN) ACPI: MCFG 7BAAF000, 003C (r1 DELL   PE_SC3          1 DELL        1)
(XEN) ACPI: MSCT 7BAAE000, 0090 (r1 DELL   PE_SC3          1 DELL        1)
(XEN) ACPI: SLIT 7BAAD000, 006C (r1 DELL   PE_SC3          1 DELL        1)
(XEN) ACPI: SRAT 7BAAB000, 1130 (r3 DELL   PE_SC3          1 DELL        1)
(XEN) ACPI: SSDT 7B959000, 1424A9 (r2 DELL   PE_SC3       4000 INTL 20121114)
(XEN) ACPI: SSDT 7B956000, 217F (r2 DELL   PE_SC3          2 INTL 20121114)
(XEN) ACPI: SSDT 7B955000, 006E (r2 DELL   PE_SC3          2 INTL 20121114)
(XEN) ACPI: PRAD 7B954000, 0132 (r2   DELL PE_SC3          2 INTL 20121114)
(XEN) ACPI: DMAR 7B953000, 00A0 (r1 DELL   PE_SC3          1 DELL        1)
(XEN) ACPI: HEST 7B952000, 017C (r1 DELL   PE_SC3          2 DELL        1)
(XEN) ACPI: BERT 7B951000, 0030 (r1 DELL   PE_SC3          2 DELL        1)
(XEN) ACPI: ERST 7B950000, 0230 (r1 DELL   PE_SC3          2 DELL        1)
(XEN) ACPI: EINJ 7B94F000, 0150 (r1 DELL   PE_SC3          2 DELL        1)
(XEN) System RAM: 16290MB (16681396kB)
(XEN) Domain heap initialised
(XEN) ACPI: 32/64X FACS address mismatch in FADT - 7b91b000/0000000000000000, using 32
(XEN) Processor #0 7:15 APIC version 21
(XEN) Processor #2 7:15 APIC version 21
(XEN) Processor #4 7:15 APIC version 21
(XEN) Processor #6 7:15 APIC version 21
(XEN) Processor #8 7:15 APIC version 21
(XEN) Processor #10 7:15 APIC version 21
(XEN) Processor #1 7:15 APIC version 21
(XEN) Processor #3 7:15 APIC version 21
(XEN) Processor #5 7:15 APIC version 21
(XEN) Processor #7 7:15 APIC version 21
(XEN) Processor #9 7:15 APIC version 21
(XEN) Processor #11 7:15 APIC version 21
(XEN) IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
(XEN) IOAPIC[1]: apic_id 9, version 32, address 0xfec01000, GSI 24-47
(XEN) Enabling APIC mode:  Phys.  Using 2 I/O APICs
(XEN) Not enabling x2APIC (upon firmware request)
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Platform timer is 14.318MHz HPET
(XEN) Detected 2397.232 MHz processor.
(XEN) Initing memory sharing.
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d Snoop Control enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Shared EPT tables enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN) Allocated console ring of 32 KiB.
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest
(XEN)  - APIC Register Virtualization
(XEN)  - Virtual Interrupt Delivery
(XEN)  - Posted Interrupt Processing
(XEN)  - VMCS shadowing
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) Brought up 12 CPUs
(XEN) Dom0 has maximum 816 PIRQs
(XEN) *** LOADING DOMAIN 0 ***
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, lsb, paddr 0x2000 -> 0x8af000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000471000000->0000000472000000 (253850 pages to be allocated)
(XEN)  Init. ramdisk: 000000047ef9a000->000000047ffff600
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff80002000->ffffffff808af000
(XEN)  Init. ramdisk: 0000000000000000->0000000000000000
(XEN)  Phys-Mach map: ffffea0000000000->ffffea0000200000
(XEN)  Start info:    ffffffff808af000->ffffffff808af4b4
(XEN)  Page tables:   ffffffff808b0000->ffffffff808b9000
(XEN)  Boot stack:    ffffffff808b9000->ffffffff808ba000
(XEN)  TOTAL:         ffffffff80000000->ffffffff80c00000
(XEN)  ENTRY ADDRESS: ffffffff80002000
(XEN) Dom0 has maximum 12 VCPUs
(XEN) Scrubbing Free RAM: ......................................................................................................................................................done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 292kB init memory.
(XEN) mm.c:803:d1 Non-privileged (1) attempt to map I/O space 000000f0
(XEN) CPU11: No irq handler for vector 56 (IRQ -2147483648, LAPIC)
(XEN) mm.c:803:d1 Non-privileged (1) attempt to map I/O space 000000f0
(XEN) mm.c:803:d1 Non-privileged (1) attempt to map I/O space 000000f0
(XEN) CPU5: No irq handler for vector 2d (IRQ -76, LAPIC)
(XEN) IRQ75 a=0080[0080,0000] v=94[ffffffff] t=PCI-MSI/-X s=00000010
(XEN) mm.c:803:d1 Non-privileged (1) attempt to map I/O space 000000f0
(XEN) mm.c:803:d1 Non-privileged (1) attempt to map I/O space 000000f0
(XEN) Pagetable walk from ffff88022ebfff08:
(XEN)  L4[0x110] = 000000044bbae067 000000000000339a
(XEN)  L3[0x008] = 000000043a3a5067 00000000000043a3
(XEN)  L2[0x175] = 000000043a22f067 0000000000004519
(XEN)  L1[0x1ff] = 80100003ceb49065 000000000022ebff
(XEN) domain_crash_sync called from entry.S: fault at ffff82d08022aa08 create_bounce_frame+0x63/0x134
(XEN) Domain 1 (vcpu#3) crashed on cpu#10:

(XEN) ----[ Xen-4.4.4_14-51.1  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    10
(XEN) RIP:    e033:[<ffffffff804180ac>]
(XEN) RFLAGS: 0000000000210246   EM: 1   CONTEXT: pv guest (d1v3)
(XEN) rax: 0000000000000000   rbx: 00000000f4534404   rcx: 00000000f423fa0c
(XEN) rdx: 0000000000000000   rsi: 0000000000000000   rdi: ffff88022ec00018
(XEN) rbp: 0000000000000000   rsp: ffff88022ebfff18   r8:  0000000000000000
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
(XEN) r12: 0000000000000000   r13: ffff88022ec001c8   r14: 0000000000000028
(XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4: 00000000001526f0
(XEN) cr3: 000000004fb48000   cr2: ffff88022ebfffe8
(XEN) ds: 002b   es: 002b   fs: fff7   gs: 0000   ss: e02b   cs: e033
(XEN) Guest stack trace from rsp=ffff88022ebfff18:
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    00000004473a3067
(XEN) mm.c:803:d2 Non-privileged (2) attempt to map I/O space 000000f0
(XEN) Pagetable walk from ffff8802275f6fd8:
(XEN)  L4[0x110] = 0000000436c2a067 000000000000339a
(XEN)  L3[0x008] = 0000000446421067 00000000000043a3
(XEN)  L2[0x13a] = 00000004462e6067 00000000000044de
(XEN)  L1[0x1f6] = 80100002161ce065 00000000002275f6
(XEN) domain_crash_sync called from entry.S: fault at ffff82d08022aa08 create_bounce_frame+0x63/0x134
(XEN) Domain 2 (vcpu#0) crashed on cpu#4:

(XEN) ----[ Xen-4.4.4_14-51.1  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    4
(XEN) RIP:    e033:[<ffffffff80418105>]
(XEN) RFLAGS: 0000000000210246   EM: 1   CONTEXT: pv guest (d2v0)
(XEN) rax: 000000010000e030   rbx: 00000000f7753000   rcx: 0000000000000000
(XEN) rdx: 0000000000000000   rsi: 0000000000000000   rdi: ffff8802275f70e8
(XEN) rbp: 0000000000000000   rsp: ffff8802275f6fe8   r8:  0000000000000000
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
(XEN) r12: 0000000000000000   r13: ffff8802275f70e8   r14: 0000000000000028
(XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4: 00000000001526f0
(XEN) cr3: 00000002162c5000   cr2: 0000000000005480
(XEN) ds: 002b   es: 002b   fs: fff7   gs: 0000   ss: e02b   cs: e033
(XEN) Guest stack trace from rsp=ffff8802275f6fe8:
(XEN)    0000000000000000 0000000000000000 0000000000000000 ffff8802208f5000
(XEN)    ffff88022090f000 00000000000000e0 ffff8802275f70e0 0000000000000019
(XEN)    0000000000000000 0000000200000001 0000000400000003 0000000600000005
(XEN)    0000000800000007 0000000a00000009 0000000c0000000b 0000000e0000000d
(XEN)    000000100000000f 0000001200000011 00000014ffffffff 0000001600000015
(XEN)    0000001800000017 0000001a00000019 0000001c0000001b 0000001e0000001d
(XEN)    000000200000001f 00000000f7753000 0000000000000000 0000000000000000
(XEN)    ffff8802275f7298 0000000000000028 0000000000000000 ffffffff80414ae8
(XEN)    0000000000000000 0000000000000028 ffff8802275f7298 0000000000000000
(XEN)    0000000000000000 00000000f7753000 0000000000000000 0000000000000000
(XEN)    0000000000000000 0000000000000000 000000010000e030 0000000000000000
(XEN)    0000000000000000 0000000000000000 ffff8802275f7298 ffffffffffffffff
(XEN)    ffffffff80418105 000000010000e030 0000000000210046 ffff8802275f7198
(XEN)    000000000000e02b ffff88030c15e3f8 0000000000000001 0000000000000000
(XEN)    ffff8802275f71a8 0000000000000000 0000000000000000 0000000000000000
(XEN)    ffff8802275f71c8 ffff8802275f71c8 0000000000000000 ffff88030c15e3f8
(XEN)    ffff88030c15e5b0 0000000000000001 0000000000000000 ffff8802275f7200
(XEN)    0000000000000000 0000000000000000 0000000000000000 ffff8802275f7220
(XEN)    ffff8802275f7220 0000000000000000 ffff88030c15e5b0 ffff88030c15e5b0
(XEN)    0000000000000001 0000000000000000 ffff8802275f7258 00000000f7753000
(XEN) mm.c:803:d3 Non-privileged (3) attempt to map I/O space 000000f0
(XEN) CPU6: No irq handler for vector 5e (IRQ -68, LAPIC)
(XEN) IRQ67 a=0008[0008,0000] v=d2[ffffffff] t=PCI-MSI/-X s=00000030
(XEN) CPU4: No irq handler for vector dc (IRQ -76, LAPIC)
(XEN) IRQ75 a=0400[0400,0000] v=6d[ffffffff] t=PCI-MSI/-X s=00000010


Please could let me know if you have seen this before? Is there any fix?

Any information will be helpfull.

Thanks,

-- 
Carlos Pinochet

Devops Engineer
(o) 22079698 
(c) +56 9 65876146
(e) [hidden email]

nisum​
consulting | digital | innovation
www.nisum.com

This e-mail and any attachments may contain confidential and privileged information, and is only meant for the intended recipient. Nisum and its logos are registered trademarks of Nisum Technologies, Inc. All rights reserved.

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users