Problem with Intel(R) Xeon(R) CPU L5640?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Problem with Intel(R) Xeon(R) CPU L5640?

admin-2
Hi,

I have two servers of the same model, HP SE316M1.

The first one, running Xen 4.4, which was installed rather long time ago.
Our backup server died (older Dell server) and was replaced by another
HP server of the same model (SE316M1).

Unfortunately I have no chance of running Xen on the new server.
I started with 4.9 and consecutively tried 4.8, 4.7 and 4.6 - always having
Xen kernel crashing shortly after boot.

I then tried a self compiled Xen v4.9 with the same result.

Both server are equipped with 48 gigs of RAM and share the same config, despite of:

  - srv1 (xen running) : HP Smart Array P800 RAID, 2 x Intel(R) Xeon(R) CPU L5520 @2.27GHz Quad Core
  - srv2 (xen craching): HP Smart Array P410 RAID, 2 x Intel(R) Xeon(R) CPU L5640 @2.27GHz Hexa Core

Also tried every combination of noapic, nolapic, acpi=off etc...

I then decided to install Xen v4.4 on the new server first in order to migrate my VMs becuase
the old server needs some hardware maintainance. But Xen v4.4 did crash as well!

Is this related to the CPU?

I also had a look into the pci.c but did not really understand what's happening there exactly.
So I did not try to catch and ignore the problem, as the server is supposed to run the VMs in a 
productive envirnment.

The following kernel log (xen 4.9) was grabbed via serial console - any ideas?:



(XEN) Xen version 4.9.0 (root@) (gcc (Debian 6.3.0-18) 6.3.0 20170516) debug=n  Sat Oct 14 15:05:58 CEST 2017
(XEN) Latest ChangeSet: Tue Jun 27 19:13:19 2017 +0100 git:c30bf55594
(XEN) Bootloader: GRUB 2.02~beta3-5
(XEN) Command line: placeholder dom0_mem=1024M loglvl=all com1=115200,8n1 console=com1 dom0_mem=1024M loglvl=all com1=115200,8n1 console=com1
(XEN) Xen image load base address: 0
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 2 seconds
(XEN) Disc information:
(XEN)  Found 1 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 0000000000097400 (usable)
(XEN)  0000000000097400 - 00000000000a0000 (reserved)
(XEN)  00000000000f0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 00000000e7622000 (usable)
(XEN)  00000000e7622000 - 00000000e762c000 (ACPI data)
(XEN)  00000000e762c000 - 00000000e762d000 (usable)
(XEN)  00000000e762d000 - 00000000ec000000 (reserved)
(XEN)  00000000fec00000 - 00000000fee10000 (reserved)
(XEN)  00000000ff800000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 0000000c17fff000 (usable)
(XEN) New Xen image base address: 0xe7000000
(XEN) ACPI: RSDP 000F4F00, 0024 (r2 HP    )
(XEN) ACPI: XSDT E7623540, 00AC (r1 HP     ProLiant        2   �     162E)
(XEN) ACPI: FACP E7623640, 00F4 (r3 HP     ProLiant        2   �     162E)
(XEN) ACPI: DSDT E7623740, 20AD (r1 HP         DSDT        1 INTL 20030228)
(XEN) ACPI: FACS E7622100, 0040
(XEN) ACPI: SPCR E7622140, 0050 (r1 HP     SPCRRBSU        1   �     162E)
(XEN) ACPI: MCFG E76221C0, 003C (r1 HP     ProLiant        1             0)
(XEN) ACPI: HPET E7622200, 0038 (r1 HP     ProLiant        2   �     162E)
(XEN) ACPI: FFFF E7622240, 0064 (r2 HP     ProLiant        2   �     162E)
(XEN) ACPI: SPMI E76222C0, 0040 (r5 HP     ProLiant        1   �     162E)
(XEN) ACPI: ERST E7622300, 01D0 (r1 HP     ProLiant        1   �     162E)
(XEN) ACPI: APIC E7622500, 015E (r1 HP     ProLiant        2             0)
(XEN) ACPI: SRAT E7622680, 0570 (r1 HP     Proliant        1   �     162E)
(XEN) ACPI: FFFF E7622C00, 0176 (r1 HP     ProLiant        1   �     162E)
(XEN) ACPI: BERT E7622D80, 0030 (r1 HP     ProLiant        1   �     162E)
(XEN) ACPI: HEST E7622DC0, 00BC (r1 HP     ProLiant        1   �     162E)
(XEN) ACPI: DMAR E7622E80, 016C (r1 HP     ProLiant        1   �     162E)
(XEN) ACPI: SSDT E7625800, 0125 (r3     HP  CRSPCI0        2   HP        1)
(XEN) ACPI: SSDT E7625940, 0211 (r3     HP  riser2a        2 INTL 20061109)
(XEN) ACPI: SSDT E7625B80, 0377 (r1     HP     pmab        1 INTL 20090625)
(XEN) ACPI: SSDT E7625F00, 636C (r1  INTEL PPM RCM         1 INTL 20061109)
(XEN) System RAM: 49141MB (50321124kB)
(XEN) SRAT: PXM 0 -> APIC 00 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 01 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 02 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 03 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 04 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 05 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 10 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 11 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 12 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 13 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 14 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 15 -> Node 0
(XEN) SRAT: PXM 1 -> APIC 20 -> Node 1
(XEN) SRAT: PXM 1 -> APIC 21 -> Node 1
(XEN) SRAT: PXM 1 -> APIC 22 -> Node 1
(XEN) SRAT: PXM 1 -> APIC 23 -> Node 1
(XEN) SRAT: PXM 1 -> APIC 24 -> Node 1
(XEN) SRAT: PXM 1 -> APIC 25 -> Node 1
(XEN) SRAT: PXM 1 -> APIC 30 -> Node 1
(XEN) SRAT: PXM 1 -> APIC 31 -> Node 1
(XEN) SRAT: PXM 1 -> APIC 32 -> Node 1
(XEN) SRAT: PXM 1 -> APIC 33 -> Node 1
(XEN) SRAT: PXM 1 -> APIC 34 -> Node 1
(XEN) SRAT: PXM 1 -> APIC 35 -> Node 1
(XEN) SRAT: Node 0 PXM 0 0-e8000000
(XEN) SRAT: Node 0 PXM 0 100000000-618000000
(XEN) SRAT: Node 1 PXM 1 618000000-c18000000
(XEN) NUMA: Allocated memnodemap from c17ef9000 - c17efa000
(XEN) NUMA: Using 15 for the hash shift.
(XEN) Domain heap initialised DMA width 32 bits
(XEN) CPU Vendor: Intel, Family 6 (0x6), Model 44 (0x2c), Stepping 2 (raw 000206c2)
(XEN) found SMP MP-table at 000f4f80
(XEN) DMI 2.7 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x908 (32 bits)
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:904,1:0], pm1x_evt[1:900,1:0]
(XEN) ACPI:             wakeup_vec[e762210c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x10] lapic_id[0x20] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x10] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x18] lapic_id[0x30] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x04] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x14] lapic_id[0x24] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x14] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x1c] lapic_id[0x34] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x12] lapic_id[0x22] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x12] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x1a] lapic_id[0x32] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x06] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x16] lapic_id[0x26] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x16] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x1e] lapic_id[0x36] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x11] lapic_id[0x21] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x09] lapic_id[0x11] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x19] lapic_id[0x31] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x05] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x15] lapic_id[0x25] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x15] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x1d] lapic_id[0x35] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x13] lapic_id[0x23] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x13] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x1b] lapic_id[0x33] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x07] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x17] lapic_id[0x27] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x17] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x1f] lapic_id[0x37] disabled)
(XEN) ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
(XEN) Overriding APIC driver with bigsmp
(XEN) ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: IOAPIC (id[0x00] address[0xfec80000] gsi_base[24])
(XEN) IOAPIC[1]: apic_id 0, version 32, address 0xfec80000, GSI 24-47
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Phys.  Using 2 I/O APICs
(XEN) ACPI: HPET id: 0x8086a201 base: 0xfed00000
(XEN) Xen BUG at pci.c:27
(XEN) ----[ Xen-4.9.0  x86_64  debug=n   Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    e008:[<ffff82d080345ce0>] pci_conf_read8+0x90/0xa0
(XEN) RFLAGS: 0000000000010002   CONTEXT: hypervisor
(XEN) rax: 0000000000000019   rbx: ffff82c000219f8e   rcx: 0000000000000010
(XEN) rdx: 0000000000000002   rsi: 0000000000000014   rdi: 0000000000000001
(XEN) rbp: 0000000000000000   rsp: ffff82d08041fb78   r8:  0000000000000001
(XEN) r9:  0000000000000014   r10: 0000000000000000   r11: 0000000000000000
(XEN) r12: ffff82c000219f84   r13: ffff82c000219f92   r14: ffff830617ff5910
(XEN) r15: ffff82c000219f8a   cr0: 0000000080050033   cr4: 00000000000000a0
(XEN) cr3: 00000000e7410000   cr2: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
(XEN) Xen code around <ffff82d080345ce0> (pci_conf_read8+0x90/0xa0):
(XEN)  24 0c 48 83 c4 18 c3 90 <0f> 0b 0f 1f 40 00 66 2e 0f 1f 84 00 00 00 00 00
(XEN) Xen stack trace from rsp=ffff82d08041fb78:
(XEN)    0000000000000048 0000000000000010 0000000000000008 ffff82d08039d30c
(XEN)    0000000000000003 ffff82c000219fec 0000000000000000 0000000000000058
(XEN)    0000000000000068 ffff82c000009e80 0000000000000000 ffff82c000219f7c
(XEN)    ffff82d0803fb270 ffff82c000219e80 0000000000000000 ffff830617ff5910
(XEN)    00000000e763ffff ffff82d08039e009 0000000c17e00000 ffff830617ff5880
(XEN)    0000000180411800 00000000000e7622 ffff8300e762c1c0 ffff82d08037ef41
(XEN)    ffff82d08039d970 ffff82d08039d970 0000000c17fff000 0000000c17e00000
(XEN)    0000000c17e00000 ffff82d0803a4713 ffff8300e7622e80 ffff82c000219e80
(XEN)    0000000000000001 ffff830000000000 0000000000000001 ffff82d0803b37ff
(XEN)    ffff8300e7622e80 ffff8300e7622e80 00000000000e7622 0000000000000001
(XEN)    ffff8300e740e018 ffff82d08027c748 0000000000000000 0000000000000022
(XEN)    000000018037eaaf 8000000000000163 00000000000001c0 80000000000001e3
(XEN)    0000000c17fff000 008001e300000000 8000000000000000 00000000000001e3
(XEN)    0000000c17fff000 00000000000e7622 ffff8300e762c1c0 ffff830000000000
(XEN)    0000000000000001 ffff82d080384000 ffff830000000000 0000000000000001
(XEN)    ffff82d080384000 ffff82d08039e487 00000000e7622e80 000000000000016c
(XEN)    ffff82d0803b4250 0000000000000000 0000000000000001 ffff82d080384000
(XEN)    0000000c17fff000 0000000c17e00000 0000000c17e00000 ffff82d0803b4c9f
(XEN)    0000000000000009 0000000000bff5b9 000000004fffffff ffff82d0803b0c10
(XEN)    0000000000000000 ffff830000095fb0 ffff830000095f10 ffff8300e7411001
(XEN) Xen call trace:
(XEN)    [<ffff82d080345ce0>] pci_conf_read8+0x90/0xa0
(XEN)    [<ffff82d08039d30c>] dmar.c#acpi_parse_dev_scope+0x10c/0x500
(XEN)    [<ffff82d08039e009>] dmar.c#acpi_parse_dmar+0x699/0xa40
(XEN)    [<ffff82d08039d970>] dmar.c#acpi_parse_dmar+0/0xa40
(XEN)    [<ffff82d08039d970>] dmar.c#acpi_parse_dmar+0/0xa40
(XEN)    [<ffff82d0803a4713>] acpi_table_parse+0x53/0x90
(XEN)    [<ffff82d0803b37ff>] tboot_parse_dmar_table+0x10f/0x120
(XEN)    [<ffff82d08027c748>] map_pages_to_xen+0x3d8/0x10e0
(XEN)    [<ffff82d080384000>] __2M_init_start+0/0x8
(XEN)    [<ffff82d080384000>] __2M_init_start+0/0x8
(XEN)    [<ffff82d08039e487>] acpi_dmar_init+0xd7/0x3a0
(XEN)    [<ffff82d0803b4250>] boot.c#acpi_parse_hpet+0/0x80
(XEN)    [<ffff82d080384000>] __2M_init_start+0/0x8
(XEN)    [<ffff82d0803b4c9f>] acpi_boot_init+0x5f/0x320
(XEN)    [<ffff82d0803b0c10>] __start_xen+0x2540/0x2e40
(XEN)    [<ffff82d0802000f3>] __high_start+0x53/0x60
(XEN) 
(XEN) 
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Xen BUG at pci.c:27
(XEN) ****************************************
(XEN) 
(XEN) Reboot in five seconds...

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Intel(R) Xeon(R) CPU L5640?

Sarah Newman
On 10/18/2017 05:09 PM, admin wrote:

> Hi,
>
> I have two servers of the same model, HP SE316M1.
>
> The first one, running Xen 4.4, which was installed rather long time ago.
> Our backup server died (older Dell server) and was replaced by another
> HP server of the same model (SE316M1).
>
> Unfortunately I have no chance of running Xen on the new server.
> I started with 4.9 and consecutively tried 4.8, 4.7 and 4.6 - always having
> Xen kernel crashing shortly after boot.
>
> I then tried a self compiled Xen v4.9 with the same result.
>
> Both server are equipped with 48 gigs of RAM and share the same config, despite of:
>
>   - srv1 (xen running) : HP Smart Array P800 RAID, 2 x Intel(R) Xeon(R) CPU L5520 @2.27GHz Quad Core
>   - srv2 (xen craching): HP Smart Array P410 RAID, 2 x Intel(R) Xeon(R) CPU L5640 @2.27GHz Hexa Core
>
> Also tried every combination of noapic, nolapic, acpi=off etc...
>
> I then decided to install Xen v4.4 on the new server first in order to migrate my VMs becuase
> the old server needs some hardware maintainance. But Xen v4.4 did crash as well!
>
> Is this related to the CPU?

It has nothing to do with the CPU model.

What I would do for unknown hardware that's crashing, in no particular order:

Review the kernel logs when booted without Xen for warnings
Check the BIOS and IPMI logs
Check for BIOS or device firmware updates
Review BIOS settings related to ACPI or PCI
Run diagnostic utilities for the the various PCI devices if applicable
Run a memory test in multi-core mode
Remove or replace individual components to see if a particular component is causing problems

--Sarah

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: Problem with Intel(R) Xeon(R) CPU L5640?

admin-2
Am Donnerstag, den 19.10.2017, 09:02 -0700 schrieb Sarah Newman:

> On 10/18/2017 05:09 PM, admin wrote:
> > Hi,
> >
> > I have two servers of the same model, HP SE316M1.
> >
> > The first one, running Xen 4.4, which was installed rather long
> > time ago.
> > Our backup server died (older Dell server) and was replaced by
> > another
> > HP server of the same model (SE316M1).
> >
> > Unfortunately I have no chance of running Xen on the new server. 
> > I started with 4.9 and consecutively tried 4.8, 4.7 and 4.6 -
> > always having
> > Xen kernel crashing shortly after boot.
> >
> > I then tried a self compiled Xen v4.9 with the same result.
> >
> > Both server are equipped with 48 gigs of RAM and share the same
> > config, despite of:
> >
> >   - srv1 (xen running) : HP Smart Array P800 RAID, 2 x Intel(R)
> > Xeon(R) CPU L5520 @2.27GHz Quad Core
> >   - srv2 (xen craching): HP Smart Array P410 RAID, 2 x Intel(R)
> > Xeon(R) CPU L5640 @2.27GHz Hexa Core
> >
> > Also tried every combination of noapic, nolapic, acpi=off etc...
> >
> > I then decided to install Xen v4.4 on the new server first in order
> > to migrate my VMs becuase
> > the old server needs some hardware maintainance. But Xen v4.4 did
> > crash as well!
> >
> > Is this related to the CPU?
>
> It has nothing to do with the CPU model.
>
> What I would do for unknown hardware that's crashing, in no
> particular order:
>
> Review the kernel logs when booted without Xen for warnings
> Check the BIOS and IPMI logs
Will do that

> Check for BIOS or device firmware updates
All up to date on both machines

> Review BIOS settings related to ACPI or PCI
Will check on that. My first idea was to compare all BIOS seetings
of both machines and make them match. But the running one is an
important productive machine - down time means $0 time...

> Run diagnostic utilities for the the various PCI devices if
> applicable
> Run a memory test in multi-core mode
> Remove or replace individual components to see if a particular
> component is causing problems

Thanks for your thoughts on this. I'll be back as soon as possible
or necessary.

>
> --Sarah
>
-- Manuel

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users