PVH dom0 creation fails - the system freezes

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

PVH dom0 creation fails - the system freezes

Bercaru, Gabriel
Hello,


For the last few days, I have been trying to get a PVH dom0 running,
however I encountered the following problem: the system seems to
freeze after the hypervisor boots, the screen goes black. I have tried to
debug it via a serial console (using Minicom) and managed to get some
more Xen output, after the screen turns black.
 
I mention that I have tried to boot the PVH dom0 using different kernel
images (from 4.9.0 to 4.18-rc3), different Xen  versions (4.10, 4.11, 4.12).

Below I attached my system / hypervisor configuration, as well as the
output captured through the serial console, corresponding to the latest
versions for Xen and the Linux Kernel (Xen staging and Kernel from the
xen/tip tree).


OS + Distro: Linux / Debian 9 Stretch
Kernel Version: 4.17-rc5, tagged with for-linus-4.18-rc5-tag from the
xen/tip tree.
Xen Version: 4.12, commit id e3f667bc5f51d0aa44357a64ca134cd952679c81
of the Xen tree.
Host system: attached cpuinfo.log
Serial console output: attached boot.log
My grub configuration file, containing the Xen command line arguments: attached
grub.log

I can provide additional info as requested.
Any ideas why this happens? Do you have any recommendations for additional
debugging?

Here are the last few lines of the boot log. The last (separated) ones were only
visible though the serial console, since at that point the screen was completely
black.
 
(XEN) *** Building a PVH Dom0 ***
(XEN) [VT-D]d0:Hostbridge: skip 0000:00:00.0 map
(XEN) [VT-D]d0:PCI: map 0000:00:14.0
(XEN) [VT-D]d0:PCI: map 0000:00:14.2
(XEN) [VT-D]d0:PCI: map 0000:00:16.0
(XEN) [VT-D]d0:PCI: map 0000:00:16.1
(XEN) [VT-D]d0:PCI: map 0000:00:17.0
(XEN) [VT-D]d0:PCI: map 0000:00:1f.0
(XEN) [VT-D]d0:PCI: map 0000:00:1f.2
(XEN) [VT-D]d0:PCI: map 0000:00:1f.4
(XEN) [VT-D]d0:PCIe: map 0000:01:00.0
(XEN) [VT-D]d0:PCIe: map 0000:02:00.0
(XEN) [VT-D]d0:PCIe: map 0000:03:00.0
(XEN) [VT-D]d0:PCIe: map 0000:04:00.0
(XEN) [VT-D]iommu_enable_translation: iommu->reg = ffff82c00021b000
(XEN) WARNING: PVH is an experimental mode with limited functionality
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Scrubbing Free RAM on 1 nodes using 4 CPUs
(XEN) ...................................................................................................................................done.
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) ***************************************************
(XEN) WARNING: CONSOLE OUTPUT IS SYNCHRONOUS
(XEN) This option is intended to aid debugging of Xen by ensuring
(XEN) that all output is synchronously delivered on the serial line.
(XEN) However it can introduce SIGNIFICANT latencies and affect
(XEN) timekeeping. It is NOT recommended for production use!
(XEN) ***************************************************
(XEN) 3... 2... 1...

(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 468kB init memory
(XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow
(XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault
(XEN) [VT-D]DMAR:[DMA Write] Request device [0000:00:14.0] fault addr 8deb3000, iommu reg = ffff82c00021b000
(XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
(XEN) print_vtd_entries: iommu #0 dev 0000:00:14.0 gmfn 8deb3
(XEN) root_entry[00] = 1021c60001
(XEN) context[a0] = 2_1021d6d001
(XEN) l4[000] = 9c00001021d6c107
(XEN) l3[002] = 9c00001021d3e107
(XEN) l2[06f] = 9c000010218c0107
(XEN) l1[0b3] = 8000000000000000
(XEN) l1[0b3] not present
(XEN) Dom0 callback via changed to Direct Vector 0xf3

Thanks,
Gabriel





Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users

boot.log (16K) Download Attachment
cpuinfo.log (13K) Download Attachment
grub.log (1K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: PVH dom0 creation fails - the system freezes

Juergen Gross-3
On 23/07/18 09:52, Bercaru, Gabriel wrote:

> Hello,
>
>
> For the last few days, I have been trying to get a PVH dom0 running,
> however I encountered the following problem: the system seems to
> freeze after the hypervisor boots, the screen goes black. I have tried to
> debug it via a serial console (using Minicom) and managed to get some
> more Xen output, after the screen turns black.
>  
> I mention that I have tried to boot the PVH dom0 using different kernel
> images (from 4.9.0 to 4.18-rc3), different Xen  versions (4.10, 4.11, 4.12).
>
> Below I attached my system / hypervisor configuration, as well as the
> output captured through the serial console, corresponding to the latest
> versions for Xen and the Linux Kernel (Xen staging and Kernel from the
> xen/tip tree).
>
>
> OS + Distro: Linux / Debian 9 Stretch
> Kernel Version: 4.17-rc5, tagged with for-linus-4.18-rc5-tag from the
> xen/tip tree.
> Xen Version: 4.12, commit id e3f667bc5f51d0aa44357a64ca134cd952679c81
> of the Xen tree.
> Host system: attached cpuinfo.log
> Serial console output: attached boot.log
> My grub configuration file, containing the Xen command line arguments: attached
> grub.log
>
> I can provide additional info as requested.
> Any ideas why this happens? Do you have any recommendations for additional
> debugging?

Add "earlyprintk=xen" to the dom0 kernel parameters.

Any reason why you are trying to use an iommu? I'd start with the simple
case without iommu.


Juergen

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: PVH dom0 creation fails - the system freezes

Roger Pau Monné-3
In reply to this post by Bercaru, Gabriel
On Mon, Jul 23, 2018 at 07:52:53AM +0000, Bercaru, Gabriel wrote:
> Hello,
>
>
> For the last few days, I have been trying to get a PVH dom0 running,
> however I encountered the following problem: the system seems to
> freeze after the hypervisor boots, the screen goes black. I have tried to
> debug it via a serial console (using Minicom) and managed to get some
> more Xen output, after the screen turns black.

I don't think it freezes, it's just extremely slow due to MTRR being
disabled and all memory treated as UC. If you left it overnight you
will probably manage to get to the Linux login prompt.

> I mention that I have tried to boot the PVH dom0 using different kernel
> images (from 4.9.0 to 4.18-rc3), different Xen  versions (4.10, 4.11, 4.12).
>
> Below I attached my system / hypervisor configuration, as well as the
> output captured through the serial console, corresponding to the latest
> versions for Xen and the Linux Kernel (Xen staging and Kernel from the
> xen/tip tree).
>
>
> OS + Distro: Linux / Debian 9 Stretch
> Kernel Version: 4.17-rc5, tagged with for-linus-4.18-rc5-tag from the
> xen/tip tree.
> Xen Version: 4.12, commit id e3f667bc5f51d0aa44357a64ca134cd952679c81
> of the Xen tree.

You are missing the MTRR series. Can you try to build Xen from the
staging branch instead of master?

Roger.

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: PVH dom0 creation fails - the system freezes

Roger Pau Monné-3
In reply to this post by Juergen Gross-3
On Mon, Jul 23, 2018 at 02:16:59PM +0200, Juergen Gross wrote:

> On 23/07/18 09:52, Bercaru, Gabriel wrote:
> > Hello,
> >
> >
> > For the last few days, I have been trying to get a PVH dom0 running,
> > however I encountered the following problem: the system seems to
> > freeze after the hypervisor boots, the screen goes black. I have tried to
> > debug it via a serial console (using Minicom) and managed to get some
> > more Xen output, after the screen turns black.
> >  
> > I mention that I have tried to boot the PVH dom0 using different kernel
> > images (from 4.9.0 to 4.18-rc3), different Xen  versions (4.10, 4.11, 4.12).
> >
> > Below I attached my system / hypervisor configuration, as well as the
> > output captured through the serial console, corresponding to the latest
> > versions for Xen and the Linux Kernel (Xen staging and Kernel from the
> > xen/tip tree).
> >
> >
> > OS + Distro: Linux / Debian 9 Stretch
> > Kernel Version: 4.17-rc5, tagged with for-linus-4.18-rc5-tag from the
> > xen/tip tree.
> > Xen Version: 4.12, commit id e3f667bc5f51d0aa44357a64ca134cd952679c81
> > of the Xen tree.
> > Host system: attached cpuinfo.log
> > Serial console output: attached boot.log
> > My grub configuration file, containing the Xen command line arguments: attached
> > grub.log
> >
> > I can provide additional info as requested.
> > Any ideas why this happens? Do you have any recommendations for additional
> > debugging?
>
> Add "earlyprintk=xen" to the dom0 kernel parameters.
>
> Any reason why you are trying to use an iommu? I'd start with the simple
> case without iommu.

The IOMMU is a hard requirement for PVH Dom0.

Roger.

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: PVH dom0 creation fails - the system freezes

Manfred Haertel, DB3HM
Roger Pau Monné schrieb:

>> Any reason why you are trying to use an iommu? I'd start with the simple
>> case without iommu.
>
> The IOMMU is a hard requirement for PVH Dom0.

Why?

--
Manfred Härtel, DB3HM    mailto:[hidden email]
                          http://rz-home.de/mhaertel

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: PVH dom0 creation fails - the system freezes

Roger Pau Monné-3
On Mon, Jul 23, 2018 at 07:04:52PM +0200, Manfred Haertel, DB3HM wrote:
> Roger Pau Monné schrieb:
>
> > > Any reason why you are trying to use an iommu? I'd start with the simple
> > > case without iommu.
> >
> > The IOMMU is a hard requirement for PVH Dom0.
>
> Why?

Because a PVH Dom0 runs in autotranslated mode (using hardware
assisted paging), and thus the Dom0 kernel is going to use guest
physical addresses in order to interact with devices. The IOMMU is
required so that the devices also have such assisted paging and write
to the translated guest physical addresses.

Roger.

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xenproject.org/mailman/listinfo/xen-users