"It's disallowed to assign <device> with shared RMRR at <address> for Dom<N>" after upgrading to 4.6

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

"It's disallowed to assign <device> with shared RMRR at <address> for Dom<N>" after upgrading to 4.6

Stephen Oberholtzer
So I finally upgraded my system -- kernel 3.16 -> 4.6, Xen 4.5 -> 4.6, qemu 2.1 -> 2.6, and now my HVM domain with PCI passthrough won't start.

The first issue was that xl submits a now-invalid command line to qemu for an empty cdrom (qemu gives a truly bizarre error message here: "Must specify either driver or file").  I gave up and removed the drive entirely; hopefully I don't need to use that drive anytime soon.

The second issue is something about shared RMRRs. Unfortunately, I can't seem to get a lot of information on what's going on here; something about a security hole?

xl create fails with the message:

libxl: error: libxl_create.c:1424:domcreate_attach_pci: libxl_device_pci_add failed: -3

I finally found in 'xl dmesg' this line:

(XEN) [VT-D] It's disallowed to assign 0000:00:14.0 with shared RMRR at <address?> for Dom4.
(XEN) XEN_DOMCTL_assign_device: assign 0000:00:14.0 to dom4 failed (-1)

Unfortunately, I can't seem to find out how to understand what's going on here.
How can I view these RMRRs to find out what's being shared? (I turned on every option I could think of for Xen's logging, but no luck.)
How do I make my VM work again?


--
-- Stevie-O
Real programmers use COPY CON PROGRAM.EXE


_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: "It's disallowed to assign <device> with shared RMRR at <address> for Dom<N>" after upgrading to 4.6

Peter Gansterer
Hi,

I want to second that:
I ran into the same issue when I tried to upgrade from 4.5 to 4.6 in february.
I didn't spend the time to get into it ... so I'm still at 4.5.

In my case it's an Asrock board with a Sandy Bridge Core-i5 and I assign the USB controllers to different domains.
- 00:14.0 xhci (stays at dom0)
- 00:1a.0 ehci (HVM1)
- 00:1d.0 ehci (HVM2)

With 4.5 it works fine, with 4.6 it doesn't.

BUT: I came as far as using "rdm_policy=relaxed" for the USB devices ... the error message goes away; but then the win7 HVM won't boot.

Maybe rdm_policy=relaxed works for you.
And I would appreciate further suggestions for my case as well :-)


- peter.


On 2016-08-01 22:09:14 Stephen Oberholtzer wrote:

> So I finally upgraded my system -- kernel 3.16 -> 4.6, Xen 4.5 -> 4.6, qemu
> 2.1 -> 2.6, and now my HVM domain with PCI passthrough won't start.
>
> The first issue was that xl submits a now-invalid command line to qemu for
> an empty cdrom (qemu gives a truly bizarre error message here: "Must
> specify either driver or file").  I gave up and removed the drive entirely;
> hopefully I don't need to use that drive anytime soon.
>
> The second issue is something about shared RMRRs. Unfortunately, I can't
> seem to get a lot of information on what's going on here; something about a
> security hole?
>
> xl create fails with the message:
>
> libxl: error: libxl_create.c:1424:domcreate_attach_pci:
> libxl_device_pci_add failed: -3
>
> I finally found in 'xl dmesg' this line:
>
> (XEN) [VT-D] It's disallowed to assign 0000:00:14.0 with shared RMRR at
> <address?> for Dom4.
> (XEN) XEN_DOMCTL_assign_device: assign 0000:00:14.0 to dom4 failed (-1)
>
> Unfortunately, I can't seem to find out how to understand what's going on
> here.
> How can I view these RMRRs to find out what's being shared? (I turned on
> every option I could think of for Xen's logging, but no luck.)
> How do I make my VM work again?
>
>
> --
> -- Stevie-O
> Real programmers use COPY CON PROGRAM.EXE

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: "It's disallowed to assign <device> with shared RMRR at <address> for Dom<N>" after upgrading to 4.6

George Dunlap
In reply to this post by Stephen Oberholtzer
On Tue, Aug 2, 2016 at 3:09 AM, Stephen Oberholtzer <[hidden email]> wrote:
> So I finally upgraded my system -- kernel 3.16 -> 4.6, Xen 4.5 -> 4.6, qemu
> 2.1 -> 2.6, and now my HVM domain with PCI passthrough won't start.
>
> The first issue was that xl submits a now-invalid command line to qemu for
> an empty cdrom (qemu gives a truly bizarre error message here: "Must specify
> either driver or file").  I gave up and removed the drive entirely;
> hopefully I don't need to use that drive anytime soon.

Stephen,

Could you make a separate thread and do a more full report of this
bug?  We definitely want to keep the CDROM functionality working.

 -George

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: "It's disallowed to assign <device> with shared RMRR at <address> for Dom<N>" after upgrading to 4.6

Wei Liu-2
In reply to this post by Stephen Oberholtzer
On Mon, Aug 01, 2016 at 10:09:14PM -0400, Stephen Oberholtzer wrote:

> So I finally upgraded my system -- kernel 3.16 -> 4.6, Xen 4.5 -> 4.6, qemu
> 2.1 -> 2.6, and now my HVM domain with PCI passthrough won't start.
>
> The first issue was that xl submits a now-invalid command line to qemu for
> an empty cdrom (qemu gives a truly bizarre error message here: "Must
> specify either driver or file").  I gave up and removed the drive entirely;
> hopefully I don't need to use that drive anytime soon.
>
> The second issue is something about shared RMRRs. Unfortunately, I can't
> seem to get a lot of information on what's going on here; something about a
> security hole?
>

Sharing RMRR is considered unsafe so it is disallowed by default, IIRC.

> xl create fails with the message:
>
> libxl: error: libxl_create.c:1424:domcreate_attach_pci:
> libxl_device_pci_add failed: -3
>
> I finally found in 'xl dmesg' this line:
>
> (XEN) [VT-D] It's disallowed to assign 0000:00:14.0 with shared RMRR at
> <address?> for Dom4.
> (XEN) XEN_DOMCTL_assign_device: assign 0000:00:14.0 to dom4 failed (-1)
>
> Unfortunately, I can't seem to find out how to understand what's going on
> here.
> How can I view these RMRRs to find out what's being shared? (I turned on
> every option I could think of for Xen's logging, but no luck.)
> How do I make my VM work again?
>

Try `man xl.cfg' and search for rmrr. There are some new options for pci
passthrough.


Wei.

>
> --
> -- Stevie-O
> Real programmers use COPY CON PROGRAM.EXE

> _______________________________________________
> Xen-users mailing list
> [hidden email]
> https://lists.xen.org/xen-users


_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users
Reply | Threaded
Open this post in threaded view
|

Re: "It's disallowed to assign <device> with shared RMRR at <address> for Dom<N>" after upgrading to 4.6

Wei Liu-2
In reply to this post by Peter Gansterer
On Tue, Aug 02, 2016 at 09:30:59AM +0200, Peter Gansterer wrote:

> Hi,
>
> I want to second that:
> I ran into the same issue when I tried to upgrade from 4.5 to 4.6 in february.
> I didn't spend the time to get into it ... so I'm still at 4.5.
>
> In my case it's an Asrock board with a Sandy Bridge Core-i5 and I assign the USB controllers to different domains.
> - 00:14.0 xhci (stays at dom0)
> - 00:1a.0 ehci (HVM1)
> - 00:1d.0 ehci (HVM2)
>
> With 4.5 it works fine, with 4.6 it doesn't.
>
> BUT: I came as far as using "rdm_policy=relaxed" for the USB devices ... the error message goes away; but then the win7 HVM won't boot.
>

Unfortunately just knowing Win7 doesn't boot is not enough. You need to
extract more information...

Wei.

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users