domU shutdown as (null)

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

domU shutdown as (null)

Valentin Vidic
Hi,

Is it normal for a domU with a lot of memory to appear as (null) for a minute
or so after shutdown?  It seems to be scrubbing memory and I'm unable to
create a new machine until this finishes.

# xl list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0 10240    56     r-----     276.9
test                                         2 204800    56     -b----      32.2

# xl shutdown test
# xl list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0 10240    56     r-----     276.9
(null)                                       2 204800    56     --ps--      32.8
...
(null)                                       2 186878    56     --ps--      32.8
...
(null)                                       2 164308    56     --ps--      32.8

# xl create /etc/xen/test.cfg
Parsing config from /etc/xen/test.cfg
libxl: notice: libxl_numa.c:518:libxl__get_numa_candidate: NUMA placement failed, performance might be affected
xc: error: panic: xc_dom_boot.c:141: xc_dom_boot_mem_init: can't allocate low memory for domain: Out of memory
libxl: error: libxl_dom.c:653:libxl__build_dom: xc_dom_boot_mem_init failed: Cannot allocate memory
libxl: error: libxl_create.c:1223:domcreate_rebuild_done: cannot (re-)build domain: -3
libxl: error: libxl.c:1575:libxl__destroy_domid: non-existant domain 3
libxl: error: libxl.c:1534:domain_destroy_callback: unable to destroy guest with domid 3
libxl: error: libxl.c:1463:domain_destroy_cb: destruction of domain 3 failed

# xl list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0 10240    56     r-----     276.9
(null)                                       2 27448    56     --ps--      32.8
...
(null)                                       2 19889    56     --ps--      32.8
...
(null)                                       2 14598    56     --ps--      32.8

The system uses Xen 4.8.1 and Linux 4.9.30-2+deb9u2.

--
Valentin

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: domU shutdown as (null)

J. Roeleveld
On Friday, August 4, 2017 10:20:31 AM CEST Valentin Vidic wrote:
> Hi,
>
> Is it normal for a domU with a lot of memory to appear as (null) for a
> minute or so after shutdown?  It seems to be scrubbing memory and I'm
> unable to create a new machine until this finishes.

Yes, I see this all the time.
Don't often restart VMs myself, especially not the big ones. So it doesn't
affect me as much.

I think it is possible to disable the memory-scrubbing. Which should speed
this up. But then the new VM can see the memory-content of the previous VM.
(potential security-risk)


--
Joost



_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: domU shutdown as (null)

Valentin Vidic
On Fri, Aug 04, 2017 at 04:21:01PM +0200, J. Roeleveld wrote:
> Yes, I see this all the time.
> Don't often restart VMs myself, especially not the big ones. So it doesn't
> affect me as much.
>
> I think it is possible to disable the memory-scrubbing. Which should speed
> this up. But then the new VM can see the memory-content of the previous VM.
> (potential security-risk)

Thanks for confirming. While I don't mind the scrubbing, the cluster
manager (Pacemaker) notices the machine name is gone and tries to
restart it. This than fails with that Out of memory error. Not sure
if there is a way around this other than restarting these big machines
manually?

--
Valentin

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: domU shutdown as (null)

Dario Faggioli-2
On Fri, 2017-08-04 at 18:29 +0200, Valentin Vidic wrote:

> On Fri, Aug 04, 2017 at 04:21:01PM +0200, J. Roeleveld wrote:
> > Yes, I see this all the time.
> > Don't often restart VMs myself, especially not the big ones. So it
> > doesn't 
> > affect me as much.
> >
> > I think it is possible to disable the memory-scrubbing. Which
> > should speed 
> > this up. But then the new VM can see the memory-content of the
> > previous VM. 
> > (potential security-risk)
>
> Thanks for confirming. While I don't mind the scrubbing, the cluster
> manager (Pacemaker) notices the machine name is gone and tries to
> restart it. This than fails with that Out of memory error. Not sure
> if there is a way around this other than restarting these big
> machines
> manually?
>
Just FYI, we're working on improving this.

Basically, memory will be left unscrubbed when destroying the domain.
It will be scrubbed afterwords, in batches, during system idle time, or
when necessary.

This is the latest version of the patch series:
https://www.mail-archive.com/xen-devel@.../msg117353.html

Hopefully, it will be included in Xen 4.10.

Regards,
Dario
--
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: domU shutdown as (null)

Valentin Vidic
On Wed, Aug 09, 2017 at 12:15:46PM +0200, Dario Faggioli wrote:

> Just FYI, we're working on improving this.
>
> Basically, memory will be left unscrubbed when destroying the domain.
> It will be scrubbed afterwords, in batches, during system idle time, or
> when necessary.
>
> This is the latest version of the patch series:
> https://www.mail-archive.com/xen-devel@.../msg117353.html
>
> Hopefully, it will be included in Xen 4.10.

That sounds great, thanks for the info.

--
Valentin

_______________________________________________
Xen-users mailing list
[hidden email]
https://lists.xen.org/xen-users
Loading...