question on balloon driver

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

question on balloon driver

Jonas Appel
Hello,

I'm trying to get into the internals of xen and especially the balloon
driver. During the time I dealt with that, several questions came up:

* Does (and if yes, how) the balloon driver in an guest OS have to be
activated?

* Does the automatic release of memory from a guest OS to Xen already
work?

* Is the limit of 100 Guest OSs (as mentioned on
http://en.wikipedia.org/wiki/Xen_%28virtual_machine_monitor%29 ) really
hardcoded somewhere or is it more a practical value (or does the limit
not exist)?

Thanks in advance & regards,

Jonas Appel


_______________________________________________
Xen-devel mailing list
[hidden email]
http://lists.xensource.com/xen-devel
Reply | Threaded
Open this post in threaded view
|

Re: question on balloon driver

Anthony Liguori
Jonas Appel wrote:

>Hello,
>
>I'm trying to get into the internals of xen and especially the balloon
>driver. During the time I dealt with that, several questions came up:
>
>* Does (and if yes, how) the balloon driver in an guest OS have to be
>activated?
>  
>
No.  In Linux, it's a normal driver that's loaded and initialized during
system startup.

>* Does the automatic release of memory from a guest OS to Xen already
>work?
>  
>
It depends on what you mean by automatic.  I'm not sure anyone has ever
posted an automated memory balancer.

>* Is the limit of 100 Guest OSs (as mentioned on
>http://en.wikipedia.org/wiki/Xen_%28virtual_machine_monitor%29 ) really
>hardcoded somewhere or is it more a practical value (or does the limit
>not exist)?
>  
>
It's a design goal, not a limit.  I've heard of folks actually getting
more than 100 domains going at once (although the domains were not
full-blown linux instances).

Regards,

Anthony Liguori

>Thanks in advance & regards,
>
>Jonas Appel
>
>
>_______________________________________________
>Xen-devel mailing list
>[hidden email]
>http://lists.xensource.com/xen-devel
>
>  
>


_______________________________________________
Xen-devel mailing list
[hidden email]
http://lists.xensource.com/xen-devel
Reply | Threaded
Open this post in threaded view
|

Re: question on balloon driver

Mark Williamson
In reply to this post by Jonas Appel
> I'm trying to get into the internals of xen and especially the balloon
> driver. During the time I dealt with that, several questions came up:

Excellent!  The more people who know about this stuff (and preferably hack on
it ;-) the better.

> * Does (and if yes, how) the balloon driver in an guest OS have to be
> activated?

It's purely manual right now: you set a target memory allocation and the
balloon driver will try and resize the guest to fit.

The target is set by an interface in /proc, or by a message on the Xen control
interface (soon to be replaced by a value in the xenstore registry).

> * Does the automatic release of memory from a guest OS to Xen already
> work?

Nope, we'd like some kind of daemon to run in dom0 and balloon guests
according to their need.  This should probably include some activity
statistics reported by the guests, so the daemon can figure out which needs
more memory.  Swap statistics might give a good idea of which guests needed
memory.

> * Is the limit of 100 Guest OSs (as mentioned on
> http://en.wikipedia.org/wiki/Xen_%28virtual_machine_monitor%29 ) really
> hardcoded somewhere or is it more a practical value (or does the limit
> not exist)?

You should be able to start several hundred guests before hitting any hard
coded limits (e.g. number of event channels  - I'm told there's a plan to
remove this restriction).  Before you get to that stage, most systems will
run out of hardware oomph to run so many guests *and* still do something
useful.  I think Ian started somewhere over 100 guests one time and IIRC he
ran out of disk images before hitting any limits :-)

Any limits will likely be addressed, since people are a) starting to use very
large machines with Xen (Unisys are running it on 32 CPU boxes) and b) using
cunning tricks to get extremely large numbers of guests running (I've heard
"thousands" mentioned).

Cheers,
Mark

_______________________________________________
Xen-devel mailing list
[hidden email]
http://lists.xensource.com/xen-devel
Reply | Threaded
Open this post in threaded view
|

Re: question on balloon driver

Mark Williamson
In reply to this post by Anthony Liguori
> It depends on what you mean by automatic.  I'm not sure anyone has ever
> posted an automated memory balancer.

One fine day, someone should write this.  It'd allow you to trade off a bit of
isolation for a bit of statistical multiplexing in the case where your
domains don't have a paranoid mistrust of each other.

VMware ESX sets a minimum, a maximum and a target percentage share for each
machine.  It then does sampling of the domains to find out how much of their
memory they're *really* using.  If they're not using all of it, they get told
to give it back using the balloon driver and it's apportioned to domains that
are using their memory more heavily.

This sort of thing should carry fairly straightforwardly across to Xen.  We
could simply get domains to report their current utilisation through the
store without needing to do sampling behind their back.

Could be a fun project for someone looking to get into Xen stuff (yes, that's
an unsubtle hint to the world at large :-) ).

Cheers,
Mark

_______________________________________________
Xen-devel mailing list
[hidden email]
http://lists.xensource.com/xen-devel
Reply | Threaded
Open this post in threaded view
|

Re: question on balloon driver

Josh Triplett-3
In reply to this post by Mark Williamson
On Tue, 2005-08-02 at 01:48 +0100, Mark Williamson wrote:

> > * Is the limit of 100 Guest OSs (as mentioned on
> > http://en.wikipedia.org/wiki/Xen_%28virtual_machine_monitor%29 ) really
> > hardcoded somewhere or is it more a practical value (or does the limit
> > not exist)?
>
> You should be able to start several hundred guests before hitting any hard
> coded limits (e.g. number of event channels  - I'm told there's a plan to
> remove this restriction).  Before you get to that stage, most systems will
> run out of hardware oomph to run so many guests *and* still do something
> useful.  I think Ian started somewhere over 100 guests one time and IIRC he
> ran out of disk images before hitting any limits :-)

At least on my machine, with current unstable, I cannot create more than
140 *completely empty* domains (just created, no resources assigned);
attempting to create more at that point gives an out-of-memory error
(which I believe is referring to the Xen hypervisor kernel memory).
Perhaps there is some switch I haven't seen which allows increasing the
amount of memory allocated to the Xen kernel?  It would also be nice if
domains had far less overhead until resources were allocated to them.

- Josh Triplett



_______________________________________________
Xen-devel mailing list
[hidden email]
http://lists.xensource.com/xen-devel
Reply | Threaded
Open this post in threaded view
|

RE: question on balloon driver

Ian Pratt
In reply to this post by Jonas Appel
 
> At least on my machine, with current unstable, I cannot
> create more than 140 *completely empty* domains (just
> created, no resources assigned); attempting to create more at
> that point gives an out-of-memory error (which I believe is
> referring to the Xen hypervisor kernel memory).
> Perhaps there is some switch I haven't seen which allows
> increasing the amount of memory allocated to the Xen kernel?  
> It would also be nice if domains had far less overhead until
> resources were allocated to them.

Sounds like you're running out of Xen heap, which is currently
statically sized.
Each domain 'shell' allocates something like 64KB of storage.

I'm not sure we'll get around to changing this anytime soon on, at least
on x86_32. On x86_64 there's no need for there to be a distinction
between domain and xen heap pages, hence the xen heap can be dynamically
sized (at least in the bottom 4GB)

Ian

_______________________________________________
Xen-devel mailing list
[hidden email]
http://lists.xensource.com/xen-devel
Reply | Threaded
Open this post in threaded view
|

Re: question on balloon driver

Keir Fraser
In reply to this post by Josh Triplett-3

On 4 Aug 2005, at 23:03, Josh Triplett wrote:

> At least on my machine, with current unstable, I cannot create more
> than
> 140 *completely empty* domains (just created, no resources assigned);
> attempting to create more at that point gives an out-of-memory error
> (which I believe is referring to the Xen hypervisor kernel memory).
> Perhaps there is some switch I haven't seen which allows increasing the
> amount of memory allocated to the Xen kernel?  It would also be nice if
> domains had far less overhead until resources were allocated to them.

You run out of Xen heap. We won;t be able to make any more available on
x86_32, but there is a boot parameter to x86/64 Xen to set the amount
of Xen heap, in megabytes.

  -- Keir


_______________________________________________
Xen-devel mailing list
[hidden email]
http://lists.xensource.com/xen-devel