19 Jun 2012 22:30
Re: vm state save/restore question
Benjamin Herrenschmidt <benh <at> kernel.crashing.org>
2012-06-19 20:30:41 GMT
2012-06-19 20:30:41 GMT
On Tue, 2012-06-19 at 16:59 +0200, Juan Quintela wrote: > >> - The hash table (mentioned above). This is just a big chunk of > memory > >> (it will routinely be 16M), so I really don't want to start > iterating > >> all elements, just a bulk load will do, and the size might actually > be > >> variable. > > This is going to kill migration download time. With current setup, we > just sent something like 1-2MB in stage 3 (i.e. after the machine is > down). Default downtime is 30ms, And 16MB is going to take around 1s > on gigabit ethenet. > > Once said that, if you told me the state that you want to sent, I can > take a look. Well, we don't have much of a choice unless we do something really fancy but that would be a second step... The MMU hash table on power is where all our translations go. What we could do is put in some knowledge about what translations are actually necessary for the guest and which ones can be rebuild (faulted in), essentially by adding knowledge to qemu/kvm about the "bolted" bit that the guests uses for translations that must not be evicted. However, that would require at least some interaction between the guest and qemu/kvm to enable the function since this bit is a guest SW construct (unless it got architected in recent PAPR, I need to dbl check). In that case, we save a "compressed" variant of the hash table (RLE style ? we need to keep the hash location of each entry ...) which contains only the bolted translations. The consequence is that post-migration, the guest will temporarily slow down as most translations will have to be faulted back in. > >> - The XICS (interrupt controller). The actual size of the interrupt > >> state array can vary (the number of interrupt sources can vary, > it's > >> fixed today by the machine code but I wouldn't rely too much on > that and > >> in any case, from the XICS driver perspective, it's not a constant, > it's > >> a variable it gets passed when initializing). > > Can you point me at the structure that you want to sent? hw/xics.c, there's several bits here but the "array" itself is the struct ics_state which has a pointer to an array of struct ics_irq_state. However, I think I figured out how to do that by using the array macro which takes the size of the array as a pointer to a variable containing that size or am I mistaken ? > >> So in both these cases, I need either code to control the save/load > >> process (old style ? hard to hook into vmstate as far as I can > tell) or > >> maybe a way to describe the array so that the array size itself is > a > >> pointer to a variable (Andreas mentioned something along those > lines). > >> Is there any doco for that stuff btw ? I haven't seen anything > >> detailed... > > > > I'm sure Juan knows more there :) > > thanks for pointing me to the discussion O Thanks for the help ! Cheers, Ben.