Re: pNFS some minor changes

Brent Welch <welch <at> panasas.com> wrote on 09/08/2005 02:05:14 PM:

> 
> >>>"J. Bruce Fields" said:
>  > 
>  > On Wed, Sep 07, 2005 at 11:25:14AM -0700, Brent Welch wrote:
>  > > Keep in mind that some applications have single files that are
>  > > multi-terabytes in size, and so distributing them over many, many
>  > > servers may be just what you want to do.
>  > > 
>  > > I think Garth has put a parameter in that specifies how much
>  > > memory the client has for the returned layout.
>  > 
>  > I see that LAYOUTGET has a "maxcount" parameter, and the server can
>  > return TOOSMALL errors.  So a client can retry with increasingly 
large
>  > buffers.  Does a client that wants to interoperate with any pNFS 
server
>  > need to be prepared to retry with an arbitrarily large buffers?
> 
> Not necessarily.  Even if my multi-terabyte file is distributed over
> 1000's of data servers, I could get back a much smaller map that just
> provides a multi-gigabyte window into that file.   So, the model is not
> to retry until you get a multi-terabyte layout, but to do your I/O
> in smaller ranges of the file.  You may need to be creative in your
> layout definition to do that efficiently, but in the worst case of a
> 64K stripe unit spread over 10 million data servers, the server
> could give out a layout for 64 Meg that listed 1000 servers.
> Now, I wouldn't implement a layout like that, partly for this reason,
> but the client could make forward progress.  I would only expect 
TOOSMALL if
> the client-supplied buffer were just a handful of bytes or something.
> 
The only problem with this approach is that if you can not fit all the 
data servers in to one layout that describes striped file you have switch 
to a one to one mapping of the file which can take many messages to 
describe for a very big file.
Marc.

_______________________________________________
nfsv4 mailing list
nfsv4 <at> ietf.org
https://www1.ietf.org/mailman/listinfo/nfsv4