Re: Setting Content-Length on a streamed file

Charles Duffy wrote:
> I'm sending a very large, dynamically generated file with
> a fixed, known length to the client. This being the case,
> I set cherrypy.response.headers['Content-Length'] to the
> known target length before returning the generator which
> actually creates the content.

I assume you want it so the user-agent can inform the user about progress?

> However, it never makes it to the client: Content-Length
> is actively deleted if it exists and we're streaming content.
> This happens in _cphttptools.py's finalize() at line 397
> in CP2.2.1. I'm presuming there's a reason this is done
> -- but would it be safe to delete Content-Length only if
> its value is None? (If it's safe, is there anything I
> should do to help make this default behavior in future
> versions -- file a bug, submit a patch or such?)

The Content-Length header is deleted in order to more-gracefully handle errors that occur in your
generator while the content is being streamed out. If you set Content-Length to 10000, and an exception is
raised on byte 10, for example, the client may hang on the socket. As
http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.4 says:

   "When a message-body is included with a message, the
    transfer-length of that body is determined by one of
    the following (in order of precedence):

    ...

    3.If a Content-Length header field (section 14.13) is
      present...

    ...

    5.By the server closing the connection."

...and _cpwsgiserver does indeed close the connection if there is no CL response header (see
_cpwsgiserver.HTTPRequest.send_headers). So it would be "safe" to send it only if your code has no bugs.
;) So IMO we can either provide a safe mechanism or an unsafe mechanism (given the various places one might
unwittingly have Content-Length set for them), and IMO we're going to stick with the safe mechanism for now.

The solution I've wanted to pursue for a while now is this: provide a different "safe" mechanism via the
"chunked" transfer-coding (see
http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.6.1), which would require
cooperation on the WSGI server side. If we had that, then we could make that the norm and allow "streaming"
content to be unsafe in rare cases. But until we have that, I have a hard time only providing an "unsafe" mechanism.

So a ticket would be a good thing. :)

Robert Brewer
System Architect
Amor Ministries
fumanchu@...

--~--~---------~--~----~------------~-------~--~----~
 You received this message because you are subscribed to the Google Groups "cherrypy-users" group.
To post to this group, send email to cherrypy-users@...
To unsubscribe from this group, send email to cherrypy-users-unsubscribe@...
For more options, visit this group at http://groups-beta.google.com/group/cherrypy-users?hl=en
-~----------~----~----~----~------~----~------~--~---