huynhjl | 1 Sep 2011 07:06
Picon

Re: Stream.toIterator behavior changed from 2.8.x to 2.9.x

Thanks, I'll keep that in mind the next time I'm tempted to use
Stream.

In this instance I can switch to an Iterator.

On Aug 31, 4:26 am, Josh Suereth <joshua.suer...@...> wrote:
> In general, Stream is not a great choice if you know the size of the stream
> will be large and you're worried about memory consumption.   I'd recommend
> making a TraversableView by creating a Traversable and immediately calling
> ".view" on it.   Then try to avoid anything that iterates until absolutely
> necessary.
>
> Hope that helps!
> - Josh
>
>
>
>
>
>
>
> On Wed, Aug 31, 2011 at 2:30 AM, huynhjl <huyn...@...> wrote:
> > Ah, I seehttps://issues.scala-lang.org/browse/SI-4835
> > *StreamIterator[+A] has O(n) memory consumption*
> > I guess it's known, though it was not indicated this worked fine in
> > 2.8.x.
>
> > On Aug 30, 10:56 pm, huynhjl <huyn...@...> wrote:
> > > I was upgrading one of my projects from 2.8.0 to 2.9.0 and kept
> > > getting out of memory errors once I started to churn large amount of
> > > data. I eventually traced it to a change in behavior in 2.9.0.
>
> > > The reduced test case is:
>
> > > val iter = Stream.from(1).toIterator
> > > iter.foreach(i => i)
>
> > > In 2.8.1, this runs in constant memory.
> > > In 2.9.0 and 2.9.1.RC4, there must be a reference hanging around as
> > > memory usage keeps increasing until a OutOfMemoryError.
>
> > > Is that a known behavior? Have I assumed too much about
> > > Stream.toIterator? It seemed like a nice way to not keep around the
> > > head of the stream.
>
> > > --Jean-Laurent


Gmane