1 Sep 2011 07:06
Re: Stream.toIterator behavior changed from 2.8.x to 2.9.x
Thanks, I'll keep that in mind the next time I'm tempted to use Stream. In this instance I can switch to an Iterator. On Aug 31, 4:26 am, Josh Suereth <joshua.suer...@...> wrote: > In general, Stream is not a great choice if you know the size of the stream > will be large and you're worried about memory consumption. I'd recommend > making a TraversableView by creating a Traversable and immediately calling > ".view" on it. Then try to avoid anything that iterates until absolutely > necessary. > > Hope that helps! > - Josh > > > > > > > > On Wed, Aug 31, 2011 at 2:30 AM, huynhjl <huyn...@...> wrote: > > Ah, I seehttps://issues.scala-lang.org/browse/SI-4835 > > *StreamIterator[+A] has O(n) memory consumption* > > I guess it's known, though it was not indicated this worked fine in > > 2.8.x. > > > On Aug 30, 10:56 pm, huynhjl <huyn...@...> wrote: > > > I was upgrading one of my projects from 2.8.0 to 2.9.0 and kept > > > getting out of memory errors once I started to churn large amount of > > > data. I eventually traced it to a change in behavior in 2.9.0. > > > > The reduced test case is: > > > > val iter = Stream.from(1).toIterator > > > iter.foreach(i => i) > > > > In 2.8.1, this runs in constant memory. > > > In 2.9.0 and 2.9.1.RC4, there must be a reference hanging around as > > > memory usage keeps increasing until a OutOfMemoryError. > > > > Is that a known behavior? Have I assumed too much about > > > Stream.toIterator? It seemed like a nice way to not keep around the > > > head of the stream. > > > > --Jean-Laurent
RSS Feed