Re: OutOfMemoryError

On 23 Aug 2004, at 19:43, Eric Preston wrote:
> I'm using Groovy SQL on RHLinux to loop through a large (400K rows)
> table and process each of the entries in the table. After some # I get
> Exception in thread "main" java.lang.OutOfMemoryError.
>
> Does groovy select all rows at once and keep in the result set?

No. Its literally just iteresting through a regular JDBC result set 
when using the eachRow() method...

>  I can
> break up the query into smaller segments, but that's a pain.
>
> I'm using 1.0 beta 6 on RHL 7.1 with JDK 1.4.1_05
>
> The groovy code is pretty simple:
>
> import groovy.sql.Sql
> import portal.util.DBUtil
> import portal.util.LyrisUtil
>
> sql = new Sql(DBUtil.getDirectConnection())
>
> // Get the users
> println "-- Get users and update other database"
>
> userCount = 0
>
> sql.eachRow("SELECT userpk FROM users") { user |
> 	println "-- Updating info for the userid: ${user.userpk}"
>
> 	try {
> 		LyrisUtil.updateMember (user.userpk)
> 		userCount++
> 	} catch (Exception e) {
> 		println "Exception for user: ${user.userpk} - ${e}"
> 	}
> }
>
> Thanks,
>
> Eric Preston
> TheServerSide.com
> eric@...

I wonder if the JDBC driver is keeping the result set in RAM? I'm 
guessing it could be your JDBC driver can't handle such a thing; as we 
don't really keep anything around other than the ResultSet & Statement 
& Connection and a lightweight wrapper around the result set.

Maybe you just need to increase your heap a little for the JDBC driver 
to work?

James
-------
http://radio.weblogs.com/0112098/