[Trac-dev] Re: Improving next_rev for scoped svn paths

Hi Tim,

Tim Hatch wrote:
> Hi all,
>
> We've had a number of tickets related to either silly memory growth  
> or slow performance on scoped svn repositories (in particular,  
> #5213).  This is a big issue for everyone since the same code paths  
> are taken for restricted changesets (which search engines will index,  
> even if your users don't click on them).
>
> I saw this happen for one big user of Trac and tracked down that the  
> cached repository isn't using a cached representation of next_rev.   
>   

Was he using a recent enough 0.11dev? In particular, was this happening 
despite of r6594?

> I'd like review of the following patch, which improves the situation  
> bigtime on my system.  In particular, I'm not aware of any backends  
> using CachedRepository other than svn, which uses numeric revs  
> (please correct me if wrong).
>   

In case this is a problem, we could add a "linear_changesets" property 
to the CachedRepository: If set to True, this would take the fast path. 
I've updated the patch along these lines (and it seems to work quite 
well, please test also on your side).
Next step would be to do that for prev_rev as well.

-- Christian

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac Development" group.
To post to this group, send email to trac-dev <at> googlegroups.com
To unsubscribe from this group, send email to trac-dev-unsubscribe <at> googlegroups.com
For more options, visit this group at http://groups.google.com/group/trac-dev?hl=en
-~----------~----~----~----~------~----~------~--~---

# HG changeset patch
# User Christian Boos <cboos <at> neuf.fr>
# Date 1209033788 -7200
# Node ID 9bf8d572e750a555b1e1d5a47f9f109020470c4f
# Parent  060476bdb0f27d6be09bea6947f29b43c71cf130
Use the vc cache for optimizing `next_rev` (only for the Subversion backend for now)

Initial patch by Tim Hatch

diff --git a/trac/versioncontrol/cache.py b/trac/versioncontrol/cache.py
--- a/trac/versioncontrol/cache.py
+++ b/trac/versioncontrol/cache.py
@@ -37,6 +37,8 @@
 
 class CachedRepository(Repository):
 
+    has_linear_changesets = False
+
     def __init__(self, db, repos, authz, log):
         Repository.__init__(self, repos.name, authz, log)
         self.db = db
@@ -236,7 +239,31 @@
         return self.repos.previous_rev(rev)
 
     def next_rev(self, rev, path=''):
-        return self.repos.next_rev(rev, path)
+        if not self.has_linear_changesets:
+            return self.repos.previous_rev(rev)
+
+        # the changeset revs are sequence of ints:
+        sql = "SELECT rev FROM node_change WHERE " + \
+              self.db.cast('rev', 'int') + " > %s"
+        args = [rev]
+
+        if path:
+            # Child changes
+            sql += " AND (path %s OR " % self.db.like()
+            args.append(self.db.like_escape(path.lstrip('/')) + '%')
+            # Parent deletion
+            components = path.lstrip('/').split('/')
+            for i in range(1, len(components)+1):
+                args.append('/'.join(components[:i]))
+            parent_insert = ','.join(('%s',) * len(components))
+            sql += " (path in (" + parent_insert + ") and change_type='D') )"
+
+        sql += " ORDER BY " + self.db.cast('rev', 'int') + " LIMIT 1"
+        
+        cursor = self.db.cursor()
+        cursor.execute(sql, args)
+        for rev, in cursor:
+            return rev
 
     def rev_older_than(self, rev1, rev2):
         return self.repos.rev_older_than(rev1, rev2)
diff --git a/trac/versioncontrol/svn_fs.py b/trac/versioncontrol/svn_fs.py
--- a/trac/versioncontrol/svn_fs.py
+++ b/trac/versioncontrol/svn_fs.py
@@ -280,6 +280,7 @@
         else:
             repos = CachedRepository(self.env.get_db_cnx(), fs_repos, None,
                                      self.log)
+            repos.has_linear_changesets = True
         if authname:
             authz = SubversionAuthorizer(self.env, weakref.proxy(repos),
                                          authname)