Merge branch 'master' into po

Conflicts:
	debian/changelog

1 files changed, 91 insertions, 0 deletions

diff --git a/doc/forum/speeding_up_ikiwiki.mdwn b/doc/forum/speeding_up_ikiwiki.mdwn
new file mode 100644
index 000000000..0b2164238
--- /dev/null
+++ b/doc/forum/speeding_up_ikiwiki.mdwn
@@ -0,0 +1,91 @@
+My website takes a fairly long time to render. It takes a long time to do
+things like add pages, too. I'd like to try and understand what takes the
+time and what I might be able to do to speed things up.
+
+I have 1,234 objects on my site (yikes!). 717 are items under "/log" which
+I think might be the main culprit because there are some complex pagespecs
+operating in that area (e.g. log/YYYY/MM/DD, YYYY/MM and YYYY for YYYY >=
+2003, YYYY <= 2008 which include every page under log/ which was modified
+in the corresponding YYYY or YYYY/MM or YYYY/MM/DD). There is very little
+linking between the world outside of /log and that within it.
+
+I was interested in generating a graphical representation of ikiwiki's idea of
+page inter-dependencies. I started by looking at the '%links' hash using the
+following plugin:
+
+    #!/usr/bin/perl
+    package IkiWiki::Plugin::deps;
+    
+    use warnings;
+    use strict;
+    use IkiWiki 3.00;
+    
+    
+    sub import {
+        hook(type => "format", id => "deps", call => \&fooble);
+    }
+    
+    my $hasrun = 0;
+    
+    sub fooble ($$) {
+        if(0 == $hasrun) {
+            $hasrun = 1;
+            open MYFILE, ">/home/jon/deps.dot";
+            foreach my $key (keys %links) {
+                my $arrref = $links{$key};
+                foreach my $page (@$arrref) {
+                    print MYFILE "$key -> $page;\n";
+                }
+            }
+            close MYFILE;
+        }
+    }
+    
+    1
+
+The resulting file was enormous: 2,734! This turns out to be because of the following code in scan() in Render.pm:
+
+    if ($config{discussion}) {$
+        # Discussion links are a special case since they're
+        # not in the text of the page, but on its template.
+        $links{$page}=[ $page."/".gettext("discussion") ];
+
+Worst case (no existing discussion pages) this will double the number of link
+relationships. Filtering out all of those, the output drops to 1,657. This
+number is still too large to really visualize: the graphviz PNG and PDF output
+engines segfault for me, the PS one works but I can't get any PS software to
+render it without exploding.
+
+Now, the relations in the links hash are not the same thing as IkiWiki's notion of dependencies. Can anyone point me at  that data structure / where I might be able to add some debugging foo to generate a graph of it?
+
+Once I've figured out that I might be able to optimize some pagespecs. I
+understand pagespecs are essentially translated into sequential perl code. I
+might gain some speed if I structure my complex pagespecs so that the tests
+which have the best time complexity vs. "pages ruled out" ratio are performed
+first.
+
+I might also be able to find some dependencies which shouldn't be there and
+remove the dependency.
+
+In general any advice people could offer on profiling ikiwiki would be great.
+I did wonder about invoking the magic profiling arguments to perl via the CGI
+wrapper.
+
+
+-- [[Jon]]
+
+> Dependencies go in the `%IkiWiki::depends` hash, which is not exported. It
+> can also be dumped out as part of the wiki state - see [[tips/inside_dot_ikiwiki]].
+>
+> It's a map from page name to increasingly complex pagespec, although
+> the `optimize-depends` branch in my git repository changes that to a
+> map from a page name to a *list* of pagespecs which are automatically
+> or'd together for use (this at least means duplicates can be weeded out).
+>
+> See [[todo/should_optimise_pagespecs]] for more on that.
+>
+> I've been hoping to speed up IkiWiki too - making a lot of photo albums
+> with my [[plugins/contrib/album]] plugin makes it pretty slow.
+>
+> One thing that I found was a big improvement was to use `quick=yes` on all
+> my `archive=yes` [[ikiwiki/directive/inline]]s. --[[smcv]]