Merge branch 'aggregateinternal'

5 files changed, 73 insertions, 24 deletions

diff --git a/IkiWiki/Plugin/aggregate.pm b/IkiWiki/Plugin/aggregate.pm
index e93492863..2fcdec9e7 100644
--- a/IkiWiki/Plugin/aggregate.pm
+++ b/IkiWiki/Plugin/aggregate.pm
@@ -120,46 +120,45 @@ sub htmlize (@) { #{{{
 	return $params{content};
 } #}}}
 
+# Used by ikiwiki-transition aggregateinternal.
 sub migrate_to_internal { #{{{
-
 	if (! lockaggregate()) {
-		error("an aggregation process is already running");
-		return;
+		error("an aggregation process is currently running");
 	}
 
 	IkiWiki::lockwiki();
 	loadstate();
+	$config{verbose}=1;
 
 	foreach my $data (values %guids) {
 		next unless $data->{page};
-
+		
 		$config{aggregateinternal} = 0;
 		my $oldname = pagefile($data->{page});
-
+		
 		$config{aggregateinternal} = 1;
 		my $newname = pagefile($data->{page});
-
-		print "I: $oldname -> $newname\n";
+		
+		debug "moving $oldname -> $newname";
 		if (-e $newname) {
 			if (-e $oldname) {
 				error("$newname already exists");
 			}
 			else {
-				print STDERR 
-					"W: already renamed to $newname?\n";
+				debug("already renamed to $newname?");
 			}
 		}
 		elsif (-e $oldname) {
 			rename($oldname, $newname) || error("$!");
 		}
 		else {
-			print "W: $oldname not found\n";
+			debug("$oldname not found");
 		}
 	}
-
+	
 	savestate();
 	IkiWiki::unlockwiki;
-
+	
 	unlockaggregate();
 } #}}}
 
diff --git a/doc/ikiwiki-transition.mdwn b/doc/ikiwiki-transition.mdwn
index 693c1db83..624268d23 100644
--- a/doc/ikiwiki-transition.mdwn
+++ b/doc/ikiwiki-transition.mdwn
@@ -25,6 +25,11 @@ Note that if the page contains wiki links with spaces, which some
 older versions of ikiwiki accepted, the prefix_directives transition will
 treat these as preprocessor directives and convert them.
 
+# aggregateinternal
+
+The `aggregateinternal` mode moves pages aggregated by the aggregate plugin
+so that the `aggregateinternal` option can be endabled 
+
 # indexdb
 
 The `indexdb` mode handles converting a plain text `.ikiwiki/index` file to
diff --git a/doc/ikiwiki.setup b/doc/ikiwiki.setup
index 6d327fd98..10cb3da1d 100644
--- a/doc/ikiwiki.setup
+++ b/doc/ikiwiki.setup
@@ -174,8 +174,12 @@ use IkiWiki::Setup::Standard {
 	# pages anonymous users can edit
 	#anonok_pagespec => "*",
 	
-	# For use with the aggregate plugin, to allow aggregation to be
-	# triggered via the web.
+	# For use with the aggregate plugin.
+	# Enable aggregation to internal pages. New wikis should set this to 1,
+	# but if you use aggregate already, read the aggregate plugin docs
+	# before enabling it.
+	#aggregateinternal => 1,
+	# Allow aggregation to be triggered via the web.
 	#aggregate_webtrigger => 1,
 	
 	# For use with the pinger plugin, how many seconds to wait before
diff --git a/doc/plugins/aggregate.mdwn b/doc/plugins/aggregate.mdwn
index 574c8b125..61743a816 100644
--- a/doc/plugins/aggregate.mdwn
+++ b/doc/plugins/aggregate.mdwn
@@ -1,10 +1,10 @@
-[[template id=plugin name=aggregate author="[[Joey]]"]]
-[[tag type/useful]]
+[[!template id=plugin name=aggregate author="[[Joey]]"]]
+[[!tag type/useful]]
 
 This plugin allows content from other feeds to be aggregated into the wiki.
-Aggregate a feed as follows
+Aggregate a feed as follows:
 
-	\[[aggregate name="example blog"
+	\[[!aggregate name="example blog" dir="example"
 	feedurl="http://example.com/index.rss"
 	url="http://example.com/" updateinterval="15"]]
 
@@ -13,12 +13,17 @@ more frequently than once every 15 minutes, and puts a page per post under
 the example/ directory in the wiki.
 
 You can then use ikiwiki's [[ikiwiki/blog]] support to create a blog of one or
-more aggregated feeds.
+more aggregated feeds. For example:
+
+	\[[!inline pages="internal(example/*)"]]
 
 ## setup
 
-Make sure that you have the [[html]] plugin enabled, as the created pages are
-in html format. The [[meta]] and [[tag]] plugins are also recommended. The
+New users of aggregate should enable the `aggregateinternal => 1` option in the
+.setup file. If you don't do so, you will need to enable the [[html]] plugin
+as well as aggregate itself, since feed entries will be stored as HTML.
+
+The [[meta]] and [[tag]] plugins are also recommended. The
 [[htmltidy]] plugin is suggested, since feeds can easily contain html
 problems, some of which tidy can fix.
 
@@ -31,7 +36,7 @@ crontab entry:
 Alternatively, you can allow `ikiwiki.cgi` to trigger the aggregation. You
 should only need this if for some reason you cannot use cron, and instead
 want to use a service such as [WebCron](http://webcron.org). To enable
-this, enable on `aggregate_webtrigger` in your setup file. The url to
+this, turn on `aggregate_webtrigger` in your setup file. The url to
 visit is `http://whatever/ikiwiki.cgi?do=aggregate_webtrigger`. Anyone
 can visit the url to trigger an aggregation run, but it will only check
 each feed if its `updateinterval` has passed.
@@ -59,9 +64,44 @@ directive:
 * `tag` - A tag to tag each post from the feed with. A good tag to use is
   the name of the feed. Can be repeated multiple times. The [[tag]] plugin
   must be enabled for this to work.
-* `template` - Template to use for creating the html pages. Defaults to
+* `template` - Template to use for creating the aggregated pages. Defaults to
   aggregatepost.
 
 Note that even if you are using subversion or another revision control
 system, pages created by aggregation will *not* be checked into revision
 control.
+
+## internal pages and `aggregateinternal`
+
+This plugin creates a page for each aggregated item. 
+
+If the `aggregateinternal` option is enabled in the setup file (which is
+recommended), aggregated pages are stored in the source directory with a
+"._aggregated" extension. These pages cannot be edited by web users, and
+do not generate first-class wiki pages. They can still be inlined into a
+blog, but you have to use `internal` in [[PageSpecs|IkiWiki/PageSpec]],
+like `internal(blog/*)`.
+
+For backward compatibility, the default is that these pages have the
+".html" extension, and are first-class wiki pages -- each one generates
+a separate HTML page in the output, and they can even be edited.
+
+That turns out to not be ideal for aggregated content, because publishing
+files for each of those pages is a waste of disk space and CPU, and you
+probably don't want to allow them to be edited. So, there is an alternative
+method that can be used (and is recommended), turned on by the
+`aggregateinternal` option in the setup file.
+
+If you are already using aggregate and want to enable `aggregateinternal`,
+you should follow this process:
+
+1. Update all [[PageSpecs|ikiwiki/PageSpec]] that refer to the aggregated
+   pages -- such as those in inlines. Put "internal()" around globs 
+   in those PageSpecs. For example, if the PageSpec was `foo/*`, it should
+   be changed to `internal(foo/*)`. This has to be done because internal
+   pages are not matched by regular globs.
+2. Use [[ikiwiki-transition]] to move all existing aggregated `.html`
+   files. The command to run is `ikiwiki-transition aggregateinternal $srcdir`,
+   or if you have changed the `htmlext` option to something other than "html",
+   `ikiwiki-transition aggregateinternal $srcdir $htmlext`
+3. Turn on `aggregateinternal` in the setup file and rebuild the wiki.
diff --git a/doc/roadmap.mdwn b/doc/roadmap.mdwn
index af46e162b..32752715d 100644
--- a/doc/roadmap.mdwn
+++ b/doc/roadmap.mdwn
@@ -42,7 +42,8 @@ backwards compatability.
 Still in the early planning stages, version 3.0 will be an opportunity to
 make significant transitions.
 
-* Default to using prefix_directives.
+* Default to using `prefix_directives`.
+* Default to using `aggregateinternal`.
 
 ----