Merge remote branch 'upstream/master' into prv/po

Conflicts:
	IkiWiki/Plugin/po.pm

15 files changed, 268 insertions, 21 deletions

diff --git a/doc/tips/Importing_posts_from_Wordpress.mdwn b/doc/tips/Importing_posts_from_Wordpress.mdwn
index 59330caa4..af8e4005d 100644
--- a/doc/tips/Importing_posts_from_Wordpress.mdwn
+++ b/doc/tips/Importing_posts_from_Wordpress.mdwn
@@ -1,9 +1,13 @@
 Use case: You want to move away from Wordpress to Ikiwiki as your blogging/website platform, but you want to retain your old posts.
 
-[This](http://git.chris-lamb.co.uk/?p=ikiwiki-wordpress-import.git) is a simple tool that generates [git-fast-import](http://www.kernel.org/pub/software/scm/git/docs/git-fast-import.html)-compatible data from a WordPress export XML file. It retains creation time of each post, so you can use Ikiwiki's <tt>--getctime</tt> to get the preserve creation times on checkout. 
+[This](http://git.chris-lamb.co.uk/?p=ikiwiki-wordpress-import.git) is a simple tool that generates [git-fast-import](http://www.kernel.org/pub/software/scm/git/docs/git-fast-import.html)-compatible data from a WordPress export XML file.
 
 WordPress categories are mapped onto Ikiwiki tags. The ability to import comments is planned.
 
+The script uses the [BeautifulSoup][] module.
+
+[BeautifulSoup]: http://www.crummy.com/software/BeautifulSoup/
+
 -----
 
 I include a modified version of this script. This version includes the ability to write \[[!tag foo]] directives, which the original intended, but didn't actually do.
diff --git a/doc/tips/add_chatterbox_to_blog.mdwn b/doc/tips/add_chatterbox_to_blog.mdwn
index aa35b9331..e07e36b07 100644
--- a/doc/tips/add_chatterbox_to_blog.mdwn
+++ b/doc/tips/add_chatterbox_to_blog.mdwn
@@ -18,4 +18,7 @@ from there, like I have on [my blog](http://kitenet.net/~joey/blog/)
 	show=5 feeds=no]]
 	"""]]
 
+* To filter out `@-replies`, append "and !*@*" to the [[ikiwiki/PageSpec]].
+  The same technique can be used for other filtering.
+
 Note: Works best with ikiwiki 3.10 or better.
diff --git a/doc/tips/comments_feed.mdwn b/doc/tips/comments_feed.mdwn
index 6f8137256..3d6a8c449 100644
--- a/doc/tips/comments_feed.mdwn
+++ b/doc/tips/comments_feed.mdwn
@@ -3,8 +3,15 @@ blog can have comments added to them. Pages with comments even have special
 feeds that can be used to subscribe to those comments. But you'd like to
 add a feed that contains all the comments posted to any page. Here's how:
 
-	\[[!inline pages="internal(*/comment_*)" template=comment]]
+	\[[!inline pages="comment(*)" template=comment]]
 
 The special [[ikiwiki/PageSpec]] matches all comments. The
-[[template|wikitemplates]] causes the comments to be displayed formatted
+[[template|templates]] causes the comments to be displayed formatted
 nicely.
+
+---
+
+It's also possible to make a feed of comments that are held pending
+moderation.
+
+	\[[!inline pages="comment_pending(*)" template=comment]]
diff --git a/doc/tips/convert_mediawiki_to_ikiwiki.mdwn b/doc/tips/convert_mediawiki_to_ikiwiki.mdwn
index 9719d9a7e..38de01109 100644
--- a/doc/tips/convert_mediawiki_to_ikiwiki.mdwn
+++ b/doc/tips/convert_mediawiki_to_ikiwiki.mdwn
@@ -48,10 +48,7 @@ in HTML, you may need to add further processing to the last line.
 
 Note that by default, `Special:Allpages` will only list pages in the main
 namespace. You need to add a `&namespace=XX` argument to get pages in a
-different namespace. The following numbers correspond to common namespaces:
-
- * 10 - templates (`Template:foo`)
- * 14 - categories (`Category:bar`)
+different namespace. (See below for the default list of namespaces)
 
 Note that the page names obtained this way will not include any namespace
 specific prefix: e.g. `Category:` will be stripped off.
@@ -59,7 +56,28 @@ specific prefix: e.g. `Category:` will be stripped off.
 ### Querying the database
 
 If you have access to the relational database in which your mediawiki data is
-stored, it is possible to derive a list of page names from this.
+stored, it is possible to derive a list of page names from this. With mediawiki's
+MySQL backend, the page table is, appropriately enough, called `table`:
+
+    SELECT page_namespace, page_title FROM page;
+
+As with the previous method, you will need to do some filtering based on the
+namespace.
+
+### namespaces
+
+The list of default namespaces in mediawiki is available from <http://www.mediawiki.org/wiki/Manual:Namespace#Built-in_namespaces>. Here are reproduced the ones you are most likely to encounter if you are running a small mediawiki install for your own purposes:
+
+[[!table data="""
+Index   | Name         | Example
+0       | Main         | Foo
+1       | Talk         | Talk:Foo
+2       | User         | User:Jon
+3       | User talk    | User_talk:Jon
+6       | File         | File:Barack_Obama_signature.svg
+10      | Template     | Template:Prettytable
+14      | Category     | Category:Pages_needing_review
+"""]]
 
 ## Step 2: fetching the page data
 
diff --git a/doc/tips/convert_mediawiki_to_ikiwiki/discussion.mdwn b/doc/tips/convert_mediawiki_to_ikiwiki/discussion.mdwn
index 683a3f8ac..d67a9131b 100644
--- a/doc/tips/convert_mediawiki_to_ikiwiki/discussion.mdwn
+++ b/doc/tips/convert_mediawiki_to_ikiwiki/discussion.mdwn
@@ -1,3 +1,11 @@
+20100428 - I just wrote a simple ruby script which will connect to a mysql server and then recreate the pages and their revision histories with Grit. It also does one simple conversion of equals titles to pounds. Enjoy! 
+
+<http://github.com/docunext/mediawiki2gitikiwiki>
+
+-- [[users/Albert]]
+
+----
+
 The u32 page is excellent, but I wonder if documenting the procedure here
 would be worthwhile. Who knows, the remote site might disappear. But also
 there are some variations on the approach that might be useful:
@@ -649,3 +657,9 @@ page.--[[users/Chadius]]
 > on mediawiki in general. It seems to produce output that could be
 > used by the [[plugins/contrib/mediawiki]] plugin, if the filenames
 > were fixed to use the right extension.  --[[Joey]] 
+
+>> Here's another I found while browsing around starting from the link you gave Joey<br />
+>> <http://github.com/scy/levitation><br />
+>> As I don't run mediawiki anymore, but I still have my xz/gzip-compressed XML dumps,
+>> it's certainly easier for me to do it this way; also a file or a set of files is easier to lug
+>> around on some medium than a full mysqld or postgres master and relevant databases.
diff --git a/doc/tips/follow_wikilinks_from_inside_vim.mdwn b/doc/tips/follow_wikilinks_from_inside_vim.mdwn
new file mode 100644
index 000000000..015a4ecee
--- /dev/null
+++ b/doc/tips/follow_wikilinks_from_inside_vim.mdwn
@@ -0,0 +1,47 @@
+The [ikiwiki-nav](http://www.vim.org/scripts/script.php?script_id=2968) plugin
+for vim eases the editing of IkiWiki wikis, by letting you "follow" the
+wikilinks on your file (page), by loading the file associated with a given
+wikilink in vim. The plugin takes care of following the ikiwiki linking rules
+to figure out which file a wikilink points to 
+
+The plugin also includes commands (and mappings) to make the cursor jump to the
+previous/next wikilink in the current file 
+
+##  Jumping to pages 
+
+To open the file associated to a wikilink, place the cursor over its text, and
+hit Enter (`<CR>`). This functionality is also available through the
+`:IkiJumpToPage` command 
+
+## Moving to next/previous wikilink in current file 
+
+`Ctrl-j` will move the cursor to the next wikilink. `Ctrl-k` will move it to the
+previous wikilink. This functionality is also available through the
+`:IkiNextWikiLink` command. This command takes one argument, the direction to
+move into 
+
+  * `:IkiNextWikiLink 0` will look forward for the wikilink 
+  * `:IkiNextWikiLink 1` will look backwards for the wikilink 
+
+## Installation
+
+Copy the `ikiwiki_nav.vim` file to your `.vim/ftplugin` directory.
+
+## Current issues: 
+
+  * The plugin only works for wikilinks contained in a single text line;
+    multiline wikilinks are not (yet) seen as such 
+
+## Notes 
+
+The official releases of the plugin are in the
+[vim.org script page](http://www.vim.org/scripts/script.php?script_id=2968)
+
+The latest version of this script can be found in the following location 
+
+<http://git.devnull.li/cgi-bin/gitweb.cgi?p=ikiwiki-nav.git;a=blob;f=ftplugin/ikiwiki_nav.vim;hb=HEAD>
+
+Any feedback you can provide is appreciated; the contact details can be found
+inside the plugin
+
+[[!tag vim]]
diff --git a/doc/tips/htaccess_file.mdwn b/doc/tips/htaccess_file.mdwn
new file mode 100644
index 000000000..6964cf24e
--- /dev/null
+++ b/doc/tips/htaccess_file.mdwn
@@ -0,0 +1,27 @@
+If you try to include a `.htaccess` file in your wiki's source, in order to
+configure the web server, you'll find that ikiwiki excludes it from
+processing. In fact, ikiwiki excludes any file starting with a dot, as well
+as a lot of other files, for good security reasons.
+
+You can tell ikiwiki not to exclude the .htaccess file by adding this to
+your setup file:
+
+	include => '^\.htaccess$',
+
+Caution! Before you do that, please think for a minute about who can edit
+your wiki. Are attachment uploads enabled? Can users commit changes
+directly to the version control system? Do you trust everyone who can
+make a change to not do Bad Things with the htaccess file? Do you trust
+everyone who *might* be able to make a change in the future? Note that a
+determined attacker who can write to the htaccess file can probably get a
+shell on your web server.
+
+If any of these questions have given you pause, I suggest you find a
+different way to configure the web server. One way is to not put the
+`.htaccess` file under ikiwiki's control, and just manually install it
+in the destdir. --[[Joey]]
+
+[Apache's documentation](http://httpd.apache.org/docs/2.2/howto/htaccess.html)
+says:
+> In general, you should never use .htaccess files unless you don't have
+> access to the main server configuration file.
diff --git a/doc/tips/html5.mdwn b/doc/tips/html5.mdwn
new file mode 100644
index 000000000..945efc4bc
--- /dev/null
+++ b/doc/tips/html5.mdwn
@@ -0,0 +1,26 @@
+First, if you just want to embed videos using the html5 `<video>` tag,
+you can do that without switching anything else to html5.
+However, if you want to fully enter the brave new world of html5, read on..
+
+Currently, ikiwiki does not use html5 by default. There is a `html5`
+setting that can be turned on, in your setup file. Rebuild with it set, and
+lots of fancy new semantic tags will be used all over the place.
+
+You may need to adapt your CSS for html5. While all the class and id names
+are the same, some of the `div` elements are changed to other things.
+Ikiwiki's default CSS will work in both modes.
+
+The html5 support is still experimental, and may break in some browsers.
+No care is taken to add backwards compatability hacks for browsers that
+are not html5 aware (like MSIE). If you want to include the javascript with
+those hacks, you can edit `page.tmpl` to do so. 
+[Dive Into HTML5](http://diveintohtml5.org/) is a good reference for
+current compatability issues and workarounds with html5.
+
+---
+
+Known ikiwiki-specific issues:
+
+* [[plugins/htmltidy]] uses `tidy`, which is not html5 aware, so if you
+  have that enabled, it will mangle it back to html4.
+* [[plugins/toc]] does not understand the html5 outline algorithm.
diff --git a/doc/tips/inside_dot_ikiwiki.mdwn b/doc/tips/inside_dot_ikiwiki.mdwn
index b81ffae8d..a74d00f47 100644
--- a/doc/tips/inside_dot_ikiwiki.mdwn
+++ b/doc/tips/inside_dot_ikiwiki.mdwn
@@ -6,9 +6,10 @@ you need/want to.
 
 ## the index
 
-`.ikiwiki/indexdb` contains a cache of information about pages, as well
-as all persisitant state about pages. It used to be a (semi) human-readable
-text file, but is not anymore.
+`.ikiwiki/indexdb` contains a cache of information about pages.
+This information can always be recalculated by rebuilding the wiki.
+(So the file is safe to delete and need not be backed up.)
+It used to be a (semi) human-readable text file, but is not anymore.
 
 To dump the contents of the file, enter a perl command like this.
 
diff --git a/doc/tips/inside_dot_ikiwiki/discussion.mdwn b/doc/tips/inside_dot_ikiwiki/discussion.mdwn
index 34d5b9252..69df369ec 100644
--- a/doc/tips/inside_dot_ikiwiki/discussion.mdwn
+++ b/doc/tips/inside_dot_ikiwiki/discussion.mdwn
@@ -6,14 +6,15 @@ My database appears corrupted:
 No idea how this happened.  I've blown it away and recreated it but, for future reference, is there any less violent way to recover from this situation?  I miss having the correct created and last edited times.  --[[sabr]]
 > update: fixed ctimes and mtimes using [these instructions](http://u32.net/Mediawiki_Conversion/Git_Import/#Correct%20Creation%20and%20Last%20Edited%20time) --[[sabr]]
 
-> That's overly complex. Just run `ikiwiki -setup your.setup -getctime`. 
+> That's overly complex. Just run `ikiwiki -setup your.setup -gettime`. 
 > BTW, I'd be interested in examining such a corrupt storable file to try
 > to see what happened to it. --[[Joey]]
 
->> --getctime appears to only set the last edited date.  It's not supposed to set the creation date, is it?  The only place that info is stored is in the git repo.
+>> --gettime appears to only set the last edited date.  It's not supposed to set the creation date, is it?  The only place that info is stored is in the git repo.
 
 >>> Pulling the page creation date out of the git history is exactly what
->>> --getctime does. --[[Joey]]
+>>> --gettime does. (It used to be called --getctime, and only do that; now
+>>> it also pulls out the last modified date). --[[Joey]]
 
 >> Alas, I seem to have lost the bad index file to periodic /tmp wiping; I'll send it to you if it happens again.  --[[sabr]]
 
diff --git a/doc/tips/laptop_wiki_with_git.mdwn b/doc/tips/laptop_wiki_with_git.mdwn
index 85e26c729..cfa565d1a 100644
--- a/doc/tips/laptop_wiki_with_git.mdwn
+++ b/doc/tips/laptop_wiki_with_git.mdwn
@@ -67,6 +67,5 @@ as many times as you want (thus being able to have a repository which is
 used by the ikiwiki CGI, and another which you can use for updating via
 git).
 
-When you want to update the server from the laptop, do a `git push` from
-within the laptop's bare repository, and likewise do a `git pull` to update
-the laptop from the server.
+Use standard git commands, run in the laptop's bare git repository
+to handle pulling from and pushing to the server.
diff --git a/doc/tips/spam_and_softwaresites.mdwn b/doc/tips/spam_and_softwaresites.mdwn
new file mode 100644
index 000000000..a07889e6b
--- /dev/null
+++ b/doc/tips/spam_and_softwaresites.mdwn
@@ -0,0 +1,87 @@
+Any wiki with a form of web-editing enabled will have to deal with
+spam. (See the [[plugins/blogspam]] plugin for one defensive tool you
+can deploy).
+
+If:
+
+ * you are using ikiwiki to manage the website for a [[examples/softwaresite]]
+ * you allow web-based commits, to let people correct documentation, or report
+   bugs, etc.
+ * the documentation is stored in the same revision control repository as your
+   software
+
+It is undesirable to have your software's VCS history tainted by spam and spam
+clean-up commits. Here is one approach you can use to prevent this. This
+example is for the [[git]] version control system, but the principles should
+apply to others.
+
+## Isolate web commits to a specific branch
+
+Create a separate branch to contain web-originated edits (named `doc` in this
+example):
+
+    $ git checkout -b doc
+
+Adjust your setup file accordingly:
+
+    gitmaster_branch => 'doc',
+
+## merging good web commits into the master branch
+
+You will want to periodically merge legitimate web-based commits back into
+your master branch. Ensure that there is no spam in the documentation
+branch. If there is, see 'erase spam from the commit history', below, first.
+
+Once you are confident it's clean:
+
+    # ensure you are on the master branch
+    $ git branch
+      doc
+    * master
+    $ git merge --ff doc
+
+## removing spam
+
+### short term
+
+In the short term, just revert the spammy commit.
+
+If the spammy commit was the top-most:
+
+    $ git revert HEAD
+
+This will clean the spam out of the files, but it will leave both the spam
+commit and the revert commit in the history.
+
+### erase spam from the commit history
+
+Git allows you to rewrite your commit history.  We will take advantage of this
+to eradicate spam from the history of the doc branch.
+
+This is a useful tool, but it is considered bad practise to rewrite the
+history of public repositories. If your software's repository is public, you
+should make it clear that the history of the `doc` branch in your repository
+is unstable.
+
+Once you have been spammed, use `git rebase` to remove the spam commits from
+the history.  Assuming that your `doc` branch was split off from a branch
+called `master`:
+
+    # ensure you are on the doc branch
+    $ git branch
+    * doc
+      master
+    $ git rebase --interactive master
+
+In your editor session, you will see a series of lines for each commit made to
+the `doc` branch since it was branched from `master` (or since the last merge
+back into `master`). Delete the lines corresponding to spammy commits, then
+save and exit your editor.
+
+Caveat: if there are no commits you want to keep (i.e. all the commits since
+the last merge into master are either spam or spam reverts) then `git rebase`
+will abort. Therefore, this approach only works if you have at least one
+non-spam commit to the documentation since the last merge into `master`. For
+this reason, it's best to wait until you have at least one
+commit you want merged back into the main history before doing a rebase,
+and until then, tackle spam with reverts.
diff --git a/doc/tips/switching_to_usedirs.mdwn b/doc/tips/switching_to_usedirs.mdwn
index 183ce00ac..92871439f 100644
--- a/doc/tips/switching_to_usedirs.mdwn
+++ b/doc/tips/switching_to_usedirs.mdwn
@@ -8,9 +8,7 @@ to usedirs, or edit your setup file and turn usedirs back off.
   or manually.
 * Since usedirs is enabled, ikiwiki will have created a bunch of new
   html files. Where before ikiwiki generated a `dest/foo.html`, now it will
-  generate `dest/foo/index.html`. But, the old html files will still be
-  present too. Remove them:
-	find dest -name \*.html -not -name index.html -exec rm {} \;
+  generate `dest/foo/index.html`. The old html files will be removed.
 * If you have a blog that is aggregated on a Planet or similar, all the
   items in the RSS or atom feed will seem like new posts, since their URLs
   have changed. See [[howto_avoid_flooding_aggregators]] for a workaround.
diff --git a/doc/tips/vim_syntax_highlighting.mdwn b/doc/tips/vim_syntax_highlighting.mdwn
index 172b763c3..18b2f1c92 100644
--- a/doc/tips/vim_syntax_highlighting.mdwn
+++ b/doc/tips/vim_syntax_highlighting.mdwn
@@ -1,4 +1,7 @@
 [[ikiwiki.vim]] is a vim syntax highlighting file for ikiwiki
-[[ikiwiki/markdown]] files. 
+[[ikiwiki/markdown]] files.
 
 Installation instructions are at the top of the file.
+
+See also: [[follow_wikilinks_from_inside_vim]]
+[[!tag vim]]
diff --git a/doc/tips/yaml_setup_files.mdwn b/doc/tips/yaml_setup_files.mdwn
new file mode 100644
index 000000000..e8ab4f144
--- /dev/null
+++ b/doc/tips/yaml_setup_files.mdwn
@@ -0,0 +1,12 @@
+Here's how to convert your existing standard format ikiwiki setup file into
+the new YAML format recently added to ikiwiki.
+
+1. First, make sure you have the [[!cpan YAML]] perl module installed.
+   (Run: `apt-get install libyaml-perl`)
+2. Run: `ikiwiki -setup my.setup -dumpsetup my.setup --set setuptype=Yaml`
+
+The format of the YAML setup file should be fairly self-explanatory.
+
+(To convert the other way, use "setuptype=Standard" instead.)
+
+--[[Joey]]