From cd5bf7eb7f74c2414a87c77141ed0c502ff7f464 Mon Sep 17 00:00:00 2001 From: "http://smcv.pseudorandom.co.uk/" Date: Thu, 15 Oct 2009 23:16:52 -0400 Subject: comments after trying to implement joey's idea --- doc/plugins/contrib/album/discussion.mdwn | 134 ++++++++++++++++++++++++++---- 1 file changed, 119 insertions(+), 15 deletions(-) diff --git a/doc/plugins/contrib/album/discussion.mdwn b/doc/plugins/contrib/album/discussion.mdwn index 5c8e74fa6..156cd7ad8 100644 --- a/doc/plugins/contrib/album/discussion.mdwn +++ b/doc/plugins/contrib/album/discussion.mdwn @@ -68,6 +68,9 @@ code or tried it yet, but here goes. --[[Joey]] > you (since the requirements for that CGI interface change depending > on the implementation). I agree that this is ugly, though. -s +>> Would you accept a version where the albumimage "viewer" pages +>> could be 0 bytes long, at least until metadata gets added? -s + * With each viewer page having next/prev links, I can see how you were having the scalability issues with ikiwiki's data structures earlier! @@ -80,7 +83,7 @@ code or tried it yet, but here goes. --[[Joey]] >> these can be presence dependencies, which will probably help with >> avoiding rebuilds of a page if the next/prev page is changed. >> (Unless you use img to make the thumbnails for those links, then it ->> would rebuild the thumbnails anyway. Have not looked at the code.) --[[Joey]] +>> would rebuild the thumbnails anyway. Have not looked at the code.) --[[Joey]] * And doesn't each viewer page really depend on every other page in the same albumsection? If a new page is added, the next/prev links @@ -108,6 +111,11 @@ code or tried it yet, but here goes. --[[Joey]] >> metadata. Er, I mean, I have a cheezy hack in `add_depends` now that does >> it to deal with a similar case. --[[Joey]] +>>> I think I was misunderstanding how early you have to call `add_depends`? +>>> The critical thing I missed was that if you're scanning a page, you're +>>> going to rebuild it in a moment anyway, so it doesn't matter if you +>>> have no idea what it depends on until the rebuild phase. -s + * One thing I do like about having individual pages per image is that they can each have their own comments, etc. @@ -121,9 +129,25 @@ code or tried it yet, but here goes. --[[Joey]] album. Think tags. So it seems it would be better to have the album directive control what pages it includes (a la inline). -> See note above about pagespecs not being very safe early on. -> You did merge my inline-with-pagenames feature, which is safe to use -> at scan time, though. +> I'm inclined to fix this by constraining images to be subpages of exactly +> one album: if they're subpages of 2+ nested albums then they're only +> considered to be in the deepest-nested one (i.e. longest URL), and if +> they're not in any album then that's a usage error. This would +> also make prev/next links sane. +> +> If you want to reference images from elsewhere in the wiki and display +> them as if in an album, then you can use an ordinary inline with +> the same template that the album would use, and I'll make sure the +> templates are set up so this works. +> +> (Implementation detail: this means that an image X/Y/Z/W/V, where X and +> Y are albums, Z does not exist and W exists but is not an album, +> would have a content dependency on Y, a presence dependency on Z +> and a content dependency on W.) +> +> Perhaps I should just restrict to having the album images be direct +> subpages of the album, although that would mean breaking some URLs +> on the existing website I'm doing all this work for... -s * Putting a few of the above thoughts together, my ideal album system seems to be one where I can just drop the images into a directory and @@ -137,15 +161,57 @@ code or tried it yet, but here goes. --[[Joey]] > Putting a JPEG in the web form is not an option from my point of > view :-) but perhaps there could just be a "web-editable" flag supplied > by plugins, and things could be changed to respect it. -> + +>> Replying to myself: would you accept patches to support +>> `hook(type => 'htmlize', editable => 0, ...)` in editpage? This would +>> essentially mean "this is an opaque binary: you can delete it +>> or rename it, and it might have its own special editing UI, but you +>> can never get it in a web form". +>> +>> On the other hand, that essentially means we need to reimplement +>> editpage in order to edit the sidecar files that contain the metadata. +>> Having already done one partial reimplementation of editpage (for +>> comments) I'm in no hurry to do another. +>> +>> I suppose another possibility would be to register hook +>> functions to be called by editpage when it loads and saves the +>> file. In this case, the loading hook would be to discard +>> the binary and use filter() instead, and the saving conversion +>> would be to write the edited content into the metadata sidecar +>> (creating it if necessary). +>> +>> I'd also need to make editpage (and also comments!) not allow the +>> creation of a file of type albumjpg, albumgif etc., which is something +>> I previously missed; and I'd need to make attachment able to +>> upload-and-rename. +>> -s + > In a way, what you really want for metadata is to have it in the album > page, so you can batch-edit the whole lot by editing one file (this > does mean that editing the album necessarily causes each of its viewers > to be rebuilt, but in practice that happens anyway). -s -> ->> Yes, that would make some sense.. It also allows putting one image in ->> two albums, with different caption etc. (Maybe for different audiences.) + +>> Replying to myself: in practice that *doesn't* happen anyway. Having +>> the metadata in the album page is somewhat harmful because it means +>> that changing the title of one image causes every viewer in the album +>> to be rebuilt, whereas if you have a metadata file per image, only +>> the album itself, plus the next and previous viewers, need +>> rebuilding. So, I think a file per image is the way to go. >> +>> Ideally we'd have some way to "batch-edit" the metadata of all +>> images in an album at once, except that would make conflict +>> resolution much more complicated to deal with; maybe just +>> give up and scream about mid-air collisions in that case? +>> (That's apparently good enough for Bugzilla, but not really +>> for ikiwiki). -s + +>> Yes, [all metadata in one file] would make some sense.. It also allows putting one image in +>> two albums, with different caption etc. (Maybe for different audiences.) +>> --[[Joey]] + +>>> Eek. No, that's not what I had in mind at all; the metadata ends up +>>> in the "viewer" page, so it's necessarily the same for all albums. -s + >> It would probably be possible to add a new dependency type, and thus >> make ikiwiki smart about noticing whether the metadata has actually >> changed, and only update those viewers where it has. But the dependency @@ -164,23 +230,26 @@ mushroom and snake. > etc as the htmlize extensions. May need some fixes to ikiwiki to support > that. --[[Joey]] +>> foo.albumjpg (etc.) for images, and foo._albummeta (with +>> `keepextension => 1`) for sidecar metadata files, seems viable. -s + Files in git repo: * index.mdwn * memes.mdwn -* memes/badger.albumimage (a renamed JPEG) +* memes/badger.albumjpg (a renamed JPEG) * memes/badger/comment_1._comment * memes/badger/comment_2._comment -* memes/mushroom.albumimage (a renamed GIF) -* memes/mushroom.meta (sidecar file with metadata) -* memes/snake.albumimage (a renamed video) +* memes/mushroom.albumgif (a renamed GIF) +* memes/mushroom._albummeta (sidecar file with metadata) +* memes/snake.albummov (a renamed video) Files in web content: * index.html * memes/index.html * memes/96x96-badger.jpg (from img) -* memes/96x96-mushroom.jpg (from img) +* memes/96x96-mushroom.gif (from img) * memes/96x96-snake.jpg (from img, hacked up to use totem-video-thumbnailer :-) ) * memes/badger/index.html (including comments) * memes/badger.jpg @@ -200,10 +269,28 @@ way to get them rendered anyway. > the image, as well as eg, smiley trying to munge it in sanitize. > --[[Joey]] +>> As long as nothing has a filter() hook that assumes it's already +>> text... filters are run in arbitrary order. We seem to be OK so far +>> though. +>> +>> If this is the route I take, I propose to have the result of filter() +>> be the contents of the sidecar metadata file (empty string if none), +>> with the `\[[!albumimage]]` directive (which no longer requires +>> arguments) prepended if not already present. This would mean that +>> meta directives in the metadata file would work as normal, and it +>> would be possible to insert text both before and after the viewer +>> if desired. The result of filter() would also be a sensible starting +>> point for editing, and the result of editing could be diverted into +>> the metadata file. -s + do=edit&page=memes/badger needs to not put the JPG in a text box: somehow divert or override the normal edit CGI by telling it that .albumimage files are not editable in the usual way? +> Something I missed here is that editpage also needs to be told that +> creating new files of type albumjpg, albumgif etc. is not allowed +> either! -s + Every image needs to depend on, and link to, the next and previous images, which is a bit tricky. In previous thinking about this I'd been applying the overly strict constraint that the ordered sequence of pages in each @@ -217,6 +304,9 @@ in order. > memoization to avoid each image in an album building the same list. > I sense that I may be missing a subtelty though. --[[Joey]] +>> I think I was misunderstanding how early you have to call `add_depends` +>> as mentioned above. -s + Perhaps restricting to "the images in an album A must match A/*" would be useful; then the unordered superset could just be "A/*". Your "albums via tags" idea would be nice too though, particularly for feature @@ -233,6 +323,9 @@ album, or something? > Ugh, yeah, that is a problem. Perhaps wanting to support that was just > too ambitious. --[[Joey]] +>> I propose to restrict to having images be subpages of albums, as +>> described above. -s + Requiring renaming is awkward for non-technical Windows/Mac users, with both platforms' defaults being to hide extensions; however, this could be circumvented by adding some sort of hook in attachment to turn things into @@ -244,13 +337,24 @@ extensions visible is a "don't do that then" situation :-) > with an extension. (Or allow specifying a full pagespec, > but I hesitate to seriously suggest that.) --[[Joey]] +>> I think that might be a terrifying idea for another day. If we can +>> mutate the extension during the `attach` upload, that'd be enough; +>> I don't think people who are skilled enough to use git/svn/..., +>> but not skilled enough to tell Explorer to show file extensions, +>> represent a major use case. -s + Ideally attachment could also be configured to upload into a specified underlay, so that photos don't have to be in your source-code control (you might want that, but I don't!). +> Replying to myself: perhaps best done as an orthogonal extension +> to attach? -s + Things that would be nice, and are probably possible: * make the "Edit page" link on viewers divert to album-specific CGI instead - of just failing or not appearing + of just failing or not appearing (probably possible via pagetemplate) + * some way to deep-link to memes/badger.jpg with a wikilink, without knowing a - priori that it's secretly a JPEG + priori that it's secretly a JPEG (probably harder than it looks - you'd + have to make a directive for it and it's probably not worth it) -- cgit v1.2.3 From 969ce8c5f889f8f2696c3ed4995f021b2a6539e1 Mon Sep 17 00:00:00 2001 From: "http://smcv.pseudorandom.co.uk/" Date: Thu, 15 Oct 2009 23:22:18 -0400 Subject: add a bit more attribution so it's clearer what Joey wrote --- doc/plugins/contrib/album/discussion.mdwn | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/doc/plugins/contrib/album/discussion.mdwn b/doc/plugins/contrib/album/discussion.mdwn index 156cd7ad8..50d6c8ddd 100644 --- a/doc/plugins/contrib/album/discussion.mdwn +++ b/doc/plugins/contrib/album/discussion.mdwn @@ -60,7 +60,7 @@ code or tried it yet, but here goes. --[[Joey]] * Needing to create the albumimage "viewer" pages for each photo seems like it will become a pain. Everyone will need to come up with their own automation for it, and then there's the question - of how to automate it when uploading attachments. + of how to automate it when uploading attachments. -J > There's already a script (ikiwiki-album) to populate a git > checkout with skeleton "viewer" pages; I was planning to make a @@ -73,7 +73,7 @@ code or tried it yet, but here goes. --[[Joey]] * With each viewer page having next/prev links, I can see how you were having the scalability issues with ikiwiki's data structures - earlier! + earlier! -J > Yeah, I think they're a basic requirement from a UI point of view > though (although they don't necessarily have to be full wikilinks). @@ -88,7 +88,7 @@ code or tried it yet, but here goes. --[[Joey]] * And doesn't each viewer page really depend on every other page in the same albumsection? If a new page is added, the next/prev links may need to be updated, for example. If so, there will be much - unnecessary rebuilding. + unnecessary rebuilding. -J > albumsections are just a way to insert headings into the flow of > photos, so they don't actually affect dependencies. @@ -117,7 +117,7 @@ code or tried it yet, but here goes. --[[Joey]] >>> have no idea what it depends on until the rebuild phase. -s * One thing I do like about having individual pages per image is - that they can each have their own comments, etc. + that they can each have their own comments, etc. -J > Yes; also, they can be wikilinked. I consider those to be > UI requirements. -s @@ -127,7 +127,7 @@ code or tried it yet, but here goes. --[[Joey]] album, but then anyone who can write to any other page on the wiki can add an image to it. 2: I may want an image to appear in more than one album. Think tags. So it seems it would be better to have the album - directive control what pages it includes (a la inline). + directive control what pages it includes (a la inline). -J > I'm inclined to fix this by constraining images to be subpages of exactly > one album: if they're subpages of 2+ nested albums then they're only @@ -156,7 +156,7 @@ code or tried it yet, but here goes. --[[Joey]] etc. (Real pity we can't just put arbitrary metadata into the images themselves.) This is almost pointing toward making the images first-class wiki page sources. Hey, it worked for po! :) But the metadata and editing - problems probably don't really allow that. + problems probably don't really allow that. -J > Putting a JPEG in the web form is not an option from my point of > view :-) but perhaps there could just be a "web-editable" flag supplied -- cgit v1.2.3 From bc4b8e4e239e56fedcc3cf065f7c6cbd224f7525 Mon Sep 17 00:00:00 2001 From: "http://smcv.pseudorandom.co.uk/" Date: Thu, 15 Oct 2009 23:27:53 -0400 Subject: not another hidden requirement... --- doc/plugins/contrib/album/discussion.mdwn | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/doc/plugins/contrib/album/discussion.mdwn b/doc/plugins/contrib/album/discussion.mdwn index 50d6c8ddd..9720589b4 100644 --- a/doc/plugins/contrib/album/discussion.mdwn +++ b/doc/plugins/contrib/album/discussion.mdwn @@ -69,7 +69,16 @@ code or tried it yet, but here goes. --[[Joey]] > on the implementation). I agree that this is ugly, though. -s >> Would you accept a version where the albumimage "viewer" pages ->> could be 0 bytes long, at least until metadata gets added? -s +>> could be 0 bytes long, at least until metadata gets added? +>> +>> The more I think about the "binaries as first-class pages" approach, +>> the more subtle interactions I notice with other plugins. I +>> think I'm up to needing changes to editpage, comments, attachment +>> and recentchanges, plus adjustments to img and Render (to reduce +>> duplication when thumbnailing an image with a strange extension +>> while simultaneously changing the extension, and to hardlink/copy +>> an image with a strange extension to a differing target filename +>> with the normal extension, respectively). -s * With each viewer page having next/prev links, I can see how you were having the scalability issues with ikiwiki's data structures @@ -350,6 +359,10 @@ underlay, so that photos don't have to be in your source-code control > Replying to myself: perhaps best done as an orthogonal extension > to attach? -s +> Yet another non-obvious thing this design would need to do is to find +> some way to have each change to memes/badger._albummeta show up as a +> change to memes/badger in `recentchanges`. -s + Things that would be nice, and are probably possible: * make the "Edit page" link on viewers divert to album-specific CGI instead -- cgit v1.2.3 From e59ba3a11326bbce1c31b0e86ebb28980cbcced3 Mon Sep 17 00:00:00 2001 From: tschwinge Date: Fri, 16 Oct 2009 03:14:03 -0400 Subject: minor: tiny rendering error --- doc/bugs/minor:_tiny_rendering_error.mdwn | 1 + 1 file changed, 1 insertion(+) create mode 100644 doc/bugs/minor:_tiny_rendering_error.mdwn diff --git a/doc/bugs/minor:_tiny_rendering_error.mdwn b/doc/bugs/minor:_tiny_rendering_error.mdwn new file mode 100644 index 000000000..5131e61fa --- /dev/null +++ b/doc/bugs/minor:_tiny_rendering_error.mdwn @@ -0,0 +1 @@ +`\[[!inline]]` is rendered with a space in front of the first closing bracket. --[[tschwinge]] -- cgit v1.2.3 From 002b6d2c41300052a056f4f95f417d5d0667f5ce Mon Sep 17 00:00:00 2001 From: tschwinge Date: Fri, 16 Oct 2009 03:19:55 -0400 Subject: shortcuts: local file. --- doc/plugins/shortcut/discussion.mdwn | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/doc/plugins/shortcut/discussion.mdwn b/doc/plugins/shortcut/discussion.mdwn index 4e11ce08c..2e2b1b281 100644 --- a/doc/plugins/shortcut/discussion.mdwn +++ b/doc/plugins/shortcut/discussion.mdwn @@ -9,4 +9,10 @@ Maybe use the `default_pageext` is better than hardcode .mdwn? > done, it will use `default_pageext` now --[[Joey]] +--- +Instead of modifying the [[basewiki]]'s [[shortcuts]] file for local needs -- +thus copying it at some point and losing continuity with upstream enhancements -- +what about handling a `shortcuts-local.mdwn` or `shortcuts/local.mdwn` (if such +a file exists in the wiki), and additionally process that one. Possibily a +conditional `\[[!inline]]` could be used. --[[tschwinge]] -- cgit v1.2.3 From 6823a51f42a268ba124b287208af7659704ec7d5 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Fri, 16 Oct 2009 10:53:05 +0200 Subject: Handling non-existing pages in parentlinks. --- doc/bugs/non-existing_pages_in_parentlinks.mdwn | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 doc/bugs/non-existing_pages_in_parentlinks.mdwn diff --git a/doc/bugs/non-existing_pages_in_parentlinks.mdwn b/doc/bugs/non-existing_pages_in_parentlinks.mdwn new file mode 100644 index 000000000..532b0acca --- /dev/null +++ b/doc/bugs/non-existing_pages_in_parentlinks.mdwn @@ -0,0 +1,5 @@ + does not exist. +Then, on +, in the +*parentlinks* line, *writing* links to the top-level *index* file. It should +rather not link anywhere at all. -- cgit v1.2.3 From 3e2f99cba10e2bf9194a5788b4174844aeb566e3 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Fri, 16 Oct 2009 11:01:22 +0200 Subject: A list of stuff that would be nice to work on for enhancing the GNU Hurd web pages. --- doc/users/tschwinge.mdwn | 61 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/doc/users/tschwinge.mdwn b/doc/users/tschwinge.mdwn index bb5cef6a6..5953cf802 100644 --- a/doc/users/tschwinge.mdwn +++ b/doc/users/tschwinge.mdwn @@ -9,3 +9,64 @@ web pages and previous wiki pages to a *[[ikiwiki]]* system; and all that while preserving the previous content's history, which was stored in a CVS repository for the HTML web pages and a TWiki RCS repository for the wiki; see . + +# Issues to Work On + +## Stability of Separate Builds + +The goal is that separate builds of the same source files should yield the +exactly same HTML code (of course, except for changes due to differences in +Markdown rendering, for example). + + * Timestamps -- [[forum/ikiwiki__39__s_notion_of_time]], [[forum/How_does_ikiwiki_remember_times__63__]] + + Git set's the current *mtime* when checking out files. The result is that + and + show different *Last + edited* timestamps. + + This can either be solved by adding a facility to Git to set the + checked-out files' *mtime* according to the *AuthorDate* / *CommitDate* + (which one...), or doing that retroactively with the + script before building, or + with a ikiwiki-internal solution. + + * HTML character entities + + + +## Tags -- [[bugs/tagged__40____41___matching_wikilinks]] + +Tags should be a separate concept from wikilinks. + +### \[[!map]] behavior + +The \[[!map]] on, for example, +, should not show +the complete hierarchy of pages, but instead just the pages that actually *do* +contain the \[[!tag open_issue_hurd]]. + +## Anchors -- [[ikiwiki/wikilink/discussion]] + +## Default Content for Meta Values -- [[plugins/contrib/default_content_for___42__copyright__42___and___42__license__42__]] + +This will decrease to be relevant, as we're going to add copyright and +licensing headers to every single file. + +## Texinfo -- [[plugins/contrib/texinfo]] + +Not very important. + +## Shortcuts -- [[plugins/shortcut/discussion]] + +## \[[!meta redir]] -- [[todo/__42__forward__42__ing_functionality_for_the_meta_plugin]] + +Implement a checker that makes sure that no pages that use \[[!meta redir]] +redirect to another page (and are thus considered legacy pages for providing +stable URLs, for example) are linked to from other wiki pages. This is useful +w.r.t. backlinks. Alternative, the backlinks to the \[[!meta redir]]-using +pages could perhaps be passed on to the referred-to page? + +## Sendmail -- [[todo/passwordauth:_sendmail_interface]] + +## Parentlinks -- [[bugs/non-existing_pages_in_parentlinks]] -- cgit v1.2.3 From 7d76e36edd27663ae7fca653f36a08a63cb36c50 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Fri, 16 Oct 2009 11:05:51 +0200 Subject: Issue with [[!meta date]] / [[!meta updated]] vs. RSS / Atom feeds? --- doc/users/tschwinge.mdwn | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/doc/users/tschwinge.mdwn b/doc/users/tschwinge.mdwn index 5953cf802..0fe09696b 100644 --- a/doc/users/tschwinge.mdwn +++ b/doc/users/tschwinge.mdwn @@ -70,3 +70,9 @@ pages could perhaps be passed on to the referred-to page? ## Sendmail -- [[todo/passwordauth:_sendmail_interface]] ## Parentlinks -- [[bugs/non-existing_pages_in_parentlinks]] + +## Unverified -- these may be bugs, but have yet to be verified + + * ikiwiki doesn't change its internal database when \[[!meta date]] / + \[[!meta updated]] are added / removed, and thusly these meta values are + not promulgated in RSS / Atom feeds. -- cgit v1.2.3 From b94d9589d39dfa720231fb982340741a139478cf Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Fri, 16 Oct 2009 11:11:59 +0200 Subject: Issue with ``no text was copied in this page'' in RSS feeds? --- doc/users/tschwinge.mdwn | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/doc/users/tschwinge.mdwn b/doc/users/tschwinge.mdwn index 0fe09696b..306338ae4 100644 --- a/doc/users/tschwinge.mdwn +++ b/doc/users/tschwinge.mdwn @@ -76,3 +76,8 @@ pages could perhaps be passed on to the referred-to page? * ikiwiki doesn't change its internal database when \[[!meta date]] / \[[!meta updated]] are added / removed, and thusly these meta values are not promulgated in RSS / Atom feeds. + + * Complicated issue w.r.t. *no text was copied in this page* + ([[plugins/cutpaste]]) in RSS feed (only; not Atom?) under some conditions + (refresh only, but not rebuild?). Perhaps missing to read in / parse some + files? -- cgit v1.2.3 From ab68f96494409e4ce8f689b5c27ef9ea3a73172c Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Fri, 16 Oct 2009 11:12:28 +0200 Subject: Discussion pages of Discussion pages (etc.)? --- doc/users/tschwinge.mdwn | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/doc/users/tschwinge.mdwn b/doc/users/tschwinge.mdwn index 306338ae4..657de85f3 100644 --- a/doc/users/tschwinge.mdwn +++ b/doc/users/tschwinge.mdwn @@ -71,6 +71,12 @@ pages could perhaps be passed on to the referred-to page? ## Parentlinks -- [[bugs/non-existing_pages_in_parentlinks]] +## Discussion Pages of Discussion Pages of... + +Is it useful to have Discussion pages of Discussion pages (etc.)? -- On +, +this possibility is offered. + ## Unverified -- these may be bugs, but have yet to be verified * ikiwiki doesn't change its internal database when \[[!meta date]] / -- cgit v1.2.3 From 07b28cfcf98c40d1c45276b18d09aab3fa491103 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Fri, 16 Oct 2009 11:24:21 +0200 Subject: TODO: How to preview changes before git://... commit. --- doc/todo/preview_changes_before_git_commit.mdwn | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 doc/todo/preview_changes_before_git_commit.mdwn diff --git a/doc/todo/preview_changes_before_git_commit.mdwn b/doc/todo/preview_changes_before_git_commit.mdwn new file mode 100644 index 000000000..e0e6ba866 --- /dev/null +++ b/doc/todo/preview_changes_before_git_commit.mdwn @@ -0,0 +1,7 @@ +ikiwiki allows to commit changes to the doc wiki over the `git://...` protocol. +It would be nice if there'd be a uniform way to view these changes before `git +push`ing. For the GNU Hurd's web pages, we include a *render_locally* script, +, with instructions on +, section +*Preview Changes*. With ikiwiki, one can use `make docwiki`, but that excludes +a set of pages, as per `docwiki.setup`. --[[tschwinge]] -- cgit v1.2.3 From a91bfd55069a0131d46937a8ef9eedc38e8d6de9 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Fri, 16 Oct 2009 11:25:23 +0200 Subject: Sign. --- doc/bugs/non-existing_pages_in_parentlinks.mdwn | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/bugs/non-existing_pages_in_parentlinks.mdwn b/doc/bugs/non-existing_pages_in_parentlinks.mdwn index 532b0acca..b4ea42b17 100644 --- a/doc/bugs/non-existing_pages_in_parentlinks.mdwn +++ b/doc/bugs/non-existing_pages_in_parentlinks.mdwn @@ -2,4 +2,4 @@ Then, on , in the *parentlinks* line, *writing* links to the top-level *index* file. It should -rather not link anywhere at all. +rather not link anywhere at all. --[[tschwinge]] -- cgit v1.2.3 From 027d28a6a1795b5b1b81aa381793f225682cc0e5 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Fri, 16 Oct 2009 11:28:06 +0200 Subject: Link to the forum from the top-level page. --- doc/index.mdwn | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/index.mdwn b/doc/index.mdwn index 93526c42c..732cf7a89 100644 --- a/doc/index.mdwn +++ b/doc/index.mdwn @@ -20,6 +20,7 @@ ikiwiki [[!version ]]. ## developer resources The [[RoadMap]] describes where the project is going. +The [[forum]] is open for discussions. [[Bugs]], [[TODO]] items, [[wishlist]] items, and [[patches|patch]] can be submitted and tracked using this wiki. -- cgit v1.2.3 From 3cf62292e11f0a8809f9f53f1be77acd80d0fb43 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Fri, 16 Oct 2009 11:33:42 +0200 Subject: Forum: discussion about the ever-growing list of pages. --- doc/forum/ever-growing_list_of_pages.mdwn | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 doc/forum/ever-growing_list_of_pages.mdwn diff --git a/doc/forum/ever-growing_list_of_pages.mdwn b/doc/forum/ever-growing_list_of_pages.mdwn new file mode 100644 index 000000000..84b9fe6ee --- /dev/null +++ b/doc/forum/ever-growing_list_of_pages.mdwn @@ -0,0 +1,12 @@ +What is overyone's idea about the ever-growing list of pages in bugs/ etc.? +Once linked to `done`, they're removed from the rendered [[bugs]] page -- but +they're still present in the repository. + +Shouldn't there be some clean-up at some point for those that have been +resolved? Or should all of them be kept online forever? + +Likewise, for example in [[forum/ikiwiki__39__s_notion_of_time]], should one +remove the text about the implementation bug that has been fixed, or should it +stay there, for reference? + +--[[tschwinge]] -- cgit v1.2.3 From 9da02428d4f62f84bdadc15f6aadf553a9d16c58 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Fri, 16 Oct 2009 11:40:35 +0200 Subject: Appetizer for the inline plugin? --- doc/users/tschwinge.mdwn | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/doc/users/tschwinge.mdwn b/doc/users/tschwinge.mdwn index 657de85f3..54614e922 100644 --- a/doc/users/tschwinge.mdwn +++ b/doc/users/tschwinge.mdwn @@ -77,6 +77,16 @@ Is it useful to have Discussion pages of Discussion pages (etc.)? -- On , this possibility is offered. +## Modifying [[plugins/inline]] for showing only an *appetizer* + +Currently ikiwiki's inline plugin will either show the full page or nothing of +it. Often that's too much. One can manually use the [[plugins/toggle]] plugin +-- see the *News* section on . Adding a new +mode to the inline plugin to only show an *appetizer* ending with *... (read +on)* after a customizable amount of characters (or lines) would be a another +possibility. The *... (read on)* would then either toggle the full content +being displayed or link to the complete page. + ## Unverified -- these may be bugs, but have yet to be verified * ikiwiki doesn't change its internal database when \[[!meta date]] / -- cgit v1.2.3 From 0a2e4e167dc0a6b9fc0b038c4174694117b74628 Mon Sep 17 00:00:00 2001 From: Jon Dowland Date: Fri, 16 Oct 2009 10:44:14 +0100 Subject: substantially expand the mediawiki tip with some of the steps. More to come --- doc/tips/convert_mediawiki_to_ikiwiki.mdwn | 101 +++++++++++++++++++++++++++-- 1 file changed, 97 insertions(+), 4 deletions(-) diff --git a/doc/tips/convert_mediawiki_to_ikiwiki.mdwn b/doc/tips/convert_mediawiki_to_ikiwiki.mdwn index f03703b46..c522eaec3 100644 --- a/doc/tips/convert_mediawiki_to_ikiwiki.mdwn +++ b/doc/tips/convert_mediawiki_to_ikiwiki.mdwn @@ -1,4 +1,97 @@ -[[sabr]] explains how to [import MediaWiki content into -git](http://u32.net/Mediawiki_Conversion/index.html?updated), including -full edit hostory. The [[plugins/contrib/mediawiki]] plugin can then be -used by ikiwiki to build the wiki. +Mediawiki is a dynamically-generated wiki which stores it's data in a +relational database. Pages are marked up using a proprietary markup. It is +possible to import the contents of a Mediawiki site into an ikiwiki, +converting some of the Mediawiki conventions into Ikiwiki ones. + +The following instructions describe ways of obtaining the current version of +the wiki. We do not yet cover importing the history of edits. + +## Step 1: Getting a list of pages + +The first bit of information you require is a list of pages in the Mediawiki. +There are several different ways of obtaining these. + +### Parsing the output of `Special:Allpages` + +Mediawikis have a special page called `Special:Allpages` which list all the +pages for a given namespace on the wiki. + +If you fetch the output of this page to a local file with something like + + wget -q -O tmpfile 'http://your-mediawiki/wiki/Special:Allpages' + +You can extract the list of page names using the following python script. Note +that this script is sensitive to the specific markup used on the page, so if +you have tweaked your mediawiki theme a lot from the original, you will need +to adjust this script too: + + from xml.dom.minidom import parse, parseString + + dom = parse(argv[1]) + tables = dom.getElementsByTagName("table") + pagetable = tables[-1] + anchors = pagetable.getElementsByTagName("a") + for a in anchors: + print a.firstChild.toxml().\ + replace('&,'&').\ + replace('<','<').\ + replace('>','>') + +Also, if you have pages with titles that need to be encoded to be represented +in HTML, you may need to add further processing to the last line. + +### Querying the database + +If you have access to the relational database in which your mediawiki data is +stored, it is possible to derive a list of page names from this. + +## Step 2: fetching the page data + +Once you have a list of page names, you can fetch the data for each page. + +### Method 1: via HTTP and `action=raw` + +You need to create two derived strings from the page titles already: the +destination path for the page and the source URL. Assuming `$pagename` +contains a pagename obtained above, and `$wiki` contains the URL to your +mediawiki's `index.php` file: + + src=`echo "$pagename" | tr ' ' _ | sed 's,&,&,g'` + dest=`"$pagename" | tr ' ' _ | sed 's,&,__38__,g'` + + mkdir -p `dirname "$dest"` + wget -q "$wiki?title=$src&action=raw" -O "$dest" + +### Method 2: via HTTP and `Special:Export` + +Mediawiki also has a special page `Special:Export` which can be used to obtain +the source of the page and other metadata such as the last contributor, or the +full history, etc. + +You need to send a `POST` request to the `Special:Export` page. See the source +of the page fetched via `GET` to determine the correct arguments. + +You will then need to write an XML parser to extract the data you need from +the result. + +### Method 3: via the database + +It is possible to extract the page data from the database with some +well-crafted queries. + +## Step 2: format conversion + +The next step is to convert Mediawiki conventions into Ikiwiki ones. These +include + + * convert Categories into tags + * ... + +## External links + +[[sabr]] used to explain how to [import MediaWiki content into +git](http://u32.net/Mediawiki_Conversion/index.html?updated), including full +edit history, but as of 2009/10/16 that site is not available. + +The [[plugins/contrib/mediawiki]] plugin can then be used by ikiwiki to build +the wiki. -- cgit v1.2.3 From 1c3fc89b8462e4cef2d3fa92dc55c4a9a0658aac Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Fri, 16 Oct 2009 11:46:27 +0200 Subject: Prefixing the HTML Title. --- doc/users/tschwinge.mdwn | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/doc/users/tschwinge.mdwn b/doc/users/tschwinge.mdwn index 54614e922..384fac09d 100644 --- a/doc/users/tschwinge.mdwn +++ b/doc/users/tschwinge.mdwn @@ -87,6 +87,12 @@ on)* after a customizable amount of characters (or lines) would be a another possibility. The *... (read on)* would then either toggle the full content being displayed or link to the complete page. +## Prefix For the HTML Title + +The title of each page (as in ``...) should be prefixed with +*GNU Project - GNU Hurd -*. We can either do this directly in `page.tmpl`, or +create a way to modify the `TITLE` template variable suitably. + ## Unverified -- these may be bugs, but have yet to be verified * ikiwiki doesn't change its internal database when \[[!meta date]] / -- cgit v1.2.3 From 53bf9301eab668cb448cf42a318647dc892a2985 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge <tschwinge@gnu.org> Date: Fri, 16 Oct 2009 11:48:26 +0200 Subject: Potential issues with the recentchanges plugin. --- doc/users/tschwinge.mdwn | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/doc/users/tschwinge.mdwn b/doc/users/tschwinge.mdwn index 384fac09d..05587a24c 100644 --- a/doc/users/tschwinge.mdwn +++ b/doc/users/tschwinge.mdwn @@ -103,3 +103,13 @@ create a way to modify the `TITLE` template variable suitably. ([[plugins/cutpaste]]) in RSS feed (only; not Atom?) under some conditions (refresh only, but not rebuild?). Perhaps missing to read in / parse some files? + + * [[plugins/recentchanges]] + + * Creates non-existing links to changes. + + * Invalid *directory link* with `--usedirs`. + + * Doesn't honor `$timeformat`. + + * Does create `recentchangees.*` files even if that is overridden. -- cgit v1.2.3 From d352af1da54851cb7dfdcb31664ad43ecfa3ae85 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge <tschwinge@gnu.org> Date: Fri, 16 Oct 2009 11:53:37 +0200 Subject: inline plugin: feedfile option w/ usedirs. --- doc/users/tschwinge.mdwn | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/doc/users/tschwinge.mdwn b/doc/users/tschwinge.mdwn index 05587a24c..7220310f3 100644 --- a/doc/users/tschwinge.mdwn +++ b/doc/users/tschwinge.mdwn @@ -93,6 +93,13 @@ The title of each page (as in `<html><head><title>`...) should be prefixed with *GNU Project - GNU Hurd -*. We can either do this directly in `page.tmpl`, or create a way to modify the `TITLE` template variable suitably. +## [[plugins/inline]] feedfile option + +Not that important. Git commit b67632cdcdd333cf0a88d03c0f7e6e62921f32c3. This +would be nice to have even when using *usedirs*. Might involve issues as +discussed in *N-to-M Mapping of Input and Output Files* on +[[plugins/contrib/texinfo]]. + ## Unverified -- these may be bugs, but have yet to be verified * ikiwiki doesn't change its internal database when \[[!meta date]] / -- cgit v1.2.3 From 5757ad87740fc50ead02aa06cadeb8b591dfcb82 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge <tschwinge@gnu.org> Date: Fri, 16 Oct 2009 11:57:49 +0200 Subject: Add a comment about the texinfo plugin. --- doc/users/tschwinge.mdwn | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/doc/users/tschwinge.mdwn b/doc/users/tschwinge.mdwn index 7220310f3..80eeae0c5 100644 --- a/doc/users/tschwinge.mdwn +++ b/doc/users/tschwinge.mdwn @@ -55,7 +55,8 @@ licensing headers to every single file. ## Texinfo -- [[plugins/contrib/texinfo]] -Not very important. +Not very important. Have to consider external commands / files / security (see +[[plugins/teximg]] source code)? ## Shortcuts -- [[plugins/shortcut/discussion]] -- cgit v1.2.3 From 811394ec92205112d3fec0d4f6aac380202f114b Mon Sep 17 00:00:00 2001 From: Jon Dowland <jon@alcopop.org> Date: Fri, 16 Oct 2009 11:20:32 +0100 Subject: add snippet for converting categories --- doc/tips/convert_mediawiki_to_ikiwiki.mdwn | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/doc/tips/convert_mediawiki_to_ikiwiki.mdwn b/doc/tips/convert_mediawiki_to_ikiwiki.mdwn index c522eaec3..ce0c684e7 100644 --- a/doc/tips/convert_mediawiki_to_ikiwiki.mdwn +++ b/doc/tips/convert_mediawiki_to_ikiwiki.mdwn @@ -1,3 +1,5 @@ +[[!toc levels=2]] + Mediawiki is a dynamically-generated wiki which stores it's data in a relational database. Pages are marked up using a proprietary markup. It is possible to import the contents of a Mediawiki site into an ikiwiki, @@ -85,7 +87,23 @@ The next step is to convert Mediawiki conventions into Ikiwiki ones. These include * convert Categories into tags - * ... + + import sys, re + pattern = r'\[\[Category:([^\]]+)\]\]' + + def manglecat(mo): + return '[[!tag %s]]' % mo.group(1).strip().replace(' ','_') + + for line in sys.stdin.readlines(): + res = re.match(pattern, line) + if res: + sys.stdout.write(re.sub(pattern, manglecat, line)) + else: sys.stdout.write(line) + +## Step 3: Mediawiki plugin + +The [[plugins/contrib/mediawiki]] plugin can be used by ikiwiki to interpret +most of the Mediawiki syntax. ## External links @@ -93,5 +111,3 @@ include git](http://u32.net/Mediawiki_Conversion/index.html?updated), including full edit history, but as of 2009/10/16 that site is not available. -The [[plugins/contrib/mediawiki]] plugin can then be used by ikiwiki to build -the wiki. -- cgit v1.2.3 From 58287a929b0ea209a062d11940232a3d31805885 Mon Sep 17 00:00:00 2001 From: Jon Dowland <jon@alcopop.org> Date: Fri, 16 Oct 2009 11:21:26 +0100 Subject: explanatory text --- doc/tips/convert_mediawiki_to_ikiwiki.mdwn | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/doc/tips/convert_mediawiki_to_ikiwiki.mdwn b/doc/tips/convert_mediawiki_to_ikiwiki.mdwn index ce0c684e7..3f771ed9b 100644 --- a/doc/tips/convert_mediawiki_to_ikiwiki.mdwn +++ b/doc/tips/convert_mediawiki_to_ikiwiki.mdwn @@ -83,10 +83,13 @@ well-crafted queries. ## Step 2: format conversion -The next step is to convert Mediawiki conventions into Ikiwiki ones. These -include +The next step is to convert Mediawiki conventions into Ikiwiki ones. - * convert Categories into tags +### categories + +Mediawiki uses a special page name prefix to define "Categories", which +otherwise behave like ikiwiki tags. You can convert every Mediawiki category +into an ikiwiki tag name using a script such as import sys, re pattern = r'\[\[Category:([^\]]+)\]\]' -- cgit v1.2.3 From 0bee92347020e3034faea5d3cc045ca443e4221c Mon Sep 17 00:00:00 2001 From: Jon Dowland <jon@alcopop.org> Date: Fri, 16 Oct 2009 11:23:55 +0100 Subject: minor reworking of page fetch section --- doc/tips/convert_mediawiki_to_ikiwiki.mdwn | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/doc/tips/convert_mediawiki_to_ikiwiki.mdwn b/doc/tips/convert_mediawiki_to_ikiwiki.mdwn index 3f771ed9b..4a750882b 100644 --- a/doc/tips/convert_mediawiki_to_ikiwiki.mdwn +++ b/doc/tips/convert_mediawiki_to_ikiwiki.mdwn @@ -53,7 +53,7 @@ Once you have a list of page names, you can fetch the data for each page. ### Method 1: via HTTP and `action=raw` -You need to create two derived strings from the page titles already: the +You need to create two derived strings from the page titles: the destination path for the page and the source URL. Assuming `$pagename` contains a pagename obtained above, and `$wiki` contains the URL to your mediawiki's `index.php` file: @@ -64,6 +64,9 @@ mediawiki's `index.php` file: mkdir -p `dirname "$dest"` wget -q "$wiki?title=$src&action=raw" -O "$dest" +You may need to add more conversions here depending on the precise page titles +used in your wiki. + ### Method 2: via HTTP and `Special:Export` Mediawiki also has a special page `Special:Export` which can be used to obtain -- cgit v1.2.3 From f6d08ceb358f6d5e51c54ab1b9e90cc4a86fad37 Mon Sep 17 00:00:00 2001 From: Jon Dowland <jon@alcopop.org> Date: Fri, 16 Oct 2009 11:44:13 +0100 Subject: notes about namespaces --- doc/tips/convert_mediawiki_to_ikiwiki.mdwn | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/doc/tips/convert_mediawiki_to_ikiwiki.mdwn b/doc/tips/convert_mediawiki_to_ikiwiki.mdwn index 4a750882b..87b1ebc48 100644 --- a/doc/tips/convert_mediawiki_to_ikiwiki.mdwn +++ b/doc/tips/convert_mediawiki_to_ikiwiki.mdwn @@ -42,6 +42,16 @@ to adjust this script too: Also, if you have pages with titles that need to be encoded to be represented in HTML, you may need to add further processing to the last line. +Note that by default, `Special:Allpages` will only list pages in the main +namespace. You need to add a `&namespace=XX` argument to get pages in a +different namespace. The following numbers correspond to common namespaces: + + * 10 - templates (`Template:foo`) + * 14 - categories (`Category:bar`) + +Note that the page names obtained this way will not include any namespace +specific prefix: e.g. `Category:` will be stripped off. + ### Querying the database If you have access to the relational database in which your mediawiki data is @@ -67,6 +77,12 @@ mediawiki's `index.php` file: You may need to add more conversions here depending on the precise page titles used in your wiki. +If you are trying to fetch pages from a different namespace to the default, +you will need to prefix the page title with the relevant prefix, e.g. +`Category:` for category pages. You probably don't want to prefix it to the +output page, but you may want to vary the destination path (i.e. insert an +extra directory component corresponding to your ikiwiki's `tagbase`). + ### Method 2: via HTTP and `Special:Export` Mediawiki also has a special page `Special:Export` which can be used to obtain -- cgit v1.2.3 From 7a9512918d981762016bdaf7e6196fe2d46c10c7 Mon Sep 17 00:00:00 2001 From: Jon Dowland <jon@alcopop.org> Date: Fri, 16 Oct 2009 11:59:27 +0100 Subject: fix step numbering --- doc/tips/convert_mediawiki_to_ikiwiki.mdwn | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/tips/convert_mediawiki_to_ikiwiki.mdwn b/doc/tips/convert_mediawiki_to_ikiwiki.mdwn index 87b1ebc48..1e5b912a9 100644 --- a/doc/tips/convert_mediawiki_to_ikiwiki.mdwn +++ b/doc/tips/convert_mediawiki_to_ikiwiki.mdwn @@ -100,7 +100,7 @@ the result. It is possible to extract the page data from the database with some well-crafted queries. -## Step 2: format conversion +## Step 3: format conversion The next step is to convert Mediawiki conventions into Ikiwiki ones. @@ -122,7 +122,7 @@ into an ikiwiki tag name using a script such as sys.stdout.write(re.sub(pattern, manglecat, line)) else: sys.stdout.write(line) -## Step 3: Mediawiki plugin +## Step 4: Mediawiki plugin The [[plugins/contrib/mediawiki]] plugin can be used by ikiwiki to interpret most of the Mediawiki syntax. -- cgit v1.2.3 From 4c9ded61db6baf9a5cbd7342055b9ad783ca2351 Mon Sep 17 00:00:00 2001 From: Jon Dowland <jon@alcopop.org> Date: Fri, 16 Oct 2009 13:24:37 +0100 Subject: response --- doc/forum/ever-growing_list_of_pages.mdwn | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/doc/forum/ever-growing_list_of_pages.mdwn b/doc/forum/ever-growing_list_of_pages.mdwn index 84b9fe6ee..85a39e107 100644 --- a/doc/forum/ever-growing_list_of_pages.mdwn +++ b/doc/forum/ever-growing_list_of_pages.mdwn @@ -10,3 +10,10 @@ remove the text about the implementation bug that has been fixed, or should it stay there, for reference? --[[tschwinge]] + +> To answer a question with a question, what harm does having the done bugs +> around cause? At some point in the future perhaps the number of done pages +> will be large enough to be a time or space concern. Do you think we've +> reached a point now? One advantage of having them around is that people +> running older versions of the Ikiwiki software may find the page explaining +> that the bug is fixed if they perform a search. -- [[Jon]] -- cgit v1.2.3