From 53a8aeb1d1cfd42ae069e173437b34489d28d157 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 2 Oct 2009 15:45:14 -0400 Subject: implemented one fix in transitive-dependencies branch, but not really happy with it --- doc/bugs/transitive_dependencies.mdwn | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) (limited to 'doc') diff --git a/doc/bugs/transitive_dependencies.mdwn b/doc/bugs/transitive_dependencies.mdwn index c61afe81e..89f0d7085 100644 --- a/doc/bugs/transitive_dependencies.mdwn +++ b/doc/bugs/transitive_dependencies.mdwn @@ -25,9 +25,15 @@ everything uses) will just result in semi-redundant dependency info being recorded. If we make destpage mandatory to fix this, goofing up will lead to this bug coming back. Ugh. -Another approach to fix it could be to say that anything that causes a +---- + +## rebuild = change approach + +[[!template id=gitbranch branch=master/transitive-dependencies author="[[joey]]"]] + +Another approach to fix it is to say that anything that causes a rebuild of B is treated as a change of B. Then when C is changed, B is -rebuilt due to dependencies, and in turn this means A is rebuild because B +rebuilt due to dependencies, and in turn this means A is rebuilt because B "changed". This is essentially what is done with wikilinks now, and why, if a sidebar @@ -40,8 +46,17 @@ here: building contact.mdwn, which depends on sidebar ... -The only downside I can see with this approach is that it involves more work. -Does the dep resolver have to keep looping until no new pages are rebuilt? -Seems worth a try to implement this approach. +Downsides here: + +* Means a minimum of 2x as much time spent resolving dependencies, + at least in my simple implementation, which re-runs the dependency + resolution loop until no new pages are rebuilt. +* Causes extra work for some transitive dependencies that we don't + actually care about. For example, changing index causes + plugins/brokenlinks to update in the first pass; if there's a second + pass, plugins/map is then updated, because it depends on plugins/brokenlinks. + (Of course, this is just a special case of the issue that a real + modification to plugins/brokenlinks causes an unnecessary update of plugins/map, + because we have only one kind of dependency.) --[[Joey]] -- cgit v1.2.3 From 537824a2791485536746e78b27d232e868560171 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 2 Oct 2009 15:46:27 -0400 Subject: fix branch name --- doc/bugs/transitive_dependencies.mdwn | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/bugs/transitive_dependencies.mdwn b/doc/bugs/transitive_dependencies.mdwn index 89f0d7085..e3043b61b 100644 --- a/doc/bugs/transitive_dependencies.mdwn +++ b/doc/bugs/transitive_dependencies.mdwn @@ -29,7 +29,7 @@ this bug coming back. Ugh. ## rebuild = change approach -[[!template id=gitbranch branch=master/transitive-dependencies author="[[joey]]"]] +[[!template id=gitbranch branch=origin/transitive-dependencies author="[[joey]]"]] Another approach to fix it is to say that anything that causes a rebuild of B is treated as a change of B. Then when C is changed, B is -- cgit v1.2.3 From 4c88d42762380928d352a26a4faaffb88cbb2406 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 2 Oct 2009 15:48:47 -0400 Subject: mention loop optimisation --- doc/bugs/transitive_dependencies.mdwn | 2 ++ 1 file changed, 2 insertions(+) (limited to 'doc') diff --git a/doc/bugs/transitive_dependencies.mdwn b/doc/bugs/transitive_dependencies.mdwn index e3043b61b..d5571cb6a 100644 --- a/doc/bugs/transitive_dependencies.mdwn +++ b/doc/bugs/transitive_dependencies.mdwn @@ -51,6 +51,8 @@ Downsides here: * Means a minimum of 2x as much time spent resolving dependencies, at least in my simple implementation, which re-runs the dependency resolution loop until no new pages are rebuilt. + (I added an optimisation that gets it down to 1.5X as much work on + average, still 2x as much worst case.) * Causes extra work for some transitive dependencies that we don't actually care about. For example, changing index causes plugins/brokenlinks to update in the first pass; if there's a second -- cgit v1.2.3 From 8bb94bb197714fcac1ac48f9b330bef4d17dd800 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 2 Oct 2009 15:56:44 -0400 Subject: split out dependency type issue into its own todo --- doc/todo/dependency_types.mdwn | 83 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+) create mode 100644 doc/todo/dependency_types.mdwn (limited to 'doc') diff --git a/doc/todo/dependency_types.mdwn b/doc/todo/dependency_types.mdwn new file mode 100644 index 000000000..db7d06914 --- /dev/null +++ b/doc/todo/dependency_types.mdwn @@ -0,0 +1,83 @@ +Moved this relevant discussion to here from +[[tracking_bugs_with_dependencies]]: --[[Joey]] + +>> it seems that there are two types of dependency, and ikiwiki +>> currently only handles one of them. The first type is "Rebuild this +>> page when any of these other pages changes" - ikiwiki handles this. +>> The second type is "rebuild this page when set of pages referred to by +>> this pagespec changes" - ikiwiki doesn't seem to handle this. I +>> suspect that named pagespecs would make that second type of dependency +>> more important. I'll try to come up with a good example. -- [[Will]] + +>>> Hrm, I was going to build an example of this with backlinks, but it +>>> looks like that is handled as a special case at the moment (line 458 of +>>> render.pm). I'll see if I can breapk +>>> things another way. Fixing this properly would allow removal of that special case. -- [[Will]] + +>>>> I can't quite understand the distinction you're trying to draw +>>>> between the two types of dependencies. Backlinks are a very special +>>>> case though and I'll be suprised if they fit well into pagespecs. +>>>> --[[Joey]] + +>>>>> The issue is that the existential pagespec matching allows you to build things that have similar +>>>>> problems to backlinks. +>>>>> e.g. the following inline: + + \[[!inline pages="define(~done, link(done)) and link(~done)" archive=yes]] + +>>>>> includes any page that links to a page that links to done. Now imagine I add a new link to 'done' on +>>>>> some random page somewhere - a page which some other page links to which didn't previously get included - the set of pages accepted by the pagespec, and hence the set of +>>>>> pages inlined, will change. But, there is no dependency anywhere on the page that I altered, so +>>>>> ikiwiki will not rebuild the page with the inline in it. What is happening is that the page that I altered affects +>>>>> the set of pages matched by the pagespec without itself being matched by the pagespec, and hence included in the dependency list. + +>>>>> To make this work well, I think you need to recognise two types of dependencies for each page (and no +>>>>> special cases for particular types of links, eg backlinks). The first type of dependency says, "The content of +>>>>> this page depends upon the content of these other pages". The `add_depends()` in the shortcuts +>>>>> plugin is of this form: any time the shortcuts page is edited, any page with a shortcut on it +>>>>> is rebuilt. The inline plugin also needs to add dependencies of this form to detect when the inlined +>>>>> content changes. By contrast, the map plugin does not need a dependency of this form, because it +>>>>> doesn't actually care about the content of any pages, just which pages it needs to include (which we'll handle next). + +>>>>> The second type of dependency says, "The content of this page depends upon the exact set of pages matched +>>>>> by this pagespec". The first type of dependency was about the content of some pages, the second type is about +>>>>> which pages get matched by a pagespec. This is the type of dependency tracking that the map plugin needs. +>>>>> If the set of pages matched by map pagespec changes, then the page with the map on it needs to be rebuilt to show a different list of pages. +>>>>> Inline needs this type of dependency as well as the previous type - This type handles a change in which pages +>>>>> are inlined, the previous type handles a change in the content of any of those pages. Shortcut does not need this type of +>>>>> dependency. Most of the places that use `add_depends()` seem to need this type of dependency rather than the first type. + +>>>>>> Note that inline and map currently achieve the second type of dependency by +>>>>>> explicitly calling `add_depends` for each page the displayed. +>>>>>> If any of those pages are removed, the regular pagespec would not +>>>>>> match them -- since they're gone. However, the explicit dependency +>>>>>> on them does cause them to match. It's an ugly corner I'd like to +>>>>>> get rid of. --[[Joey]] + +>>>>> Implementation Details: The first type of dependency can be handled very similarly to the current +>>>>> dependency system. You just need to keep a list of pages that the content depends upon. You could +>>>>> keep that list as a pagespec, but if you do this you might want to check that the pagespec doesn't change, +>>>>> possibly by adding a dependency of the second type along with the dependency of the first type. + +>>>>>> An example of the current system not tracking enough data is +>>>>>> described in [[bugs/transitive_dependencies]]. +>>>>>> --[[Joey]] + +>>>>> The second type of dependency is a little more tricky. For each page, we'd need a list of pagespecs that +>>>>> the page depended on, and for each pagespec you'd want to store the list of pages that currently match it. +>>>>> On refresh, you'd need to check each pagespec to see if the set of pages that match it has changed, and if +>>>>> that set has changed, then rebuild the dependent page(s). Oh, and for this second type of dependency, I +>>>>> don't think you can merge pagespecs. If I wanted to know if either "\*" or "link(done)" changes, then just checking +>>>>> to see if the set of pages matched by "\* or link(done)" changes doesn't work. + +>>>>> The current system works because even though you usually want dependencies of the second type, the set of pages +>>>>> referred to by a pagespec can only change if one of those pages itself changes. i.e. A dependency check of the +>>>>> first type will catch a dependency change of the second type with current pagespecs. +>>>>> This doesn't work with backlinks, and it doesn't work with existential matching. Backlinks are currently special-cased. I don't know +>>>>> how to special-case existential matching - I suspect you're better off just getting the dependency tracking right. + +>>>>> I also tried to come up with other possible solutions: e.g. can we find the dependencies for a pagespec? That +>>>>> would be the set of pages where a change on one of those pages could lead to a change in the set of pages matched by the pagespec. +>>>>> For old-style pagespecs without backlinks, the dependency set for a pagespec is the same as the set of pages the pagespec matches. +>>>>> Unfortunately, with existential matching, the set of pages that each +>>>>> pagespec depends upon can quickly become "*", which is not very useful. -- [[Will]] -- cgit v1.2.3 From 8c2d221ca93ed6fbe8f093408c4e2cea835e5b4c Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 2 Oct 2009 16:17:56 -0400 Subject: split off todo item for multiple dependency types --- doc/bugs/transitive_dependencies.mdwn | 2 +- doc/todo/dependency_types.mdwn | 46 +++++++++++++- doc/todo/tracking_bugs_with_dependencies.mdwn | 86 ++------------------------- 3 files changed, 50 insertions(+), 84 deletions(-) (limited to 'doc') diff --git a/doc/bugs/transitive_dependencies.mdwn b/doc/bugs/transitive_dependencies.mdwn index d5571cb6a..546f4f3aa 100644 --- a/doc/bugs/transitive_dependencies.mdwn +++ b/doc/bugs/transitive_dependencies.mdwn @@ -59,6 +59,6 @@ Downsides here: pass, plugins/map is then updated, because it depends on plugins/brokenlinks. (Of course, this is just a special case of the issue that a real modification to plugins/brokenlinks causes an unnecessary update of plugins/map, - because we have only one kind of dependency.) + because we have [[only_one_kind_of_dependency|todo/dependency_types]].) --[[Joey]] diff --git a/doc/todo/dependency_types.mdwn b/doc/todo/dependency_types.mdwn index db7d06914..6d722aab9 100644 --- a/doc/todo/dependency_types.mdwn +++ b/doc/todo/dependency_types.mdwn @@ -1,5 +1,24 @@ -Moved this relevant discussion to here from -[[tracking_bugs_with_dependencies]]: --[[Joey]] +Ikiwiki currently only has one type of dependency between pages +(plus wikilinks special cased in on the side). This has resulted in various +problems, and it's seemed for a long time to me that ikiwiki needs to get +smarter about what types of dependencies are supported. + +### unnecessary work + +The current single dependency type causes the depending page to be rebuilt +whenever a matching dependency is added, removed, or *modified*. But a +great many things don't care about the modification case, and often cause +unnecessary page rebuilds: + +* meta only cares if the pages are added or removed. Content change does + not matter (unless show=title is used). +* brokenlinks, orphans, pagecount, ditto +* inline in archive mode cares about page title, author changing, but + not content. (Ditto for meta with show=title.) +* Causes extra work when solving the [[bug/transitive_dependencies]] + problem. + +### two types of dependencies needed for [[tracking_bugs_with_dependencies]] >> it seems that there are two types of dependency, and ikiwiki >> currently only handles one of them. The first type is "Rebuild this @@ -81,3 +100,26 @@ Moved this relevant discussion to here from >>>>> For old-style pagespecs without backlinks, the dependency set for a pagespec is the same as the set of pages the pagespec matches. >>>>> Unfortunately, with existential matching, the set of pages that each >>>>> pagespec depends upon can quickly become "*", which is not very useful. -- [[Will]] + +### proposal + +I propose the following. --[[Joey]] + +* Add a second type of dependency, call it an "contentless dependency". +* `add_depends` defaults to adding a regular ("full") dependency, as + before. (So nothing breaks.) +* `add_depends($page, $spec, content => 0)` adds an contentless dependency. +* Contentless dependencies are stored in `%depends_contentless` and + `%depends_contentless_simple`, which are stored in the index similarly + to the existing hashes. +* `refresh` only looks at added/removed pages when resolving contentless + dependencies. + +This seems straightforwardly doable. I'd like [[Will]]'s feedback on it, if +possible. The type types of dependencies I am proposing are not identical +to the two types he talks about above, but I hope are close enough that +they can be used. + +This doesn't deal with the stuff that only depend on the metadata of a +page, as collected in the scan pass, changing. But it does leave a window +open for adding such a dependency type later. diff --git a/doc/todo/tracking_bugs_with_dependencies.mdwn b/doc/todo/tracking_bugs_with_dependencies.mdwn index 3894df5f6..5f3ece290 100644 --- a/doc/todo/tracking_bugs_with_dependencies.mdwn +++ b/doc/todo/tracking_bugs_with_dependencies.mdwn @@ -360,7 +360,10 @@ account all comments above (which doesn't mean it is above reproach :) ). --[[W > --[[Joey]] >> There is one issue that I've been thinking about that I haven't raised anywhere (or checked myself), and that is how this all interacts with page dependencies. ->> Firstly, I'm not sure anymore that the `pagespec_merge` function will continue to work in all cases. +>> +>>> I've moved the discussion of that to [[dependency_types]]. --[[Joey]] +>> +>> I'm not sure anymore that the `pagespec_merge` function will continue to work in all cases. >>> The problem I can see there is that if two pagespecs >>> get merged and both use `~foo` but define it differently, @@ -413,86 +416,7 @@ account all comments above (which doesn't mean it is above reproach :) ). --[[W >>> My [[remove-pagespec-merge|should_optimise_pagespecs]] branch has now >>> solved all this by deleting the offending function :-) --[[smcv]] ->> Secondly, it seems that there are two types of dependency, and ikiwiki ->> currently only handles one of them. The first type is "Rebuild this ->> page when any of these other pages changes" - ikiwiki handles this. ->> The second type is "rebuild this page when set of pages referred to by ->> this pagespec changes" - ikiwiki doesn't seem to handle this. I ->> suspect that named pagespecs would make that second type of dependency ->> more important. I'll try to come up with a good example. -- [[Will]] - ->>> Hrm, I was going to build an example of this with backlinks, but it ->>> looks like that is handled as a special case at the moment (line 458 of ->>> render.pm). I'll see if I can breapk ->>> things another way. Fixing this properly would allow removal of that special case. -- [[Will]] - ->>>> I can't quite understand the distinction you're trying to draw ->>>> between the two types of dependencies. Backlinks are a very special ->>>> case though and I'll be suprised if they fit well into pagespecs. ->>>> --[[Joey]] - ->>>>> The issue is that the existential pagespec matching allows you to build things that have similar ->>>>> problems to backlinks. ->>>>> e.g. the following inline: - - \[[!inline pages="define(~done, link(done)) and link(~done)" archive=yes]] - ->>>>> includes any page that links to a page that links to done. Now imagine I add a new link to 'done' on ->>>>> some random page somewhere - a page which some other page links to which didn't previously get included - the set of pages accepted by the pagespec, and hence the set of ->>>>> pages inlined, will change. But, there is no dependency anywhere on the page that I altered, so ->>>>> ikiwiki will not rebuild the page with the inline in it. What is happening is that the page that I altered affects ->>>>> the set of pages matched by the pagespec without itself being matched by the pagespec, and hence included in the dependency list. - ->>>>> To make this work well, I think you need to recognise two types of dependencies for each page (and no ->>>>> special cases for particular types of links, eg backlinks). The first type of dependency says, "The content of ->>>>> this page depends upon the content of these other pages". The `add_depends()` in the shortcuts ->>>>> plugin is of this form: any time the shortcuts page is edited, any page with a shortcut on it ->>>>> is rebuilt. The inline plugin also needs to add dependencies of this form to detect when the inlined ->>>>> content changes. By contrast, the map plugin does not need a dependency of this form, because it ->>>>> doesn't actually care about the content of any pages, just which pages it needs to include (which we'll handle next). - ->>>>> The second type of dependency says, "The content of this page depends upon the exact set of pages matched ->>>>> by this pagespec". The first type of dependency was about the content of some pages, the second type is about ->>>>> which pages get matched by a pagespec. This is the type of dependency tracking that the map plugin needs. ->>>>> If the set of pages matched by map pagespec changes, then the page with the map on it needs to be rebuilt to show a different list of pages. ->>>>> Inline needs this type of dependency as well as the previous type - This type handles a change in which pages ->>>>> are inlined, the previous type handles a change in the content of any of those pages. Shortcut does not need this type of ->>>>> dependency. Most of the places that use `add_depends()` seem to need this type of dependency rather than the first type. - ->>>>>> Note that inline and map currently achieve the second type of dependency by ->>>>>> explicitly calling `add_depends` for each page the displayed. ->>>>>> If any of those pages are removed, the regular pagespec would not ->>>>>> match them -- since they're gone. However, the explicit dependency ->>>>>> on them does cause them to match. It's an ugly corner I'd like to ->>>>>> get rid of. --[[Joey]] - ->>>>> Implementation Details: The first type of dependency can be handled very similarly to the current ->>>>> dependency system. You just need to keep a list of pages that the content depends upon. You could ->>>>> keep that list as a pagespec, but if you do this you might want to check that the pagespec doesn't change, ->>>>> possibly by adding a dependency of the second type along with the dependency of the first type. - ->>>>>> An example of the current system not tracking enough data is ->>>>>> described in [[bugs/transitive_dependencies]]. ->>>>>> --[[Joey]] - ->>>>> The second type of dependency is a little more tricky. For each page, we'd need a list of pagespecs that ->>>>> the page depended on, and for each pagespec you'd want to store the list of pages that currently match it. ->>>>> On refresh, you'd need to check each pagespec to see if the set of pages that match it has changed, and if ->>>>> that set has changed, then rebuild the dependent page(s). Oh, and for this second type of dependency, I ->>>>> don't think you can merge pagespecs. If I wanted to know if either "\*" or "link(done)" changes, then just checking ->>>>> to see if the set of pages matched by "\* or link(done)" changes doesn't work. - ->>>>> The current system works because even though you usually want dependencies of the second type, the set of pages ->>>>> referred to by a pagespec can only change if one of those pages itself changes. i.e. A dependency check of the ->>>>> first type will catch a dependency change of the second type with current pagespecs. ->>>>> This doesn't work with backlinks, and it doesn't work with existential matching. Backlinks are currently special-cased. I don't know ->>>>> how to special-case existential matching - I suspect you're better off just getting the dependency tracking right. - ->>>>> I also tried to come up with other possible solutions: e.g. can we find the dependencies for a pagespec? That ->>>>> would be the set of pages where a change on one of those pages could lead to a change in the set of pages matched by the pagespec. ->>>>> For old-style pagespecs without backlinks, the dependency set for a pagespec is the same as the set of pages the pagespec matches. ->>>>> Unfortunately, with existential matching, the set of pages that each ->>>>> pagespec depends upon can quickly become "*", which is not very useful. -- [[Will]] + Patch updated to use closures rather than inline generated code for named pagespecs. Also includes some new use of ErrorReason where appropriate. -- [[Will]] -- cgit v1.2.3 From 1df5c5a22c7e04635c669f7d977da0318dfde9cf Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 2 Oct 2009 16:22:47 -0400 Subject: fix --- doc/todo/dependency_types.mdwn | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/todo/dependency_types.mdwn b/doc/todo/dependency_types.mdwn index 6d722aab9..215a65c8a 100644 --- a/doc/todo/dependency_types.mdwn +++ b/doc/todo/dependency_types.mdwn @@ -15,7 +15,7 @@ unnecessary page rebuilds: * brokenlinks, orphans, pagecount, ditto * inline in archive mode cares about page title, author changing, but not content. (Ditto for meta with show=title.) -* Causes extra work when solving the [[bug/transitive_dependencies]] +* Causes extra work when solving the [[bugs/transitive_dependencies]] problem. ### two types of dependencies needed for [[tracking_bugs_with_dependencies]] -- cgit v1.2.3 From c1ca46a785164eec099e003c0883245ca728e461 Mon Sep 17 00:00:00 2001 From: "http://jmtd.livejournal.com/" Date: Fri, 2 Oct 2009 17:23:37 -0400 Subject: response --- doc/bugs/brokenlinks_accumulates_duplicate_items.mdwn | 2 ++ 1 file changed, 2 insertions(+) (limited to 'doc') diff --git a/doc/bugs/brokenlinks_accumulates_duplicate_items.mdwn b/doc/bugs/brokenlinks_accumulates_duplicate_items.mdwn index 395c7789e..2091570d1 100644 --- a/doc/bugs/brokenlinks_accumulates_duplicate_items.mdwn +++ b/doc/bugs/brokenlinks_accumulates_duplicate_items.mdwn @@ -18,3 +18,5 @@ ikiwiki version 3.14159265. > If you can get to the problem after rebuilding with the current ikiwiki, > and then refreshing a few times, I guess I will need a copy of the wiki > source and the `.ikiwiki` directory to reproduce this. --[[Joey]] + +>> Hi Joey, thanks for your response. I've reproduced it post rebuild and after having ran ikiwiki-transition and many refreshes (both resulting from content changes and otherwise) unfortunately, with ikiwiki 3.14159265 (different machine to above report, though). I will contact you privately to provide a git URL and a copy of my .ikiwiki. -- [[Jon]] -- cgit v1.2.3 From a56fd6fde977cc48170b61d3518b13a3e1e1a936 Mon Sep 17 00:00:00 2001 From: "http://jmtd.livejournal.com/" Date: Fri, 2 Oct 2009 17:25:18 -0400 Subject: +bugs/ backlinks onhover thing can go weird --- doc/users/jon.mdwn | 1 + 1 file changed, 1 insertion(+) (limited to 'doc') diff --git a/doc/users/jon.mdwn b/doc/users/jon.mdwn index 960486b0c..e37090485 100644 --- a/doc/users/jon.mdwn +++ b/doc/users/jon.mdwn @@ -29,3 +29,4 @@ I am also working on some ikiwiki hacks: I am currently mostly interested in ikiwiki usability issues: * [[bugs/the login page is unclear when multiple methods exist]] + * [[bugs/backlinks onhover thing can go weird]] -- cgit v1.2.3 From 1d0d98eec4370f6daaf513f9553c5813e57e837e Mon Sep 17 00:00:00 2001 From: "http://jmtd.livejournal.com/" Date: Fri, 2 Oct 2009 17:30:46 -0400 Subject: wishlist/todo item: disable/enable directives by pagespec --- doc/todo/pagespec_to_disable_ikiwiki_directives.mdwn | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 doc/todo/pagespec_to_disable_ikiwiki_directives.mdwn (limited to 'doc') diff --git a/doc/todo/pagespec_to_disable_ikiwiki_directives.mdwn b/doc/todo/pagespec_to_disable_ikiwiki_directives.mdwn new file mode 100644 index 000000000..4211c2d10 --- /dev/null +++ b/doc/todo/pagespec_to_disable_ikiwiki_directives.mdwn @@ -0,0 +1,5 @@ +I would like some pages (identified by pagespec) to not expand ikiwiki directives (wikilinks, or \[[!, or both, or perhaps either). + +I will tag this [[wishlist]]. It's something I might try myself. It's part of my thinking about how to handle [[comments]], as I'm still ruminating on alternatives to [[smcv]]'s approach. (with the greatest of respect to smcv!) (Perhaps my attempt will try to factor out the no-directives-allowed logic from the comments plugin). + + -- [[Jon]] -- cgit v1.2.3 From 1ead5f7987ef06f872fb638777d6621f06eb6caf Mon Sep 17 00:00:00 2001 From: "http://jmtd.livejournal.com/" Date: Fri, 2 Oct 2009 17:31:40 -0400 Subject: --- doc/users/jon.mdwn | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/users/jon.mdwn b/doc/users/jon.mdwn index e37090485..b873a51e1 100644 --- a/doc/users/jon.mdwn +++ b/doc/users/jon.mdwn @@ -21,7 +21,7 @@ I am also working on some ikiwiki hacks: * an alternative approach to [[plugins/comments]] (see [[todo/more flexible inline postform]] for one piece of the puzzle; for some investigation into making the post - form more integrated) + form more integrated); possibly also [[todo/pagespec to disable ikiwiki directives]] * a system for [[forum/managing_todo_lists]] (see also [[todo/interactive todo lists]] and for the current WIP). -- cgit v1.2.3 From 06a1ad7e760258eb0689e327825c15e2aa5234b2 Mon Sep 17 00:00:00 2001 From: "http://www.cse.unsw.edu.au/~willu/" Date: Fri, 2 Oct 2009 18:28:11 -0400 Subject: Add reference to new plugin --- doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn (limited to 'doc') diff --git a/doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn b/doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn new file mode 100644 index 000000000..4cfbbbf3c --- /dev/null +++ b/doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn @@ -0,0 +1,5 @@ +At the moment, IkiWiki allows you to set the template for individual pages using the [[plugins/pagetemplate]] directive/plugin, but not for whole sections of the wiki. + +I've written a new plugin, sectiontemplate, available in the `page_tmpl` branch of my git repository that allows setting the template by pagespec in the config file. + +-- [[Will]] -- cgit v1.2.3 From 7ef22f25969741fb7aed9be650dd026dd6dddcc9 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 2 Oct 2009 18:45:56 -0400 Subject: improve wording --- doc/ikiwiki/directive/pagetemplate.mdwn | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/ikiwiki/directive/pagetemplate.mdwn b/doc/ikiwiki/directive/pagetemplate.mdwn index 0e4066f34..8ad901c1a 100644 --- a/doc/ikiwiki/directive/pagetemplate.mdwn +++ b/doc/ikiwiki/directive/pagetemplate.mdwn @@ -1,6 +1,8 @@ The `pagetemplate` directive is supplied by the [[!iki plugins/pagetemplate desc=pagetemplate]] plugin. -This directive allows a page to be created using a different wikitemplates. +This directive allows a page to be displayed using a different template than +the default `page.tmpl` template. + The page text is inserted into the template, so the template controls the overall look and feel of the wiki page. This is in contrast to the [[ikiwiki/directive/template]] directive, which allows inserting templates -- cgit v1.2.3 From 6e133959bc69877459fec2ea32788d0656b92168 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 2 Oct 2009 18:47:15 -0400 Subject: fix wording here too --- doc/plugins/pagetemplate.mdwn | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'doc') diff --git a/doc/plugins/pagetemplate.mdwn b/doc/plugins/pagetemplate.mdwn index afd5eb500..53f069d0d 100644 --- a/doc/plugins/pagetemplate.mdwn +++ b/doc/plugins/pagetemplate.mdwn @@ -2,8 +2,8 @@ [[!tag type/chrome]] This plugin provides the [[ikiwiki/directive/pagetemplate]] -[[ikiwiki/directive]], which allows a page to be created using a different -[[template|wikitemplates]]. +[[ikiwiki/directive]], which allows a page to be displayed +using a different [[template|wikitemplates]] than the default. This plugin can only use templates that are already installed in `/usr/share/ikiwiki/templates` (or wherever ikiwiki is configured to look for -- cgit v1.2.3 From db64972b65c7504b6d2bbed69e438203eef0a518 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 2 Oct 2009 18:51:52 -0400 Subject: combine with pagetemplate? --- doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn | 6 ++++++ 1 file changed, 6 insertions(+) (limited to 'doc') diff --git a/doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn b/doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn index 4cfbbbf3c..ed9740068 100644 --- a/doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn +++ b/doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn @@ -3,3 +3,9 @@ At the moment, IkiWiki allows you to set the template for individual pages using I've written a new plugin, sectiontemplate, available in the `page_tmpl` branch of my git repository that allows setting the template by pagespec in the config file. -- [[Will]] + +> This is an excellent idea and looks ready for merging. +> Before I do though, would it perhaps make sense to merge +> this in with the pagetemplate plugin? They do such a similar thing, +> and unless letting users control the template by editing a page is not +> wanted, I can't see any reason not to combine them. --[[Joey]] -- cgit v1.2.3 From 9673806a6d94b320bf2289dd091175a742bcee26 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Fri, 2 Oct 2009 18:58:07 -0400 Subject: consistency with edittemplate? --- doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn b/doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn index ed9740068..b9c397358 100644 --- a/doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn +++ b/doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn @@ -8,4 +8,10 @@ I've written a new plugin, sectiontemplate, available in the `page_tmpl` branch > Before I do though, would it perhaps make sense to merge > this in with the pagetemplate plugin? They do such a similar thing, > and unless letting users control the template by editing a page is not -> wanted, I can't see any reason not to combine them. --[[Joey]] +> wanted, I can't see any reason not to combine them. --[[Joey]] + +> One other idea is moving the pagespec from setup file to a directive. +> The edittemplate plugin does that, and it might be nice +> to be consistent with that. To implement it, you'd have to +> make the scan hook store the template pagespecs in `%pagestate`, +> so a bit more complicated. --[[Joey]] -- cgit v1.2.3 From 5de7ba82911ee1ab966a755e97931d5b6a3bacb1 Mon Sep 17 00:00:00 2001 From: "http://www.cse.unsw.edu.au/~willu/" Date: Sat, 3 Oct 2009 01:39:06 -0400 Subject: response --- doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn | 8 ++++++++ 1 file changed, 8 insertions(+) (limited to 'doc') diff --git a/doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn b/doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn index b9c397358..cd57c1257 100644 --- a/doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn +++ b/doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn @@ -15,3 +15,11 @@ I've written a new plugin, sectiontemplate, available in the `page_tmpl` branch > to be consistent with that. To implement it, you'd have to > make the scan hook store the template pagespecs in `%pagestate`, > so a bit more complicated. --[[Joey]] + +>> I started with the pagetemplate plugin, which is why they're so similar. I guess they could be combined. +>> I had a brief think about having the specs and templates defined in a directive rather than the config file, but it got tricky. +>> How do you know what needs to be rebuilt when? There is probably a solution, maybe even an obvious one, but I thought that getting something done was more important than polishing it. +>> In the worst case, admins can always use the web interface to change the setup :). + +>> I wanted this to put comments on my new blog, and was more interested in that goal than this subgoal. I've moved most of my web pages to IkiWiki and there is only one small part that is the blog. +>> I wanted to use [[Disqus comments|tips/Adding_Disqus_to_your_wiki/]], but only on the blog pages. (I'm using Disqus rather than IkiWiki comments because I don't want to have to deal with spam, security, etc. I'll happily just let someone else host the comments.) -- [[Will]] -- cgit v1.2.3 From ffa73790b5c381df6e0385995b720dd24188969b Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Sat, 3 Oct 2009 14:14:30 -0400 Subject: Fix a bug that could lead to duplicate links being recorded for tags. Here I was bitten by perl's aliasing of foreach variables to the loop array contents, and match_link accidentially changed the contents of %links. In Jon's testcase, a tag added an absolute link, which was made relative by the above bug, and then the link was added again in preprocess, and turned into a duplicate. --- IkiWiki.pm | 6 +++--- debian/changelog | 2 ++ doc/bugs/brokenlinks_accumulates_duplicate_items.mdwn | 3 +++ 3 files changed, 8 insertions(+), 3 deletions(-) (limited to 'doc') diff --git a/IkiWiki.pm b/IkiWiki.pm index 974e36902..2637f6017 100644 --- a/IkiWiki.pm +++ b/IkiWiki.pm @@ -2052,10 +2052,10 @@ sub match_link ($$;@) { else { return IkiWiki::SuccessReason->new("$page links to page $p matching $link") if match_glob($p, $link, %params); - $p=~s/^\///; + my ($p_rel)=$p=~/^\/?(.*)/; $link=~s/^\///; - return IkiWiki::SuccessReason->new("$page links to page $p matching $link") - if match_glob($p, $link, %params); + return IkiWiki::SuccessReason->new("$page links to page $p_rel matching $link") + if match_glob($p_rel, $link, %params); } } return IkiWiki::FailReason->new("$page does not link to $link"); diff --git a/debian/changelog b/debian/changelog index b6383bacb..ca5409af7 100644 --- a/debian/changelog +++ b/debian/changelog @@ -8,6 +8,8 @@ ikiwiki (3.14159266) UNRELEASED; urgency=low to the body tag. * Support RPC::XML 0.69's incompatable object instantiation method. * mirrorlist: Display nothing if list is empty. + * Fix a bug that could lead to duplicate links being recorded + for tags. -- Joey Hess Sun, 27 Sep 2009 17:40:03 -0400 diff --git a/doc/bugs/brokenlinks_accumulates_duplicate_items.mdwn b/doc/bugs/brokenlinks_accumulates_duplicate_items.mdwn index 2091570d1..d07b2381d 100644 --- a/doc/bugs/brokenlinks_accumulates_duplicate_items.mdwn +++ b/doc/bugs/brokenlinks_accumulates_duplicate_items.mdwn @@ -20,3 +20,6 @@ ikiwiki version 3.14159265. > source and the `.ikiwiki` directory to reproduce this. --[[Joey]] >> Hi Joey, thanks for your response. I've reproduced it post rebuild and after having ran ikiwiki-transition and many refreshes (both resulting from content changes and otherwise) unfortunately, with ikiwiki 3.14159265 (different machine to above report, though). I will contact you privately to provide a git URL and a copy of my .ikiwiki. -- [[Jon]] + +>>> Found the bug that was causing duplicates to get in, and fixed it. +>>> [[done]] --[[Joey]] -- cgit v1.2.3 From c7bdd17087475ef0e659cb5f3b204e2619929dff Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Sat, 3 Oct 2009 17:17:48 -0400 Subject: clarify --- doc/ikiwiki/directive/pagestats.mdwn | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/ikiwiki/directive/pagestats.mdwn b/doc/ikiwiki/directive/pagestats.mdwn index 426f3e4af..66f851dbd 100644 --- a/doc/ikiwiki/directive/pagestats.mdwn +++ b/doc/ikiwiki/directive/pagestats.mdwn @@ -12,7 +12,7 @@ And here's how to create a table of all the pages on the wiki: \[[!pagestats style="table"]] -The optional `among` parameter limits counting to pages that match a +The optional `among` parameter limits display to pages that match a [[ikiwiki/PageSpec]]. For instance, to display a cloud of tags used on blog entries, you could use: -- cgit v1.2.3 From bd958f91a2b71761e9aa20fa25a6702dab3b4b8d Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Sat, 3 Oct 2009 17:38:47 -0400 Subject: did a scratch implementation of dependancy types, but found it more complex --- doc/todo/dependency_types.mdwn | 37 ++++++++++++++++++++++++++++++++----- 1 file changed, 32 insertions(+), 5 deletions(-) (limited to 'doc') diff --git a/doc/todo/dependency_types.mdwn b/doc/todo/dependency_types.mdwn index 215a65c8a..6218222f7 100644 --- a/doc/todo/dependency_types.mdwn +++ b/doc/todo/dependency_types.mdwn @@ -12,7 +12,7 @@ unnecessary page rebuilds: * meta only cares if the pages are added or removed. Content change does not matter (unless show=title is used). -* brokenlinks, orphans, pagecount, ditto +* brokenlinks, orphans, pagecount, ditto (generally) * inline in archive mode cares about page title, author changing, but not content. (Ditto for meta with show=title.) * Causes extra work when solving the [[bugs/transitive_dependencies]] @@ -109,9 +109,6 @@ I propose the following. --[[Joey]] * `add_depends` defaults to adding a regular ("full") dependency, as before. (So nothing breaks.) * `add_depends($page, $spec, content => 0)` adds an contentless dependency. -* Contentless dependencies are stored in `%depends_contentless` and - `%depends_contentless_simple`, which are stored in the index similarly - to the existing hashes. * `refresh` only looks at added/removed pages when resolving contentless dependencies. @@ -121,5 +118,35 @@ to the two types he talks about above, but I hope are close enough that they can be used. This doesn't deal with the stuff that only depend on the metadata of a -page, as collected in the scan pass, changing. But it does leave a window +page, as collected in the scan pass, changing. But it does leave a window open for adding such a dependency type later. + +---- + +I implemented the above in a branch. +[[!template id=gitbranch branch=origin/dependency-types author="[[joey]]"]] + +Then I found some problems: + +* pagestats is often used with a pagespec that uses `tagged()`. + A pure contentless dependency does not work for that, it needs to look + at link info. +* orphans and brokenlinks cannot use contentless dependencies because they + need to update when links change. +* Something simple like pagecount, that seems like it could use a + contentless dependency, can have a pagespec that uses metadata, like + `author()` or `copyright()`. + +Now I'm thinking about having a contentless dependency look at page +metadata, and fire if the metadata changes. And it seems links should +either be included in that, or there should be a way to make a dependency +that fires when a page's links change. (And what about backlinks?) + +It's easy to see when a page's links change, since there is `%oldlinks`. +To see when metadata is changed is harder, since it's stored in the +pagestate by the meta plugin. + +(Alternative: Make add_depends look at the pagespec. Ie, if it is a simple +page name, or a glob, we know a contentless dependency can be valid. +If's more complex, convert the dependency from contentless to full. Finding +a non-ad-hoc, non-sucky way to do that could be hard.) -- cgit v1.2.3 From ba11568f1e1e0b0da0490d817262f13539736262 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Sat, 3 Oct 2009 17:43:23 -0400 Subject: response --- doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn | 6 ++++++ 1 file changed, 6 insertions(+) (limited to 'doc') diff --git a/doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn b/doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn index cd57c1257..d0c09796f 100644 --- a/doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn +++ b/doc/todo/Set_templates_for_whole_sections_of_the_site.mdwn @@ -23,3 +23,9 @@ I've written a new plugin, sectiontemplate, available in the `page_tmpl` branch >> I wanted this to put comments on my new blog, and was more interested in that goal than this subgoal. I've moved most of my web pages to IkiWiki and there is only one small part that is the blog. >> I wanted to use [[Disqus comments|tips/Adding_Disqus_to_your_wiki/]], but only on the blog pages. (I'm using Disqus rather than IkiWiki comments because I don't want to have to deal with spam, security, etc. I'll happily just let someone else host the comments.) -- [[Will]] + +>>> Yes, handing the rebuild is a good reason not to use directives for +>>> this. +>>> +>>> I do still think combining this with pagetemplate would be good. +>>> --[[Joey]] -- cgit v1.2.3 From 7b692b5d6ec2c1cf86b627c4e3ed7a5d0a751580 Mon Sep 17 00:00:00 2001 From: "http://kaizer.se/" Date: Sun, 4 Oct 2009 14:43:38 -0400 Subject: Updated pproc-indent by catching only indent at beginning of line --- ...esolve_native_reStructuredText_links_to_ikiwiki_pages.mdwn | 11 +++++++++++ 1 file changed, 11 insertions(+) (limited to 'doc') diff --git a/doc/todo/Resolve_native_reStructuredText_links_to_ikiwiki_pages.mdwn b/doc/todo/Resolve_native_reStructuredText_links_to_ikiwiki_pages.mdwn index e42f22970..ca7b282fa 100644 --- a/doc/todo/Resolve_native_reStructuredText_links_to_ikiwiki_pages.mdwn +++ b/doc/todo/Resolve_native_reStructuredText_links_to_ikiwiki_pages.mdwn @@ -271,6 +271,17 @@ Perl I've ever written!_) >>> Well, seems you want to match the indent at the start of the line containing >>> the directive, even if the directive does not start the line. That would >>> be quite hard to make a regexp do, though. --[[Joey]] +>> +>> I wasted a long time getting the simpler `indent($1, handle->($2,$,4))` to +>> work (remember, I don't know perl at all). Somehow `$1` does not arrive, I +>> made a simple testcase that worked, and I conclude something inside $handle +>> results in the value of $1 not arriving as it should! +>> +>> Anyway, instead a very simple incremental patch is in [pproc-indent][ppi] +>> where the indentation regex is `(^[ \t]+|)` instead, which seems to work +>> very well (and the regex is multiline now as well). I'm happy to rebase the +>> changes if you want or you can just squash the four patches 1+3 => 1+1 +>> -- [[ulrik]] [ppi]: http://github.com/engla/ikiwiki/commits/pproc-indent -- cgit v1.2.3 From 6f1ebdd6922a2c96fd57c71cd2052992d13f9c22 Mon Sep 17 00:00:00 2001 From: Joey Hess Date: Sun, 4 Oct 2009 15:53:54 -0400 Subject: update --- doc/todo/dependency_types.mdwn | 24 ++++++++++++++---------- 1 file changed, 14 insertions(+), 10 deletions(-) (limited to 'doc') diff --git a/doc/todo/dependency_types.mdwn b/doc/todo/dependency_types.mdwn index 6218222f7..58b5ee955 100644 --- a/doc/todo/dependency_types.mdwn +++ b/doc/todo/dependency_types.mdwn @@ -10,7 +10,7 @@ whenever a matching dependency is added, removed, or *modified*. But a great many things don't care about the modification case, and often cause unnecessary page rebuilds: -* meta only cares if the pages are added or removed. Content change does +* map only cares if the pages are added or removed. Content change does not matter (unless show=title is used). * brokenlinks, orphans, pagecount, ditto (generally) * inline in archive mode cares about page title, author changing, but @@ -128,14 +128,11 @@ I implemented the above in a branch. Then I found some problems: -* pagestats is often used with a pagespec that uses `tagged()`. - A pure contentless dependency does not work for that, it needs to look - at link info. -* orphans and brokenlinks cannot use contentless dependencies because they - need to update when links change. * Something simple like pagecount, that seems like it could use a contentless dependency, can have a pagespec that uses metadata, like `author()` or `copyright()`. +* pagestats, orphans and brokenlinks cannot use contentless dependencies + because they need to update when links change. Now I'm thinking about having a contentless dependency look at page metadata, and fire if the metadata changes. And it seems links should @@ -146,7 +143,14 @@ It's easy to see when a page's links change, since there is `%oldlinks`. To see when metadata is changed is harder, since it's stored in the pagestate by the meta plugin. -(Alternative: Make add_depends look at the pagespec. Ie, if it is a simple -page name, or a glob, we know a contentless dependency can be valid. -If's more complex, convert the dependency from contentless to full. Finding -a non-ad-hoc, non-sucky way to do that could be hard.) +Quick alternative: Make add_depends look at the pagespec. Ie, if it +is a simple page name, or a glob, we know a contentless dependency +can be valid. If's more complex, convert the dependency from +contentless to full. + +There is a lot to dislike about this method. Its parsing of the +pagespec, as currently implemented, does not let plugins add new types of +pagespecs that are contentless. Its pagespec parsing is also subject to +false negatives (though these should be somewhat rare, and no false +positives). Still, it does work, and it makes things like simple maps and +pagecounts much more efficient. -- cgit v1.2.3