Freezing pages : a very efficient caching system (specific to WebsiteBaker)

[quinto]

Project : frozen bread doug

Add a classical "webpage caching" feature (like joomla or CMS Made Simple) in WB would be cool, but as the core of WB wasn't conceived with "caching" in mind i know it can be a pain to add this feature, so i've an idea to make an even better system than the classical "caching" :

"Page Freezing"

I think it would be great to have the possibility to "freeze" pages, by "freezing" i mean to transform the php pages from the modules into statics generated html files, then store theses html pages in the /temp directory (or another) and rewrite the php page (the file in pages directory) in a such way that it just hardlink the corresponding html page (include with an array[] to redirect the links to the static pages).

I'm thinking it just need theses modifications to work while keeping compatibility with existing modules :

-1 we should give a new possibility for modules developpers to inform if their modules support page freezing or not: for example a gallery, a wysiwyg section, or even a news section can benefit greatly of being freezed, on an other hand for rss feed readers, forms with captchas or ajax-based modules freezing would have no sense.

So module developpers who wants their modules pages being "freezable" just have to edit their info.php by adding a new variable :
$module_freezable = "true";

-2 in the table "pages" add fields "freezable" and "frozen" which are set to false by default.

-3 in the core file of WB, in "wb/admin/pages/add.php" make a require of "WB_PATH.'/modules/'.$module.'/info.php", you have to do this BEFORE adding the page in the DB (statement : "INSERT INTO ".TABLE_PREFIX."pages.....") so u can modify this statement to insert the value from "$module_freezable" into the sql field "freezable".Now we can easily know if the page is "freezable" by querying the DB, it will be useful for the next step.

-4 now we will be able to modify "wb/admin/pages/index.php" and to give user some controls to "freeze pages, for each line on the pages list we will have 3 possibilities:

   - freezable is set to false -> just display the line.

   - freezable is set to true and frozen is set to true -> display an icon to show that this page is frozen, if u want give the user ability to manually "unfreeze" pages u can even display a link to (warm_page($page_id,$section_id)) (explained later).

   - freezable is set to true and frozen is set to false -> display a button or a link (to freeze the page) to a framework function (freeze_page($page_id,$section_id)) which will do the followings actions:
   -crawl only the good pages (url of the links must match with page_id, section_id and be limited to the url of the website), each matching link must be stored in an array[] alongside with the corresponding links to the static generated html page

   -for each matching page, generate html and store it were you decided.
   -when the "site crawling" is over rewrite the "frozen" php page (the file in pages directory) in a such way that it just hardlink the corresponding html page (include with an array[] to redirect the links to the static pages).
   -in the DB update the field "frozen" to true.

-5 we must make an another function (warm_page($page_id,$section_id)) which test if the page is frozen and then restore the php module page to its normal state (like when the page is generated by "wb/admin/pages/add.php") and set field frozen to false, this function will be called everytime the user modify the content of a "freezable module", the module developper will take care of it by calling the function everytime there's an action which modify the content displayed by the module.

Notice : this may help http://www.phpclasses.org/package/2253-PHP-Crawl-a-site-following-and-retrieving-linked-pages.html


What do u think about this idea ?
I'm sure this system can dramatically the speed in WebsiteBaker.

Here is my opinion on my "freezing page" concept :

Pros :
- 100%  compatible : it can be implemented into the core and WB and all the existings addons wil continue to work.
- once frozen a frozen page use less ressources than a regular page (don't have to check if cache must be freshen every time the page is displayed).
- can be easily implemented into existings modules (you can start with wysiwyg and news module to test it!).
- WB will be the "speed of light" CMS : with this system on a website where all the pages are frozen you can virtually have none sql query nor file access while displaying pages (there are no caching system i know with such features).
Cons :
- if they wants their modules being "freezables", modules developpers will have to rework (slightly) their modules.
- modules developpers must be clever to understand how and when use freezing feature.
- can be tricky to code (i think to "website crawler" script.
- can be "ressource hungry" during crawling for writing the frozen pages.

[chio]

I have some sites where I implemented a caching function.

It works simple:
There is a directory /cms where the wb-pages are, and a directory /site (or something) where the full pages are saved as static html. (real static, with static http-header also)
Each WB page has a "publish page"-button, which calls a script, that saves the complete page "as is" (but of course without this button) in the static-pages directory.
And theres another button to publish the whole site; this is necessary if there are changes in the menu structure.

The aim of this is to have more security and mistakes while editing the content are not visible immediatly in the frontend. The problems: A lot of modules dont work anymore, eg. Forms. And: editing people are sometimes confused about what page they see - the cache or the real pages.

Your "freezing pages" would be a better solution, for there would be a mix of static and dynamic pages, and I also thought about a similar solution, but there are still some problems, especially with the menue. But if you dont cache the whole page (with the menue also), the speed-up effects are small.

And: The difference between opening a static page and opening a WB-Page is 0.2 seconds. (if the server isnt too slow)

[quinto]

I have some sites where I implemented a caching function.

It works simple:
There is a directory /cms where the wb-pages are, and a directory /site (or something) where the full pages are saved as static html. (real static, with static http-header also)
Each WB page has a "publish page"-button, which calls a script, that saves the complete page "as is" (but of course without this button) in the static-pages directory.
And theres another button to publish the whole site; this is necessary if there are changes in the menu structure.

The aim of this is to have more security and mistakes while editing the content are not visible immediatly in the frontend. The problems: A lot of modules dont work anymore, eg. Forms. And: editing people are sometimes confused about what page they see - the cache or the real pages.

Your "freezing pages" would be a better solution, for there would be a mix of static and dynamic pages, and I also thought about a similar solution, but there are still some problems, especially with the menue. But if you dont cache the whole page (with the menue also), the speed-up effects are small.

And: The difference between opening a static page and opening a WB-Page is 0.2 seconds. (if the server isnt too slow)

For the menus i don't think this would be a big problem if the "spider" script is well tuned, the ideal case would be to implement some "caching levels" in the function (freeze_page($page_id,$section_id,$caching_level)).

Where you would have the choice to cache the whole page, or just the page content without Droplets or even think about a solution where you could specify if a droplet is "cacheable" or not.

The aim of this would be to reduce the database access of WebsiteBaker.

But i still think the "freezing page" feature would be a great improvement for our favorite CMS, and like WebsiteBaker is like no other CMS, i prefer use a caching solution like no others

For the moment i'm working on a big module (i don't know if it will be open-source or not for the moment, but i hope i can give it to the others people), in one or two month i will have the time to test "freezine page".