Differences

This shows you the differences between two versions of the page.

--- wiki:tips:nls [2007-06-25 19:04] – mfh
+++ tips:nls [2013-01-12 01:39] (current) – [(Unofficial) National Language Support (NLS)] Klap-in
@@ Line 1: / Line 1: @@
+====== (Unofficial) National Language Support (NLS) ======
+The current version is **2005-09-22e-1.1.0**.  You can find
+  * [[http://www.reeseo.net/misc/dokuwiki_nls/index.en|list of the previous versions]],
+  * working examples, and
+  * the latest update (if any)
+of this technique in [[http://www.reeseo.net/|my web site]], though you cannot modify pages in my site.
+Please discuss here if you have any comment.
+----
+The National Language Support (NLS) feature introduced in this document
+is not a built-in functionality nor an official plugin of DokuWiki,
+but an unofficial patch applied to the 2005-09-22e version.
+A future version of DokuWiki may officially ship another NLS feature
+which is irrelevant to one described in this document.
+Please look for plugins or consult [[tips:browserlanguagedetection|Browser Language Detection]]
+for more information about NLS in DokuWiki.
+===== Suitable Situation =====
+The technique introduced in this document is suitable for situations such that:
+  * You cannot add or change the domain name and host name of your web site arbitrarilly.  This is, for example, you cannot use ''**en**.wikipedia.org'' or ''**ko**.wikipedia.org'' according to the translating language as in the [[http://www.wikipedia.org|WikiPedia]].
+  * You want to operate only one installation of DokuWiki because of convenience of the user management and so on.  In other word, you don't want to operate two or more DokuWiki installations according to the languages of UI's or translations.
+  * Each document in your site will be translated into various languages, where the set of translation languages need not be the same for every document.  For example, a document ''foo'' will be translated into three languages A, B, and C, whereas another document ''bar'' will be translated into four languages B, C, D, and E.  Furthermore, yet another document ''baz'' will be only in one language, C.
+===== Features (as visitors) =====
+The technique in this page works as follows:
+  * Readers visiting your web site can freely choose any existing translations for the documents they read.  If they don't specify any language explicitly, they will be redirected to appropriate translations according to the language settings of their web browsers.
+  * User Interface (UI) language is decided only by the language setting of the reader's web browser, independently from the translation language chosen.  (If no language in the browser's setting is prepared by the DokuWiki installation, the default language configured by the site administrator will be used.)
+  * The documents translated in different languages have different URL's, and readers can move from one translation to another not by submitting forms but via simple links.  Therefore, every existing translations will be recorded  to search engines properly.((This is only for the content translations.  UI language has nothing to do with URL, and web crawlers will take only one language according to their own settings or site default.))
+===== Prerequisites (as administrators and users) =====
+In order to use the technique introduced in this document,
+the following conditions are required:((Some of these may change or disappear
+in a future version (1.2.x or above) of this technique.))
+  * DokuWiki version 2005-09-22e is needed.  (Other versions may work, though not tested.)
+  * For ALL documents in the wiki site to which this technique is about to be applied, the 'translation flags' which denote what language they are translated in should be appended at the ends of their ID's.  You will not be able to read any document without a translation flag after this techique is applied, and **should append flags to all the documents** in your site before applying this technique.
+The 'translation flag' is a kind of language code that is appended at the end of page ID,
+which denotes in what language it is translated.
+When //pageID// is the valid ID format (including namespaces) permitted by DokuWiki,
+the new page ID with a translation flag should be
+//pageID//''\.[a-z][a-z](-[a-z][a-z])?''
+in regular expression.
+For example, ''foo:bar.ko'' is a Korean translation whereas ''foo:bar.en'' is an English translation
+for the same content.
+Because the page ID ''foo:bar'' contains no translation flag,
+readers will be redirected to an appropriate translation pages according to their browser setting
+if they query this kind of page.
+Though they will be redirected basically to the most appropriate one
+among existing translations as far as possible,
+they will be redirected to a non-existing page if no existing translation matches to their browser's setting.
+In other words, readers will not be able to read ''foo:bar'' any more,
+therefore the site administrator should append translation flags to all the pages in his/her site
+before using this technique.
+A method I used and recommend for appending translation flags to pre-existings page is:
+  - Check in what language a pre-existing page ''foo:bar'' is written.  -> Let's assume that this page is written in ''zz'' language.
+  - Create a new page ''foo:bar.zz'' copying the content of ''foo:bar'' verbatim.
+  - Modify links ID's of all the internal links in the copied content appropiately.
+  - Delete ''foo:bar''.
+This procedure, of course, is very tedious, but I don't know better way to change page ID's in DokuWiki.
+===== Algorithm (as developers) =====
+The algorithm is very simple:
+  - A reader query a page.
+  - If the queried page has no translation flag (e.g. ''foo:bar'') and the reader is not doing indexing
+    - Extract languages from the browser setting and sort them by their priority.
+    - If one of these language (e.g. ''ko'') matches an existing translation page, redirect the reader to that page (e.g. ''foo:bar.ko'').
+    - If none matches, redirect him/her to the default translation page configured by the administrator (e.g. ''foo:bar.{$conf['lang']}'') regardless of whether such page exists or not.
+  - If the queried page has a translation flag, show it to the reader.
+  - Aumatically choose an appropriate language for UI by comparing browser settings and ''DOKU_INC/inc/lang/*'' in the DokuWiki installation.  (If none matches, choose ''$conf['lang']''.)
+  - Overwrite chosen language for UI on ''$conf['lang']''.
+  - Reset locale.
+  - Provide API in order for administrators to be able to add a menu (choosing other translations) to their template.
+That's all.  :-)
+===== Code and Usage =====
+Please remind again that you should append translation flags to all the page ID's before using this.
+==== Creating inc/NLS.php File ====
+Create ''inc/NLS.php'' with the following code
+(Administrators can modify two arrays ''$NLS_locarr'' and ''$NLS_langname'' in the first part of this code):
+<code php>
+<?php
+/**
+ * National Language Support (NLS) script for DokuWiki
+ *
+ * @version    2005-09-22e-1.1.0
+ * @license    GPL 2 (http://www.gnu.org/licenses/gpl.html)
+ * @author     CHA Reeseo <http://www.reeseo.net/>
+ *
+ * Usage:
+ *     Include this script into 'doku.php',
+ *     between the inclusions of 'inc/pageutils.php' and 'inc/html.php'
+ *
+ * CAUTION:
+ *     This script demands a special policy that EVERY page should denote
+ *     its own language (.xx or .xx-xx) at the end of its ID.
+ *     For example, 'foo:bar.ko' is a page translated into Korean
+ *     and 'foo:bar.en' is a page containing the same content in English.
+ *
+ *     Quering any page having ID without explicit language notation,
+ *     you will be redirected to its appropriate 'localized' page.
+ *     ('Unlocalized' page will be invisible.)
+ *
+ *     Please rename all the pre-existing pages before applying this script.
+ *
+ * To do:
+ *     - Upgrade NLS_locale function
+ */
+/* ---------------------------------------------------------------
+ * Configuration options: You can modify or add something to these */
+$NLS_locarr['ko'] = 'ko_KR';
+$NLS_locarr['en'] = 'en_US';
+$NLS_langname['ko'] = '한국어';
+$NLS_langname['en'] = 'English';
+/* Configuration options end
+ * --------------------------------------------------------------- */
+// Setting $ID
+$ID = getID();
+// Redirecting if no language is specified at the end of $ID
+if (! NLS_pagelang($ID)) {
+        $target = NLS_page4browser($ID);
+        if (! array_key_exists("idx", $_GET) && $_GET['do'] != 'recent')  // No redirection when indexing
+                header('Location: ' . wl($target));
+}
+// Resetting $conf['lang'] according to the language setting of user's browser
+$conf['lang'] = NLS_UI4browser();
+// Resetting $lang array
+@require_once(DOKU_INC.'inc/lang/'.$conf['lang'].'/lang.php');
+// Resetting locale
+setlocale(LC_ALL, NLS_locale($conf['lang']));
+/**
+ * Getting locale string
+ *
+ * FIXME: What a poor function this is!
+ */
+function NLS_locale($ln = NULL) {
+        global $conf;
+        global $lang;
+        global $NLS_locarr;
+        if (! $ln) $ln = $conf['lang'];
+        $loc = array_key_exists($ln, $NLS_locarr) ? $NLS_locarr[$ln] : $ln;
+        $loc .= '.';
+        $loc .= array_key_exists('encoding', $lang) ? strtoupper($lang['encoding']) : 'UTF-8';
+        return $loc;
+}
+/**
+ * Printing links to other translations of the given page
+ * (API for template files such as DOKU_TPL/main.php
+ */
+function NLS_transmenu($pid = NULL, $delimiter = ",\n", $withself = FALSE) {
+        global $NLS_langname;
+        if (! $pid) {
+                global $ID;
+                $pid = $ID;
+        }
+        $currplang = NLS_pagelang($pid);
+        $tpages = NLS_transpages($pid);
+        if ($currplang && ! $withself)
+                unset($tpages[$currplang]);
+        if ($tpages) {
+                $first = TRUE;
+                foreach ($tpages as $ln => $tid) {
+                        $repr = array_key_exists($ln, $NLS_langname) ? $NLS_langname[$ln] : $ln;
+                        if (! $first)
+                                echo $delimiter;
+                        $first = FALSE;
+                        if ($currplang == $ln)
+                                echo "<em>$repr</em>";
+                        else
+                                echo "<a href=\"".wl($tid)."\">$repr</a>";
+                }
+        } else {
+                echo "None.";
+        }
+}
+/**
+ * Selecting a redirection target (page) which best match the browser setting
+ *
+ * Default: page of $conf['lang'] (whether it exists or not)
+ * Choice : among existing translations, highest priority for the browser
+ */
+function NLS_page4browser($pid = NULL) {
+        if (! $pid) {
+                global $ID;
+                $pid = $ID;
+        }
+        $blang = NLS_browserlang();
+        $existing_pages = NLS_transpages($pid);
+        $pid_base = NLS_ID_base($pid);
+        $tmp_page = $pid_base . '.' . $conf['lang'];    // default page
+        foreach ($blang as $lang_str => $priority) {
+                $lang_str = str_replace("_", "-", strtolower($lang_str));
+                if (array_key_exists($lang_str, $existing_pages))
+                        $tmp_page = $pid_base . '.' . $lang_str;
+                elseif (array_key_exists(substr($lang_str, 0, 2), $existing_pages))
+                        $tmp_page = $pid_base . '.' . substr($lang_str, 0, 2);
+        }
+        return $tmp_page;
+}
+/**
+ * Selecting a language for UI
+ *
+ * Default: $conf['lang']
+ * Choice : among existing 'inc/lang/*', highest priority for the browser
+ */
+function NLS_UI4browser() {
+        global $conf;
+        $tmp_lang = $conf['lang'];      // This ($conf['lang']) is the default!
+        $blang = NLS_browserlang();
+        foreach ($blang as $lang_str => $priority) {
+                $lang_str = str_replace("_", "-", strtolower($lang_str));
+                if (is_dir(DOKU_INC . "inc/lang/" . $lang_str))
+                        $tmp_lang = $lang_str;
+                elseif (is_dir(DOKU_INC . "inc/lang/" . substr($lang_str, 0, 2)))
+                        $tmp_lang = substr($lang_str, 0, 2);
+        }
+        return $tmp_lang;
+}
+/**
+ * Getting a sorted (by priority) array of the languages
+ * from the language setting of the user's web browser
+ */
+function NLS_browserlang() {
+        $acclang_arr = split(" *, *", trim($_SERVER['HTTP_ACCEPT_LANGUAGE']));
+        foreach ($acclang_arr as $acclang) {
+                if (ereg("^(.+) *;.+= *(.+)$", $acclang, $acclang_parts))
+                        $acclang_sorted[$acclang_parts[1]] = (double)($acclang_parts[2]);
+                else
+                        $acclang_sorted[$acclang] = 1.0;
+        }
+        asort($acclang_sorted, SORT_NUMERIC);
+        reset($acclang_sorted);
+        return $acclang_sorted;
+}
+/**
+ * Getting an associative array (lang => ID) of
+ * all the translated pages for the given page
+ */
+function NLS_transpages($pid = NULL) {
+        global $conf;
+        if (! $pid) {
+                global $ID;
+                $pid = $ID;
+        }
+        $transpages = array();
+        $pid_base_path = $conf['datadir'] . '/' . str_replace(":", "/", NLS_ID_base($pid));
+        foreach(glob($pid_base_path . ".*.txt") as $fn) {
+                $aid = str_replace("/", ":", substr($fn, strlen($conf['datadir']) + 1, -4));
+                if ($alang = NLS_pagelang($aid)) {
+                        $transpages[$alang] = $aid;
+                }
+        }
+        return $transpages;
+}
+/**
+ * Drop language notation (.xx or .xx-xx) from the given ID
+ */
+function NLS_ID_base($pid = NULL) {
+        if (! $pid) {
+                global $ID;
+                $pid = $ID;
+        }
+        if (substr($pid, -3, 1) == '.')
+                return substr($pid, 0, -3);
+        elseif (substr($pid, -6, 1) == '.' && substr($pid, -3, 1) == '-')
+                return substr($pid, 0, -6);
+        else
+                return $pid;
+}
+/**
+ * Get language (xx, xx-xx, or NULL) from the given ID
+ */
+function NLS_pagelang($pid = NULL) {
+        if (! $pid) {
+                global $ID;
+                $pid = $ID;
+        }
+        if (substr($pid, -3, 1) == '.')
+                return substr($pid, -2);
+        elseif (substr($pid, -6, 1) == '.' && substr($pid, -3, 1) == '-')
+                return substr($pid, -5);
+        else
+                return NULL;
+}
+?>
+</code>
+==== Modifying doku.php File ====
+Now, let's add an inclusion of ''inc/NLS.php'' at the beginning part of ''doku.php''.
+**The location of this inclusion is very important!**
+Though NLS script should be loaded as early as possible,
+it should be placed after ''inc/pageutils.php'' because NLS script uses it.
+Please insert the line of inclusion just after including ''inc/pageutils.php''
+(This code shows only the first few lines):
+<code php>
+<?php
+/**
+ * DokuWiki mainscript
+ *
+ * @license    GPL 2 (http://www.gnu.org/licenses/gpl.html)
+ * @author     Andreas Gohr <andi@splitbrain.org>
+ */
+//  xdebug_start_profiling();
+  if(!defined('DOKU_INC')) define('DOKU_INC',realpath(dirname(__FILE__)).'/');
+  require_once(DOKU_INC.'inc/init.php');
+  require_once(DOKU_INC.'inc/common.php');
+  require_once(DOKU_INC.'inc/pageutils.php');
+  require_once(DOKU_INC.'inc/NLS.php');       // insert this line!
+  require_once(DOKU_INC.'inc/html.php');
+  require_once(DOKU_INC.'inc/auth.php');
+  require_once(DOKU_INC.'inc/actions.php');
+  //import variables
+...
+</code>
+==== Adding Menu (for Selecting Translations) to the Template ====
+For the last, let's add a menu for selecting translations to the template
+in order for the readers to choose another translations of current page.
+Just insert the following code into the ''main.php'' or so in the template you are using:
+<code php>
+Other translations of this page: <?php NLS_transmenu(); ?>
+</code>
+''NLS_transmenu()'' function is an API which generate a menu to the template
+and defines following parameters:
+''NLS_transmenu(//$pid//, //$delimiter//, //$withself//)''
+Each parameter is:
+  * string ''//$pid//'': The function generate a list of translation of the document which is identified by this variable.  The default is DokuWiki's global variable ''$ID'', that is, the current document.
+  * string ''//$delimiter//'': If there are more than one translations, they will be separated by the delimiter assigned to this variable.  The default value is ''%%",\n"%%''.
+  * boolean ''//$withself//'': Menu will include the language of the ''$pid'' document itself if the value assigned to this variable is ''True'', and will exclude it if the value is ''False''.  The default is ''False''.
+===== To do =====
+  * As written in the code above as a comment, ''NLS_locale()'' function sucks!  :-(  An upgrade is seriously needed and it's the final goal of the version 1.1.x series.
+===== Discussion =====
+I like this concept. In reading through the dokuwiki site I see that namespaces are/can also be used. For reference please check [[multilingual_content]] and also [[multilingual_content:local.php]]. At the bottom of the **local.php** discussion Stéphane Gully shows how he made [[http://www.pxxo.net/]] with french and english translations and graphic language selectors. The end result seems very similar to your site [[http://www.reeseo.net/home.en]].
+I would love to see these two approaches merged...
+  * The automatic creation of mirror documents in the first language in the alternate languages with a to be translated tag - as you suggest in this article.
+  * The link between the documents is automatic (as in your site) and just clicking on a language will show the translated doc in the right place - or show that it has not yet been done...as in both of your approaches.
+My main concern is the document naming convention. You suggest to add the language code at the end of your page name **foo:bar.ko** whereas the multilingual discussion suggests the creation of a language namespace at the front end **ko:foo:bar**.
+:?: What are the pro's and con's of these two approaches?  - //Tito Vergara 12/01/06//
+----
+I like the present approach better than the multilang. content which adds the "lang:" in a too prominent place (IMHO) - in particular if a sidebar nav.menu (e.g. roundbox..) displays the namespace components as folders. Also since availability of a translation may rather depend on the given page, and some pages might not need translation.
+OTOH, I'd like it to be even more discrete and at the same time "intelligent". What about the following idea:
+  * get $LANG from REQUEST or SESSION (see below)
+  * if foo:bar is requested, check if foo:bar.LANG exists, then display it (without showing the extension in the URL), else display foo:bar (if exists) (or foo:bar.$DEFAULTLANG)
+There are several advantages of the URLs not containing the .LANG : esthetical, portability, readability,...
+Setting the $LANG (SESSION or COOKIE or GET) would be done
+  * from GET (i.e. if "&LANG=xx" is added to the URL)
+  * if avail, from the last click on a "lang. flag" or similar explicit choice
+  * from browser setting
+  * site default (questionable if this should set the LANG or leave it empty, which might be not equivalent)
+--- //MFH, 25.6.07//