Differences

This shows you the differences between two versions of the page.

--- teams:i18n:translation-check [2018-07-27 18:51] – Automatic update: no problem found. schplrtz.au
+++ teams:i18n:translation-check [2023-11-05 09:16] (current) – more than 2.5 year with nothing to report. DW translation has checks too. This page and the checks are no more needed. Stopping my daily checks. schplurtz
@@ Line 1: / Line 1: @@
-====== DokuWiki PHP translation checks =====
-This page shows the result of checks on translation strings. The results
-are presented language by language. All languages are tested but only those with troubles are reported.
-Every day, the test is run. This page is updated only when there is something new to report.
-Last report date : 2018-07-27T16:51:35+00:00
-**Explanation for translators:**
-DokuWiki sometimes uses a function called [[phpfn>sprintf]] to insert certain words into a given language string to complete a sentence. Where a word (or number) has to be inserted is determined by a so called format string. Format strings start with a percent sign, optionally followed by numbers and dots and end with a single letter.
-The most common format string is a simple ''%s'' for inserting a string. A ''%d'' indicates a decimal number and ''%.2f'' will insert a floating point number rounded to 2 digits after the decimal point.
-When translating a string it is important to **leave the format strings intact**. They have to appear exactly as in the original. If a string contains several format strings, **order does matter**. Make sure the logical order of words or numbers to be inserted are the same as in the original.
-===== summary =====
-No problem found.
-Congratulation to all contributors !
-------------------
-Report generated in 1.4139678478241 seconds.
-===== technical details =====
-  * percent test 1\\ This test checks that there are as many ''%format'' in translation as in original english. many small presentation problems may arise when the the number of % sign differs. This test also catches typos where, say, ''%s'' is mis typed as ''s%'' in translation.
-  * non existing arg test\\ Translators may change the order of argument using ''%position$format'', so this test checks that, if there are n argument in original string, all eventual position in translation is in 1..n range.
-  * percent test 2\\ checks that each %thing in translation has a corresponding %samething in english, in the correct order and that each %thing in english has a corresponding %samething in translation.
-  * supernumerary translations\\ This test detects 3 possible errors :
-    - Old translations that remain in locale files but that are no more needed. Those unneeded strings are just useless, not harmful at all. They should be removed anyway.
-    - Typo in the translation key name. For example, if a translator types ''$lang['btn_cr**ae**te']='some text';'' instead of ''$lang['btn_cr**ea**te']='some text';'', then ''btn_craete'' will be reported as supernumerary.
-    - Misplaced translation key. For example ''$lang['notsavedyet']'' instead of ''$lang['js']['notsavedyet']'', in which case ''$lang['notsavedyet']'' is reported as supernumerary, and [[http://translate.dokuwiki.org/|DokuWiki translation tool]] reports ''$lang['js']['notsavedyet']'' as untranslated.
-===== script code =====
-This page was produced with the following php code. This page is automagically updated by an ugly daily cron job of mine. Any change in the script below will stop the automatic updates.
-<code php checklanguage.php>
-#! /usr/bin/php5
-<?php
-/*
- * checklanguage.php simple checks on english dokuwiki texts and their translations
- * Copyright (C) 2011, 2013 Schplurtz le Déboulonné <schplurtz laposte.net>
- *
- *     This program is free software: you can redistribute it and/or modify
- *    it under the terms of the GNU General Public License as published by
- *    the Free Software Foundation, either version 3 of the License, or
- *    (at your option) any later version.
- *
- *    This program is distributed in the hope that it will be useful,
- *    but WITHOUT ANY WARRANTY; without even the implied warranty of
- *    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- *    GNU General Public License for more details.
- *
- *    You should have received a copy of the GNU General Public License
- *    along with this program.  If not, see http://www.gnu.org/licenses/.
- */
-/*
- This script runs some automated tests on dokuwiki translation strings.
- It checks printf format strings. Current code is a mess but detects
- many problems.
- /schplurtz
-*/
-# ADJUST TO THE BASE OF YOUR DW. no other thing need be changed...
-$base_english=dirname(__FILE__);
-$base_tr=$base_english;
-$vu=array( 'lang' => array(), 'file' => array(), 'pre' => array() );
-$debug_log = false; // if true, get log and function call in /tmp/log. function l()
-$timestart=0;
-if( function_exists( 'microtime' )) $timestart=microtime(1);
-# get all langs
-exec( 'cd '.$base_english.'/inc/lang && ls', $langdirs );
-# get all english PHP language files
-exec( 'cd '.$base_english.' && find . -name "*.php" | grep lang/en/', $english_files );
-pageheader();
-$npbfound=0;
-foreach( $langdirs as $language ) {
-    l( "checking language $language" );
-    foreach( $english_files as $en_fi ) {
-        l( "checking translation for $en_fi" );
-        $fen = $base_english . '/' . $en_fi;
-        $tr_fi = str_replace( 'lang/en/', 'lang/'.$language.'/', $en_fi );
-        $ftr = $base_tr . '/' . $tr_fi;
-        if( !file_exists( $ftr ) )
-            /* nothing to check if no translation exists */
-            continue;
-        l( "found file $ftr" );
-        $lang=array();
-        include( $fen );
-        l( "loaded original english " . count( $lang ) . " strings" );
-        $org=$lang;
-        include( $ftr );
-        l( "loaded translation      " . count( $lang ) . " strings" );
-        $npbfound += chklang( $org, $lang, $en_fi, $tr_fi );
-    }
-}
-summary( $npbfound );
-pagefooter();
-function chklang( &$en, &$tr, $prenfi, $prtrfi, $strkeys='' ) {
-    l( "chklang $prenfi, $prtrfi, '$strkeys'" );
-    $ntrouble=0;
-    global $language;
-    $vukeys=array();
-    foreach( $en as $k =>$v ) {
-        if( is_array( $v ) ) {
-            $ntrouble += chklang( $v, $tr[$k], $prenfi, $prtrfi, $strkeys.'['.$k.']' );
-            continue;
-        }
-        // FIXME: preg_match english once and cache result in global array ?
-        // Can't use substr_count( $v, % ). It could match %thing that are not format.
-        preg_match_all( '/%(?:([0-9]+)\$)?([-.0-9hl]*?[%dufsc])/', $v, $nen );
-        preg_match_all( '/%(?:([0-9]+)\$)?([-.0-9hl]*?[%dufsc])/', $tr[$k], $ntr );
-        /*
-         * we have the full %thing in $nen[0] and $ntr[0],
-         *         the eventual ordering part in $nen[1] and $ntr[1],
-         *         just the format part in $nen[2] and $ntr[2].
-         * $ntr[0] is necessarily an array : if no % in translation, then test 1
-         * detects the situation and we don't get here...
-         * example
-         * Given this string :
-         *         schproutch %2$s with %1$04d in %-20s plouf
-         * we have this in $ntr array.
-         * Array
-         * (
-         *     [0] => Array
-         *         (
-         *             [0] => %2$s
-         *             [1] => %1$04d
-         *             [2] => %-20s
-         *         )
-         *     [1] => Array
-         *         (
-         *             [0] => 2
-         *             [1] => 1
-         *             [2] =>
-         *         )
-         *     [2] => Array
-         *         (
-         *             [0] => s
-         *             [1] => 04d
-         *             [2] => -20s
-         *         )
-         *
-         * )
-         */
-        // test number 1 : same count of % sign in both strings
-        $pen = count( $nen[0] );
-        $ptr = count( $ntr[0] );
-        if( $pen != $ptr ) {
-            prestart($language, $prenfi, $prtrfi);
-            echo
-                 ($pen - $ptr) < 0 ? '- ' : '+ ',
-                 '$lang',$strkeys,'[',$k,']=',$v,"\n",
-                 ($pen - $ptr) > 0 ? '- ' : '+ ',
-                 '  $tr',$strkeys,'[',$k,']=',$tr[$k],"\n";
-            $ntrouble++;
-            continue;
-        }
-        // no need to continue if no % in English string.
-        if( !$pen )
-            continue;
-        // Since order can be set by translator, set order for each format. we'll need that
-        order_fmt( $nen );
-        order_fmt( $ntr );
-        $expectargen=max( $nen[1] );
-        $expectargtr=max( $ntr[1] );
-        // and test 2 : check translator don't want to print a non existing arg.
-        if( $expectargtr > $expectargen ) {
-            prestart($language, $prenfi, $prtrfi);
-            headkeypb( $vukeys, $strkeys, $k, $v, $tr[$k] );
-            echo
-                 '  # ',
-                 '$lang',$strkeys,'[',$k,'] : ';
-            printf( "%4d '%%'\n", $expectargen );
-            foreach( $ntr[1] as $k2 => $v2 ) {
-               if( $v2 > $expectargen ) {
-                   echo '  > ',
-                        '  $tr',$strkeys,'[',$k,'] : ';
-                   printf( "%8s\n", $ntr[0][$k2] );
-                   $ntrouble++;
-               }
-            }
-            continue;
-        }
-        if( min( $ntr[1] ) < 1 ) { //weird. Should never happen
-            prestart($language, $prenfi, $prtrfi);
-            headkeypb( $vukeys, $strkeys, $k, $v, $tr[$k] );
-            echo
-                 '  # ',
-                 '$lang',$strkeys,'[',$k,'] : ';
-             printf( "%4d '%%'\n", $expectargen );
-             foreach( $ntr[1] as $k2 => $v2 ) {
-                if( $v2 < 1 ) {
-                    echo '  < ',
-                         '  $tr',$strkeys,'[',$k,'] : ';
-                    printf( "%8s\n", $ntr[0][$k2] );
-                    $ntrouble++;
-                }
-             }
-            continue;
-        }
-        // test #? ... Test all args are used :
-        /* '%3s %2s' => arg 1 never printed... */
-        foreach( range( 1, $expectargen ) as $argnum ) {
-            if( !array_key_exists( $argnum, $ntr[3] ) ) {
-                prestart($language, $prenfi, $prtrfi);
-                headkeypb( $vukeys, $strkeys, $k, $v, $tr[$k] );
-                echo '  ? arg ', $argnum, ' not used in ', $language, "\n";
-            }
-        }
-        // test 3 check that each %thing in translation has a matching %samething in original
-        /* FIXME: undefined indexes when english is "%s %d" and tr is "%1$s %1$s" */
-        $orderpb=false;
-        foreach( $ntr[1] as $i2 => $v2 ) {
-            if( !array_key_exists( $ntr[1][$i2], $nen[3] ) )
-                continue;
-            if( $ntr[2][$i2] != $nen[2][$nen[3][$ntr[1][$i2]]] ) {
-                $ntrouble++;
-                $orderpb=true;
-                prestart($language, $prenfi, $prtrfi);
-                headkeypb( $vukeys, $strkeys, $k, $v, $tr[$k] );
-                echo '  = ', $ntr[2][$i2][strlen($ntr[2][$i2])-1], ' : ', $ntr[0][$i2], "\n",
-                     '  ≠ ', $nen[2][$nen[3][$ntr[1][$i2]]][strlen($nen[2][$i2])-1], ' : ', $nen[0][$nen[3][$ntr[1][$i2]]], "\n";
-            }
-        }
-        if( $orderpb )
-            continue;
-        // and vice versa
-        foreach( $nen[1] as $i2 => $v2 ) {
-            if( !array_key_exists( $nen[1][$i2], $ntr[3] ) )
-                continue;
-            if( $nen[2][$i2] != $ntr[2][$ntr[3][$nen[1][$i2]]] ) {
-                $ntrouble++;
-                $orderpb=true;
-                prestart($language, $prenfi, $prtrfi);
-                headkeypb( $vukeys, $strkeys, $k, $v, $tr[$k] );
-                echo '  ≠ ', $nen[2][$i2][strlen($nen[2][$i2])-1], ' : ', $nen[0][$i2], "\n",
-                     '  = ', $ntr[2][$ntr[3][$nen[1][$i2]]][strlen($ntr[2][$i2])-1], ' : ', $ntr[0][$ntr[3][$nen[1][$i2]]], "\n";
-            }
-        }
-        if( $orderpb )
-            continue;
-    }
-    // things to do only once per language file
-    if( '' == $strkeys ) {
-        // test n : supernumerary translations
-        if( count( $diff=array_diff_key( $tr, $en ) ) ) {
-                preend($language, $prenfi, $prtrfi);
-                fileheader($language, $prenfi, $prtrfi);
-                echo "Those translations are present in $language but not in original English\n";
-                prestart($language, $prenfi, $prtrfi);
-                foreach( $diff as $k => $v ) {
-                    echo '$lang',$strkeys,'[',$k,']=',$v,"\n";
-                    $ntrouble++;
-                }
-                preend($language, $prenfi, $prtrfi);
-        }
-        filefooter( $language, $prenfi, $prtrfi);
-    }
-    l( "leaving chklang $prenfi, $prtrfi, '$strkeys'" );
-    return $ntrouble;
-}
-function headkeypb( &$vukeys, $strkeys, $k, $ven, $vtr ) {
-    if( array_key_exists( $k, $vukeys ) )
-        return;
-    $vukeys[$k]=1;
-    echo
-         '= ',
-         '$lang',$strkeys,'[',$k,']=',$ven,"\n",
-         '= ',
-         '  $tr',$strkeys,'[',$k,']=',$vtr,"\n";
-}
-function summary( $ntrouble ) {
-    global $vu, $timestart;
-    echo "\n===== summary =====\n\n";
-    if( $ntrouble ) {
-        $nlang = count( $vu['lang'] );
-        printf( "There %s %d problem%s in %d language%s.\n",
-                ($ntrouble > 1) ? 'were' : 'was',
-                $ntrouble,
-                ($ntrouble > 1) ? 's' : '',
-                $nlang,
-                ($nlang > 1) ? 's' : ''
-        );
-    }
-    else {
-        echo 'No problem found.', "\n\n",
-             'Congratulation to all contributors !', "\n";
-        echo "\n";
-    }
-    if ($timestart) {
-        echo "\n------------------\n";
-        echo "Report generated in ", microtime(1) - $timestart, " seconds.\n";
-    }
-}
-function pageheader() {
-?>
-====== DokuWiki PHP translation checks =====
-This page shows the result of checks on translation strings. The results
-are presented language by language. All languages are tested but only those with troubles are reported.
-Every day, the test is run. This page is updated only when there is something new to report.
-Last report date : <?php echo date( 'c' ),"\n"; ?>
-**Explanation for translators:**
-DokuWiki sometimes uses a function called [[phpfn>sprintf]] to insert certain words into a given language string to complete a sentence. Where a word (or number) has to be inserted is determined by a so called format string. Format strings start with a percent sign, optionally followed by numbers and dots and end with a single letter.
-The most common format string is a simple ''%s'' for inserting a string. A ''%d'' indicates a decimal number and ''%.2f'' will insert a floating point number rounded to 2 digits after the decimal point.
-When translating a string it is important to **leave the format strings intact**. They have to appear exactly as in the original. If a string contains several format strings, **order does matter**. Make sure the logical order of words or numbers to be inserted are the same as in the original.
-<?php
-}
-function pagefooter() {
-?>
-===== technical details =====
-  * percent test 1\\ This test checks that there are as many ''%format'' in translation as in original english. many small presentation problems may arise when the the number of % sign differs. This test also catches typos where, say, ''%s'' is mis typed as ''s%'' in translation.
-  * non existing arg test\\ Translators may change the order of argument using ''%position$format'', so this test checks that, if there are n argument in original string, all eventual position in translation is in 1..n range.
-  * percent test 2\\ checks that each %thing in translation has a corresponding %samething in english, in the correct order and that each %thing in english has a corresponding %samething in translation.
-  * supernumerary translations\\ This test detects 3 possible errors :
-    - Old translations that remain in locale files but that are no more needed. Those unneeded strings are just useless, not harmful at all. They should be removed anyway.
-    - Typo in the translation key name. For example, if a translator types ''$lang['btn_cr**ae**te']='some text';'' instead of ''$lang['btn_cr**ea**te']='some text';'', then ''btn_craete'' will be reported as supernumerary.
-    - Misplaced translation key. For example ''$lang['notsavedyet']'' instead of ''$lang['js']['notsavedyet']'', in which case ''$lang['notsavedyet']'' is reported as supernumerary, and [[http://translate.dokuwiki.org/|DokuWiki translation tool]] reports ''$lang['js']['notsavedyet']'' as untranslated.
-===== script code =====
-This page was produced with the following php code. This page is automagically updated by an ugly daily cron job of mine. Any change in the script below will stop the automatic updates.
-<?php
-/*
- * must break </ code> in 2 pieces otherwise it is not possible to
- * self-include this file in doku page
- */
-echo '<', 'code php checklanguage.php>', "\n";
-readfile( __FILE__ );
-echo '</', 'code>', "\n";
-}
-function langheader( $language ) {
-    l( "langheader $language" );
-    global $vu;
-    if( !array_key_exists( $language, $vu['lang'] )) {
-        $vu['lang'][$language]=1;
-        echo "\n", '===== ', $language, ' =====', "\n";
-    l( "           $language was not seen. printing header" );
-    }
-}
-function fileheader( $language, $en_file, $tr_file ) {
-    global $vu;
-    langheader( $language );
-    l( "fileheader $language $tr_file $en_file" );
-    if( !array_key_exists( $tr_file, $vu['file'] )) {
-        l( "fileheader $language $tr_file not seen. printing header" );
-        $vu['file'][$tr_file] = 1;
-        echo
-            "\n! ",$en_file,"\\\\\n",
-            '! ',$tr_file,"\\\\\n";
-    }
-}
-function prestart( $language, $en_file, $tr_file ) {
-    global $vu;
-    fileheader( $language, $en_file, $tr_file );
-    l( "prestart $language $tr_file $en_file" );
-    if( !array_key_exists( $tr_file, $vu['pre'] )) {
-        l( "prestart $language $tr_file not seen. printing open tag" );
-        $vu['pre'][$tr_file] = 1;
-        echo '<file>';
-    }
-}
-function preend( $language, $en_file, $tr_file ) {
-    global $vu;
-    l( "preend $language $tr_file" );
-    if( array_key_exists( $tr_file, $vu['pre'] )) {
-        echo '<', "/file>\n";
-        l( "       $language $tr_file pre tag open. closing pre" );
-        unset( $vu['pre'][$tr_file] );
-    }
-}
-function filefooter( $language, $en_file, $tr_file ) {
-    global $vu;
-    preend( $language, $en_file, $tr_file );
-    l( "filefooter $language $tr_file" );
-}
-function langfooter() {
-    return;
-}
-function l( $s ) {
-    global $debug_log;
-    if( ! $debug_log ) return;
-    file_put_contents( '/tmp/log', "$s\n", FILE_APPEND );
-}
-/*
- * fills in numbering info of format string.
- * adds a reverse index array so it is easy to find
- * the original format knowing the translated format even
- * when there are printf %order$ thing in both strings
- *
- * printf( '%3$s %2$s %s %s %5$s %s', 1, 2, 3, 4, 5 );
- * => 3 2 1 2 5 3
- */
-function order_fmt( &$arr ) {
-    $ordered=Array();
-    $reverse_id=Array();
-    $autoid=1;
-    for( $i=0; $i < count($arr[1]); ++$i ) {
-        if( empty($arr[1][$i] ))
-            $arr[1][$i] = $autoid++;
-        $reverse_id[$arr[1][$i]]=$i;
-    }
-    $arr[]=$reverse_id;
-}
-</code>