DokuWiki

It's better when it's simple

User Tools

Site Tools


tips:synchronise_with_tomboy

How to synchronise Tomboy with Dokuwiki

I wrote a little script which synchronises Tomboy notes (one-way) with a Dokuwiki installation. This script just a “works for me” solution, but it can be used as a starting point.

Usually, you would upload the DokuWiki files via ftp or whatever you like, but i did it by mounting my webserver via sshfs into my filesystem.

Usage

I set up Ubuntu to run a cronjob every minute calling the script presented below.

This script does the following:

  • it checks if anything has changed in your tomboy notes
  • if it has, it converts the tomboy markup to DokuWiki markup
  • it saves the old DokuWiki page (if it exists) as an old revision, and saves the converted markup as the latest version
  • it marks the entries in the .changes files for the revisions to work properly

Just drop the .php file in any folder you want, give it “chmod +x”, and let cron run it!

* * * * * /home/name/bin/sync_personal_wiki.php

License

You can use this code as you like (public domain), but it would be nice if you could include a little comment on this page, or send me a message to my email address ( arturh at arturh (dot) de)

Code

sync_personal_wiki.php
#!/usr/bin/php
<?php
 
// ----------------------------------------------------------------------------------------
// CONFIGURATION
// ----------------------------------------------------------------------------------------
 
// path to tomboy data directory, ending with /
$tomboy_path = "/home/arturh/.tomboy/";
 
// path where checksums are saved
$checksum_path = $tomboy_path . ".md5";
 
// namespace name in DokuWiki syntax, like "" or "mynamespace" or "mynamespace:myothernamespace"
$dokuwiki_namespace = "my:tomboy";
 
// path to DokuWiki data directory, ending with /
$dokuwiki_datapath = "/home/arturh/htdocs/example.de/wiki/data/";
 
// DokuWiki username which should be displayed as editing user in log
$dokuwiki_username = "ahallman";
 
// title of tomboy pages which should not be synchronized (lower case, blanks replaced with _)
$blacklist = array();
 
// if empty, all tomboy pages which are not in the blacklist are synchronized (syntax like $blacklist)
// if filled, only pages which are in the whitelist will be synchronized
$whitelist = array();
 
// ----------------------------------------------------------------------------------------
// END OF CONFIGURATION (do not edit below this line)
// ----------------------------------------------------------------------------------------
 
$allowedTags = array("bold", "italic", "strikethrough", "highlight", "size:small", "size:large", "size:huge");
 
// read in old md5 values
$checksum = array();
$firstrun = false;
if(file_exists($checksum_path))
{
     $content = file($checksum_path);
     foreach($content AS $line) {
          $l = explode(' ', $line);
          $checksum[$l[0]] = trim($l[1]);
     }
} else $firstrun = true;
 
 
if(!file_exists($dokuwiki_datapath))
	exit("Datapath does not exist!");
 
// sync each tomboy file
if ($handle = opendir($tomboy_path)) {
   while (false !== ($file = readdir($handle))) {
       if ($file[0] != "." && $file != "." && $file != ".." && !is_dir($tomboy_path.$file)) {
           $md5 = md5_file($tomboy_path.$file);
           if($firstrun || ($checksum[$file] && $checksum[$file] != $md5)) {
               sync_page($tomboy_path.$file);
           } 
           $checksum[$file] = $md5;
       }
   }
   closedir($handle);
}
 
// write current md5 values to file
$fp = fopen($checksum_path, "w");
foreach($checksum AS $filename => $md5sum) {
     fwrite($fp, $filename . " " . $md5sum . "\n");
}
fclose($fp);
 
 
// convert from tomboy markup to dokuwiki markup
function convertWikiMarkup($markup) {
     // one little hack: first line of each tomboy page is the h1
     $markup = preg_replace('/^(.*?)'."\n/",'<size:huge>$1</size:huge>'."\n", $markup);
 
     // now simple regexp replaces with tomboy and dokuwiki markup
     $markup = preg_replace('/<bold>(.*)<\\/bold>/','**$1**', $markup);
     $markup = preg_replace('/<italic>(.*)<\\/italic>/','//$1//', $markup);
     $markup = preg_replace('/<strikethrough>(.*)<\\/strikethrough>/','<del>$1</del>', $markup);
     $markup = preg_replace('/<highlight>(.*)<\\/highlight>/','##$1##', $markup);
     $markup = preg_replace('/<size:small>(.*)<\\/size:small>/','(($1))', $markup);
     $markup = preg_replace('/<size:large>(.*)<\\/size:large>/','===== $1 =====', $markup);
     $markup = preg_replace('/<size:huge>(.*)<\\/size:huge>/','====== $1 ======', $markup);
 
     // one last thing: if it is not a hX, we need a "\\ " for creating newlines at each line
 
     $lines = explode("\n", $markup);
 
     for($i=0; $i<count($lines); $i++) {
          $lines[$i] = preg_replace('/^\\* (.*)$/','  * $1', $lines[$i]);
          $lines[$i] = preg_replace('/^\\*\\* (.*)$/','    * $1', $lines[$i]);
     }    
 
     $markup = implode("\n", $lines);
 
 
     return $markup;
}
 
// XML Handler function
function startElement($parser, $name, $attrs)
{
   global $wiki, $allowedTags;
 
   if($name == "NOTE-CONTENT") {
       $wiki["noteContent_search"] = 1;
       $wiki["noteContent"] = "";
   }
 
   if($name == "TITLE") {
       $wiki["title_search"] = 1;
       $wiki["title"] = "";
   }
 
   if(in_array(strtolower($name), $allowedTags)) {
       $wiki["noteContent"] .= "<".strtolower($name).">";
   }
}
 
// XML Handler function
function endElement($parser, $name)
{
   global $wiki, $allowedTags;
 
   if($name == "NOTE-CONTENT" && $wiki["noteContent_search"] == 1) {
      unset($wiki["noteContent_search"]);
      $wiki["noteContent"] = convertWikiMarkup($wiki["noteContent"]);
   }
 
   if($name == "TITLE" && $wiki["title_search"] == 1) {
      unset($wiki["title_search"]);
   }
 
   if(in_array(strtolower($name), $allowedTags)) {
       $wiki["noteContent"] .= "</".strtolower($name).">";
   }
}
 
// XML Handler function
 
function characterData($parser, $data)
{
   global $wiki;
 
   if($wiki["noteContent_search"] == 1) {
      $wiki["noteContent"] .= $data;
   }
 
   if($wiki["title_search"] == 1) {
      $wiki["title"] .= $data;
   }
}
 
function mkdir_recursive($path, $rights = 0777)
{
  echo "mkdir " . $path . "\n";
  $folder_path = array(
   strstr($path, '.') ? dirname($path) : $path);
 
  while(!@is_dir(dirname(end($folder_path)))
         && dirname(end($folder_path)) != '/'
         && dirname(end($folder_path)) != '.'
         && dirname(end($folder_path)) != '')
   array_push($folder_path, dirname(end($folder_path)));
 
  while($parent_folder_path = array_pop($folder_path))
   if(!@mkdir($parent_folder_path, $rights))
     user_error("Can't create folder \"$parent_folder_path\".");
 
  mkdir($path);
}
 
 
// main function which does all the work
function sync_page($path) {
     global $wiki, $dokuwiki_datapath, $dokuwiki_namespace, $dokuwiki_username, $whitelist, $blacklist;
 
     // analyse tomboy's XML page
     $xml_parser = xml_parser_create();
     xml_set_element_handler($xml_parser, "startElement", "endElement");
     xml_set_character_data_handler($xml_parser, "characterData");
     if (!($fp = fopen($path, "r"))) {
        die("could not open XML input");
     }
 
     while ($data = fread($fp, 4096)) {
        if (!xml_parse($xml_parser, $data, feof($fp))) {
            die(sprintf("XML error: %s at line %d",
                        xml_error_string(xml_get_error_code($xml_parser)),
                        xml_get_current_line_number($xml_parser)));
        }
     }
     xml_parser_free($xml_parser);
     fclose($fp);
 
     $target = str_replace(" ", "_", strtolower($wiki["title"]));
 
     if(in_array($target, $blacklist)) return;
     if(count($whitelist) && !in_array($target, $whitelist)) return ;
 
     echo "syncing " . $target . "\n";
 
     $target_filename = $target . ".txt";
     $namespace = str_replace(":", "/", $dokuwiki_namespace);  
 
 
     // make sure directories exist!
     if(!file_exists($dokuwiki_datapath."pages/".$namespace)) {
          mkdir_recursive($dokuwiki_datapath."pages/".$namespace);
     }
     if(!file_exists($dokuwiki_datapath."attic/".$namespace)) {
          mkdir_recursive($dokuwiki_datapath."attic/".$namespace);
     }
     if(!file_exists($dokuwiki_datapath."meta/".$namespace)) {
          mkdir_recursive($dokuwiki_datapath."meta/".$namespace);
     }
 
     if($namespace) $namespace .= "/";
 
     // file was changed in tomboy, so copy old page to attic and overwrite
     if(file_exists($dokuwiki_datapath."pages/".$namespace.$target_filename) && filesize($dokuwiki_datapath."pages/".$namespace.$target_filename) != strlen($wiki["noteContent"])) {
          $target_filename_attic = str_replace(" ", "_", strtolower($wiki["title"]).".".time().".txt.gz");
 
          // compress old data
          $data = implode("", file($dokuwiki_datapath."pages/".$namespace.$target_filename));
          $gzdata = gzencode($data, 9);
          $fp = fopen($dokuwiki_datapath."attic/".$namespace.$target_filename_attic, "w");
          fwrite($fp, $gzdata);
          fclose($fp);          
 
          // write new data      
          if (!($fp = fopen($dokuwiki_datapath."pages/".$namespace . $target_filename, "w"))) {
             die("could not open dokuwiki data directory for writing");
          }
 
          fwrite($fp, $wiki["noteContent"]);
          fclose($fp);
 
          $changelog = time() . "\t127.0.0.1\tE\t".str_replace("/",":",$namespace).$target."\t" . $dokuwiki_username."\n";
 
          // write to global changelog
          if (!($fp = fopen($dokuwiki_datapath."meta/_dokuwiki.changes", "a"))) {
             die("could not open dokuwiki changes file for writing");
          }
          fwrite($fp, $changelog);
          fclose($fp);
 
          // write to page changelog
          if (!($fp = fopen($dokuwiki_datapath."meta/".$namespace.$target.".changes", "a"))) {
             die("could not open page changes file for writing");
          }
          fwrite($fp, $changelog);
          fclose($fp);         
     }
 
     else {
          if (!($fp = fopen($dokuwiki_datapath."pages/".$namespace.$target_filename, "w"))) {
             die("could not open dokuwiki data directory for writing");
          }
 
          fwrite($fp, $wiki["noteContent"]);
          fclose($fp);
     }
}
 
?>

Discussion

Great idea ! Have you an idea how to synchronize list items formats ?

And I note that's necessary that you exclude accentued characters in titles notes (encodage).

Thibaud, 2010/08/22


Thank you for sharing this! It works well.

Notes to other users:

  • Tomboy Notes Location as of 10/30/2011: ~/.local/share/tomboy/ Tomboy Directories
  • Keep / out of the notes titles. Otherwise, it tries to make a new directory when writing to your wiki.

-Sarah Reed, 10/30/2011


Nice work!

Additional note:

  • It does not handle the combination of large and bold

- Rich B. 2014-08-31

tips/synchronise_with_tomboy.txt · Last modified: 2014-08-31 16:28 by 71.191.231.24

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki