DokuWiki

It's better when it's simple

User Tools

Site Tools


plugin:referrers

Referrers Plugin

Compatible with DokuWiki

(current version), Angua

plugin This plugin logs referrers for your site to the page :wiki:referrers, so you can easily see where your traffic comes from

Last updated on
2009-05-26
Provides
Action
Repository
Source

This extension has not been updated in over 2 years. It may no longer be maintained or supported and may have compatibility issues.

Similar to googleanalytics

Tagged with logging, statistics

License

It's not required, but please consider linking to this page from your site if you like this product.

Download

Before downloading or using this product, make sure you understand and accept the terms of the license.

After downloading, make sure to follow the install instructions or upgrading instructions below; trust me, they're worth reading.

  • Older downloads are available on request only.

Note: http://cloud.github.com/downloads/tatewake/dokuwiki-plugin-referrers/referrers-stable.tar.gz always points to the latest stable version!

Installation

Use the plugin manager to install, or…

  1. download the tarball/zip-file
  2. unpack it into <dokuwiki>/lib/plugins/
  3. login as admin and change the plugin in the configuration manager

using git:

% cd <dokuwiki>/lib/plugins/
% git clone git://github.com/tatewake/dokuwiki-plugin-referrers.git referrers

Debian install (or probably any other distribution):

# cd /usr/share/dokuwiki/lib/plugins/
# wget http://cloud.github.com/downloads/tatewake/dokuwiki-plugin-referrers/referrers-stable.tar.gz
# tar -xvzf referrers-stable.tar.gz
# chown -Rv 33 referrers/

Note: http://cloud.github.com/downloads/tatewake/dokuwiki-plugin-referrers/referrers-stable.tar.gz always points to the latest stable version!

About

This plugin logs referrers for your site, so you can see where your traffic comes from.

Set the options for this plugin via the Configuration Settings menu from the DokuWiki admin menu. (It will be near the bottom of the page.)

WarningThe log will be located at :wiki:referrers, so you will need to create that page for this plugin to work.

Install

As a plugin all you need to do is unpack the file into the lib/plugins/ directory (you should end up with a lib/plugins/referrers folder.)

Remember to create a page in the wiki namespace called referrers.

Upgrading

To upgrade, remove the original lib/plugins/referrers folder, and install the new version as instructed above. You may wish to make a note of your settings first though.

What's New

May 26, 2009

  • Plugin is now an action plugin that requires no additional PHP code modification.
  • Referrer spam blocking has been removed in order to simplify the code.

April 29, 2007

  • More referrer spam blocking is in place.
  • Better support for not logging spiders and search engines.
  • Non-standard browser/client IDs will also get logged.

September 3, 2006

  • Initial release

Discussion

  • Start all subtopics using H2. (Heading 2.)

Adding username to referrers output

For anyone finding this plug-in as useful as I am, here's a mod to code.php that will add the user's DokuWiki username as the third column of the wiki:referrers output. Note: After this change, you will have to add a “ | ” between the IPs and ref urls for all the old lines in your wiki:referrers page. Otherwise, old data will NOT display properly. If the old data isn't important, just delete the old lines…much easier. Here's the details:

function re_log_referrers_3($refignore)
{
...
                                #fwrite($fp, "^ hostname ^ ip address ^ referrer ^\n");
                                fwrite($fp, "^ hostname ^ ip address ^ username ^ referrer ^\n");
                        }
                        #write new ref
                        #$rh = getenv("REMOTE_ADDR");
                        $rh = $_SERVER[REMOTE_ADDR];
                        $ru = $_SERVER[REMOTE_USER];
                        if (strcmp($ru, "") == 0) { $ru = "Unknown"; }

                        fwrite($fp, "| %%".gethostbyaddr($rh)."%% | ");
                        #fwrite($fp, $rh);
                        fwrite($fp, $rh . " | " . $ru);

Bobby Metz 07/25/2007

Disabling Null entries

Though rare, some servers don't provide an HTTP_REFERER value and the plug-in will record NULL in the referrer column. While advantageous for access logs, this also allows for the logging of internal POST requests made by your DokuWiki itself. So any internal links that are clicked or users who directly browse to your DokuWiki will be recorded as a NULL. Also, when a remote website redirects to your DokuWiki you will end up with an entry for that server followed by a NULL entry where the doku.php has internally redirected based on the GET attributes. Since these entries do not list websites referencing your page, here's how to disable them:

        # HTTP_REFERRER is blank when user directly requests the page
        # or remote server does not provide it's address
        if (strcmp($refhi, "") > 0)
        if (re_contains($ref,$refignore) == FALSE)
        if (re_contains($ref,' 420source.com') == FALSE)

Bobby Metz 07/25/2007

Referrer script is slow

Moved from monobook for dokuwiki.

Hi I have a quite serious issue with using the template, let me give a few information on my configuration first * Fedora core 5 with Xampp * I switched my website root from /opt/lampp/htdocs to /opt/lampp/htdocs/dokuwiki in httpd.conf The problem is that everything is working just fine with the default template but that the wiki turns very slow when using monobook except if browsing on the server using the localhost address. To be clear, with monobook : * using http://localhost on the server : FAST * using http://192.168.x.y on the server : VERY SLOW (3-4 seconds per page) * using http://192.168.x.y on another machine of the network : VERY SLOW (3-4 seconds per page) Anybody got this problem before ? Thx in advance


Did you for some reason include other.php in main.php? If so, this would be the reason for the slow down. (Its an experimental disabled module for the time being.) If not, the only other thing I can think of is this…

From my experience, slow downs in php are mostly due to file system related activity; I use a fair amount of “file_exists” and “include” calls in the template. So the culprit could be that your PHP is getting choked by the amount of file calls. (Possibly your file system is a slow one?)

Beyond this, in my opinion, there are probably a few things in the apache config that might improve your performance. (Though I don't know what it would be.) I'd suggest investingating PHP speed problems with Apache and your machine config; it'll probably boil down to that. — Terence J. Grant 05/27/2006 21:17


Hi, and thank you for your answer. This morning, I debugged main.php and I found out that it was this part of the code that was slowing down the page generation :

 <?php @include(dirname(__FILE__).'/referrers.php'); ?> 

I just removed it this way and it's working fine now

 <?php /*@include(dirname(__FILE__).'/referrers.php');*/ ?> 

Is it safe to just comment it ? I'm not interested in the feature anyway. Thanks again


I had originally thought that might be the case as well, but for whatever reason I forgot to mention that possibility. If anything, I would imagine the reverse lookups in referrers.php would be the slowdown, but again it could be due to the file system calls. You can comment that out and everything will work just fine. — Terence J. Grant 05/29/2006 02:02


The root cause of this problem is that machine with local, fake NAT IP addresses (192.168.X.Y) don't have reverse IP maps. That means that anytime the server tries to find out the name of a system on that subnet to generate the referrer info, it's going to try to find it via DNS, which will take a few seconds to time-out before returning nothing useful. When hitting the server from the 127.0.0.1 address, DNS knows that's always the localhost so the response is instant. While the “right” approach to this problem is to screen out the NAT subnets at the DNS level, the reality of this situation is that you're always going to run into situations where Internet hosts don't have valid reverse IP addresses periodically; also, many people don't have access to the broken DNS server or the knowledge to fix it.

This sort of problem is always coming up with using gethostbyaddr in any programming language. At a minimum, consider screening out and ignoring the RFC 1918 private IP addresses 127.X.X.X, 10.X.X.X, 172.16-31.X.X, and 192.168.X.X, in your code, which will get rid of this issue altogether for most users. Alternately or additionally, you may want to consider using a more robust implementation with a quicker timeout than the default, because this can be the root of a denial of service attack of sorts. There is a discussion of this issue with some workarounds at http://us3.php.net/gethostbyaddr you may want to look into; just calling the function naked the way you do in referrers.php is asking for problems like this to appear at random times for users. — Greg Smith 06/21/2006 08:56


Thanks for the information and tips, I'll look at making this better for the next release. — Terence J. Grant 06/21/2006 11:36

Modifying for https sites

I needed to block refs for my site which does https only. This takes a simple change to code.php. Following the $a & $b used by the author, you can do this:

$a = 'http://'.$refignore;
$b = 'http://www.'.$refignore;
$c = 'https://'.$refignore;
...
        if (re_beginsWith($ref,$a) == FALSE)
        if (re_beginsWith($ref,$b) == FALSE)
        if (re_beginsWith($ref,$c) == FALSE)

However, it seemed that the author added this as an afterthought and since most of the traffic will be interanl linking, I thought a couple of slight modifications to the original code was better, i.e. no more $a, $b, etc.

#$a = 'http://'.$refignore;
#$b = 'http://www.'.$refignore;
        if (re_contains($ref,$refignore) == FALSE)
        if (re_contains($ref,'420source.com') == FALSE)

The first line will match any ref url including your page. Using the re_beginsWith just makes extra comparisons, depending on how you specify your page url in the local_pref.php file.

Bobby Metz 07/25/2007

Well, I'm going to go for $a, $b, $c rather than just do a contains. If someone does a “google translate” on your site, pops “yoursite.xxx” into a search engine, or has something like a blog entry with your site name in the url, you won't get it logged in the referrer logs if you use the “contains” function. So although $a, $b, $c, etc isn't horribly elegant, I think I'd rather keep it this way. (So I'll add the https ones for the next version.) — Terence J. Grant 08/02/2007 07:43

Manually Create Preferences File

After installing this plug-in I was surprised to find that the local preferences for it are not loaded from the main DokuWiki local.php file. There is a nice configuration page, but for high security sites which don't allow web-based admin changes this is a problem. There is no documentation outlining where and how the preferences file should be created. So, if anyone out there needs it, here's the skinny:

File name: lib/tpl/referrers/local_pref.php Default settings to add:

<?php
$re_settings['referrer-ignore'] = '';
$re_settings['dontcountadmin'] = '';
$re_settings['dontcountusers'] = '';
?>

Bobby Metz 07/25/2007

Whoops, ok this'll be added to the next release, thanks. — Terence J. Grant 08/02/2007 07:53

Speedier Exec - Getting rid of getenv lag

With dicussions of file system slowness above, I too began digging for ways to use this very useful plug-in while minimize user impact. Here's another means to increase responsiveness of the referrers plug-in…remove all the getenv calls. While this won't produce monumental speed increases, there is no point in making the system work harder when all the variables needed by this module are already loaded by PHP in the $_SERVER array. On my system I can save about 4ms per getenv call I remove. There are a total of nine such calls in the code.php file, so 36ms faster give or take on an average day. Here's what to change in code.php:

function re_log_referrers_3($refignore)
{
...
        #$refhi = getenv("HTTP_REFERER");
        $refhi = $_SERVER[HTTP_REFERER];
...
                        #$rh = getenv("REMOTE_ADDR");
                        $rh = $_SERVER[REMOTE_ADDR];
...
                        #if (re_beginsWith(getenv(HTTP_USER_AGENT), "Mozilla") == FALSE)
                        #if (re_beginsWith(getenv(HTTP_USER_AGENT), "Opera") == FALSE)
                        if (re_beginsWith($_SERVER[HTTP_USER_AGENT], "Mozilla") == FALSE)
                        if (re_beginsWith($_SERVER[HTTP_USER_AGENT], "Opera") == FALSE)
...
function re_log_referrers_2($refignore)
{
        global $_SERVER;
        #$x = gethostbyaddr(getenv("REMOTE_ADDR"));
        $x = gethostbyaddr($_SERVER[REMOTE_ADDR]);
...
        #if (!re_contains(strtolower(getenv(HTTP_USER_AGENT)), "bot"))
        #if (!re_contains(strtolower(getenv(HTTP_USER_AGENT)), "crawl"))
        #if (!re_contains(strtolower(getenv(HTTP_USER_AGENT)), "spider"))
        #if (!re_contains(strtolower(getenv(HTTP_USER_AGENT)), "larbin"))
        if (!re_contains(strtolower($_SERVER[HTTP_USER_AGENT]), "bot"))
        if (!re_contains(strtolower($_SERVER[HTTP_USER_AGENT]), "crawl"))
        if (!re_contains(strtolower($_SERVER[HTTP_USER_AGENT]), "spider"))
        if (!re_contains(strtolower($_SERVER[HTTP_USER_AGENT]), "larbin"))

Bobby Metz 07/25/2007

Ok, this will be in for the next version, thanks. — Terence J. Grant 08/02/2007 07:42

Question

The instructions say to insert a couple of php code lines in “your template… between <head> and </head>” without specifying

a file, or even a file type. A typical template has like two dozen files in there. Are we talking about some css file?

Typically you'll want to do this for any PHP (or HTML) files containing the string “<head>”; at a minimum you can do just main.php.

If you only do main.php though you'll miss some hits to certain types of pages, so I suggest searching for head tags through all the php files in a template to make sure you get them all. — Terence J. Grant 08/11/2007 13:38

All referrers are NULL?

Hi -
I had been running a fairly old version of Dokuwiki/Monobook for a long time. After I upgraded to the latest versions of Dokuwiki, Monobook software and added the referrers plugin, I noticed that all logged visits have a referrer value of NULL. I do not understand why, because my Apache “combined” log shows non-null and correct referrer fields.
I would have assumed that the HTTP_REFERER environment as well as $_SERVER parameters (I have tried both in the code.php script file) would carry the referrer information. Any ideas why I am getting empty values? — Eric Hameleers 18/nov/2008 13:45

Edit: It kept bugging me so I looked at the code.php again and saw a problem when the $re_settings['referrer-ignore'] would be set to the default empty value. All non-null referrers will be filtered out of the resulting referrers.txt file because of the lines:

$a = 'http://'.$refignore;
$b = 'http://www.'.$refignore;

and the checks on

        if (re_beginsWith($ref,$a) == FALSE)
        if (re_beginsWith($ref,$b) == FALSE)

So I manually created ./lib/plugins/referrers/local_pref.php as documented in one of the earlier sections of this page, and as a side effect, changing the defaults for the plugin in the admin section of the wiki works now (it used to spit errors at me). Now, with the value of $re_settings['referrer-ignore'] set to my Wiki URL, real referrer values are pouring in :-)
Eric Hameleers 18/nov/2008 14:54

Automatic commit of referrers page versions

Just a tip…
I thought it was a shame to see the referral information disappear at midnight, when the referrers.txt is reset. The DokuWiki has a few php scripts useful for automation, ./bin/dwpage.php being one of those. So, I added the following to my crontab which causes the actual content of referrers.txt page to be committed to the attic. Now, I can recall old revisions of the wiki:referrers page.

# Archival of the wiki referrers page (otherwise all data is lost at day change)
0 0 * * *       /usr/bin/php /xxxxxxx/htdocs/dokuwiki/bin/dwpage.php -m "Automatic archival" commit /xxx/htdocs/refempty.txt wiki:referrers 2>/dev/null

Where the external file refempty.txt only contains some placeholder text (it will be overwritten in a matter of minutes anyway).
Eric Hameleers 19/nov/2008 01:28

FIXME Peut on avoir plus d'explications pour archiver la page refferrers car je ne comprend pas ou on doit insérer cette ligne ?
Can we have more explanation to archive page refferrers because I do not understand or you must insert this line?

plugin/referrers.txt · Last modified: 2015-09-13 19:36 by Aleksandr