I wrote a bit for the Furasta.Org overview page yesterday. I decided to share what I wrote (re-written independent from Furasta.Org) because when writing it I couldn’t find anything online documenting it. I know there are things like this – WordPress’s Dashboard is one example – but it didn’t seem to be documented anywhere.

Demo

Download Example

I’m not going to go into the details of how it’s done. If you download the well documented example, you should be able to figure out how it works. Any suggestions for improvement are welcome.

This example is somewhat different from how it is actually done in Furasta.Org, where I use the caching system to store files based on the user’s id insuring that each user can have their own separate configuration. The key difference, noted in the documentation, is that in Furasta.Org I don’t know how many divs need sorting because the plugin architecture has access to what I have called ‘Overview Items’. So to deal with that issue I created a class to deal with the items, which is visible here.

Packt launches fifth annual Open Source Awards

The 2010 Open Source Awards was launched last month by Packt, inviting people to visit www.PacktPub.com and submit nominations for their favorite Open Source project. Now in its fifth year, the Award has been adapted from the established Open Source CMS Award with the wider aim of encouraging, supporting, recognizing and rewarding all Open Source projects.
WordPress won the 2009 Open Source Content Management System (CMS) Award in what was a very close contest with MODx and SilverStripe. While MODx was the first runner up, SilverStripe, a Most Promising CMS Award winner in 2008, made its way to the second runner up position in its first year in the Open Source CMS Award final.
The 2010 Award will feature a prize fund of $24,000 with several new categories introduced. While the Open Source CMS Award category will continue to recognize the best content management system, Packt is introducing categories for the Most Promising Open Source Project, Open Source E-Commerce Applications, Open Source JavaScript Libraries and Open Source Graphics Software. CMSes that won the Overall CMS Award in previous years will continue to compete against one another in the Hall of Fame CMS category.
These new categories will ensure that the Open Source Awards is the ultimate platform to recognise excellence within the community while supporting projects both new and old. “We believe that the adaption of the Award and the new categories will provide a new level of accessibility, with the Award recognizing a wider range of Open Source projects; both previous winners while at the same time, encouraging new projects” said Julian Copes, organizer of this year’s Awards.
Packt has opened up nominations for people to submit their favorite Open Source projects for each category at http://www.PacktPub.com/open-source-awards-home . The top five in each category will go through to the final, which begins in the last week of September. For more information on the categories, please visit Packt’s website http://www.PacktPub.com/blog/packt’s-2010-open-source-awards-announcement

Recently I was adapting a newsletter plugin for Wordpress and needed the PHP mail() function for testing. However an Ubuntu desktop install is missing Sendmail – the MTA that PHP expects to find on a Linux PC. I use a local Apache/MySQL server on a laptop to do a lot of my development – I don’t [...]

Despite all the implications of using LIKE in MySQL, sometimes it’s quite useful for a proof-of-concept to be able to use it.

Even though this is a rather trivial example, I hope it will be of help to whoever is wondering how to use a LIKE in a where using a Zend_Db_Select::where() or Zend_Db_Select::orWhere() and did not manage to find decent documentation about it.

<script src="http://gist.github.com/534198.js">

As you can see, we bind a parameter to the where() method using the ? symbol, then when assigning the value of this parameter — second parameter — we append the wildcard character %. The new binded parameter value is then string% and which gets quoted and executed giving you the expected output.

HTML Sanitisation (defined below) has been with us for a long time, ever since the first genius who came up with the idea of allowing potentially untrustworthy third party HTML to be dynamically patched into their own markup. The years have not taken this kindly, and third-party HTML inclusion has remained one of the most complex and underappreciated vectors for security vulnerabilities.

In this article, I take a look at some of the solutions PHP developers rely upon to perform HTML Sanitisation. Mostly because few others have done it or written about such solutions in any great detail (at least publicly). HTML Sanitisation has a very low profile in PHP. It's rarely mentioned, usually not understood all that well, and examining some of the solutions in this area with more deliberate attention is worth doing. Also, it's valuable research since I am writing my own HTML Sanitisation library (bias alert!) for a future Zend Framework 2.0 proposal. Knowing what the competition is up to does no harm! Finally, I was simply curious. Nobody seems too pushed to look closely at all these HTML Sanitisation solutions despite the fact that there are other developers (I think) who wouldn't touch most of them with a barge pole.

One somewhat remarkable example, just to illustrate why I figured this article was worth the time, is HTMLPurifier's Comparison analysis where HTMLPurifier is compared against a number of other HTML Sanitisers. The comparison is remarkable because it seems inclined to err on the side of giving HTMLPurifier's competitors the benefit of the doubt. Unfortunately, this means the analysis is often flawed and its conclusions suspect. Also, it assists in legitimising other solutions in the minds of readers by making assumptions of safety. Not that this reflects on HTMLPurifier's functionality, incidentally, which I have always maintained is the only HTML Sanitiser worth looking at.

Back on track…

What is HTML Sanitisation?

HTML is an amazingly dangerous thing. It can contain Javascript, CSS, or malformed markup, or even gigantic images that laugh at your dual 32" monitor setup. Each of these, in their own way, can damage the experience of an end user, whether it be by Cross-Site Scripting (XSS), Phishing or simply mangling the page until it's unusable and/or defaced with scriptkiddie jibes.

There are two ways of dealing with these threats to the HTML output of an application: escape output so that the only HTML rendered by the browser is the application's (anything else being neutered by HTML entities), or by sanitising output so that any additional HTML it contains, that is renderable by a browser, is stripped of any potentially damaging markup. This article concerns the second option.

HTML Sanitisation may therefore be defined as any means of filtering HTML to ensure that a) Cross-Site Scripting (XSS) vulnerabilities are removed, b) Phishing vulnerabilities are removed, c) the HTML is well formed and adheres to an acceptable HTML standard, and d) the HTML contains no obvious means of breaking expected web page rendering.

I won't claim this is a perfect definition but it covers most of the salient points you'll likely encounter.

So there are, broadly speaking, four primary objectives of HTML Sanitisation, any one of which is capable of preventing damage to end users or web application functionality (including javascript powered client side functionality). Each is, in its own way, quite a difficult proposition requiring suitable tools and specialised knowledge. However, with some objectives we can measure our success somewhat reliably. The question of this article being: how well do HTML Sanitisers in PHP measure up to these objectives?

The Candidates

Since this is intended as a brief examination (just a few million words long!), I decided to select four candidate HTML Sanitisers meeting certain conditions. These conditions included:

1. Having a release at some point in the past two years;
2. Describing itself as a HTML sanitiser/filter to prevent Bad Things;
3. Having a design clearly in line with an intent to filter XSS/Phishing; and
4. Having no publicly acknowledged long standing security vulnerabilities.

The great part about applying these conditions is that I pretty much eliminated stacks of HTML Sanitisers (as some might claim them as being). Outside of those, it also eliminates anything users might misconstrue as a HTML Sanitiser (for example PHP's strip_tags() function or Zend Framework's Zend_Filter_StripTags class). What we are left with is pretty thin on the ground, but fits what I'd expect a reasonably educated PHP developer to swing with. From what remained, I selected four candidates (or maybe these were the only four left

Truncated by Planet PHP.ie, read more at the original (another 23169 bytes)

Ever considered developing a RESTful API? Ever wondered what is FRAPI and how it works? Well apart from reading the frameworks’s website, there was no real way to assess FRAPI as a RESTful API Framework — Not until recently.

In order to ease adoption and make it more accessible for people to evaluate FRAPI, we’ve put an Amazon AMI together. This AMI comes pre-installed with Linux Lucid Lynx (Ubuntu 10.04 LTS), NGINX as the webserver, Memcached — (And no, port 11211 isn’t opened to the public), PHP5-FPM, APC and obviously FRAPI.

If you’d like to give this public AMI a spin, just go to Amazon’s instance management section, click on “Launch Instance, go to community AMIs, and search for : “ami-8c28c3e5” . Once you found it, click on “Select” (And make sure to select Port HTTP (80) when asked about which ports to open).

It’s running, now what?

You can go directly to the AMI’s host name in a browser and you will get FRAPI’s default invalid action error, or you can follow these steps to customise it a bit more.

Now that you see your new shiny FRAPI-AMI instance running in your management console, you have a few options:

First option is: Find the “Public DNS” and the “Public IP Address” of the instance you just launched and point your domain-name’s DNS to the IP you found — Your domains could be: api.mydomain.com and admin.api.mydomain.com

Second Option is: If you do not currently hold a domain name for your API, edit your /etc/hosts file (Or the equivalent in your OS) to have an entry that points to the IP of your instance

10.0.0.1 api.mydomain.com admin.api.mydomain.com.

Once you are done with either Option 1 or Option 2, you’ll need to connect to your instance over SSH (ssh ubuntu@hostname-of-the-instance -i your-amazon-cert.pem). Upon connection, you’ll be greeted with a welcome screen giving you some information about FRAPI and the server’s software : <script src="http://gist.github.com/515503.js"/>

Once you are connected, make sure to “sudo su” and then edit the files that are in /home/ubuntu/sites/. The only thing you have to modify in those files are the “server_name” details. Put either your domain name or your made up domain name that is setup in your /etc/hosts file. The end result should be an admin file with something that contains something along the lines of <script src="http://gist.github.com/515519.js"/>

and an api configuration file that contains

<script src="http://gist.github.com/515520.js"/>

Don’t give up, you are nearly there, now you have to run : “/etc/init.d/nginx restart && /etc/init.d/php5-fpm restart” and you are done.

If you go to the api.mydomain.com in your browser you should now see FRAPI’s default invalid action XML error. If you go to admin.api.mydomain.com you should be able to login with the username “admin” and password “password” — Hint: You should go to the “configurations -> users” section and modify that password.

I want to code something!

SSH into your instance, and go to /home/ubuntu/development/custom/Actions, you will see all your API Controllers there and you can follow the steps here

Truncated by Planet PHP.ie, read more at the original (another 814 bytes)

A

In January of this year, I had the idea of writing a HTML Sanitiser for PHP. Why not? All PHP has is HTMLPurifier and a bunch of random solutions that are about as secure as the average wooden gate. If you think that's harsh, wait for my next blog post . HTMLPurifier is the only secure by default HTML Sanitiser in PHP. Fact. But the darn thing is gigantic and slow. That has never stopped me using it (for years), even if I had to do a little funky engineering so I could minimise the performance hit. Other developers, however, have often abandoned HTMLPurifier, falling into the trap of believing that alternative solutions will serve them just as well.

That's the state of HTML Sanitisation in PHP - pick a big slow library that crushes Cross-Site Scripting and Phishing attacks, or use yet another regular expression based sanitiser that a) barely manages a fraction of HTMLPurifier's features and b) can probably be exploited by any scriptkiddie working with a stack of data cards. It says an awful lot about security standards among PHP developers that such delusions are uncomprehendingly rampant.

In case you haven't noticed, I'm biased. Sue me.

I have opined since forever that regular expression sanitisers are nothing short of insane. Since the problem with HTMLPurifier is speed and size, I started thinking about ways to build something like HTMLPurifier that was fast, small and almost as feature packed as HTMLPurifier. At first, this sounds like an impossible task. The typical suggestion is to use regular expressions, but I'm not completely insane...yet. Instead I borrowed a concept called a DOM Filter and chucked in a helpful dose of HTML Tidy. The result was Wibble.

Wibble is basically a DOM Filter. It loads up HTML into PHP DOM, applies a set of filters against all nodes in the DOM, passes the output through HTML Tidy, and then hands it back to the user - sanitised and well-formed. It's almost stupid in its obviousness. Better, this allows Wibble to skip regular expression dependence. It operates far more like HTMLPurifier by relying on a DOM representation (no string parsing to funk around with) partnered with Tidy for cleanup.

Of course, there have to be regular expressions somewhere. And whitelists. And other stuff. Wibble is really an amalgamation of borrowed concepts. It's hard to be too original in HTML Sanitisation because originality is a good way to shoot yourself in the foot (hence regex is EVIL!), so I wasn't going to spend too long digging my own grave when there is a wealth of sanitisation resources in the programming world. Wibble's approach borrows elements from Ruby's loofah, Python's HTML5Lib, and Java's AntiSamy. Wibble mixes and matches from the useful design elements each of these offers, serving them up on top of PHP's DOM and Tidy extensions with its own distinctive twists.

I completed the first Wibble prototype recently, so I figured that with something that was at that 90% point where the remaining 10% would be in-depth sanity testing, cleanup and documentation, it was time to see how it compared to some other PHP solutions (HTMLPurifier and HtmLawed). I had some fairly conservative performance objectives so the results came as a pleasant surprise.

If you are a benchmark fiend, you can download and independently fiddle with my benchmark process from http://github.com/padraic/wibble-benchmarks. Note that the current benchmark uses a Wibble prototype - there are additional elements that need to be added over time. The benchmark currently uses three sample snippets of HTML: Small (blog comment size), Medium (markup heavy with limited textual content), and Big (markup light with lots of textual content). It operates by filtering each HTML sample 200 times with each benchmarked HTML sanitisation solution. Each iteration includes the instantiation and setup phases of each solution (where relevant) to reflect the most likely real world experience of using sanitisation as a once off (non-repeating in same request) process. I use PEAR's Benchmark package to record the aggregate run time per loop of sanitisation tasks. All operations occur within one single PHP process with HTMLPurifier caching enabled (Wibble and HtmLawed do not use caching). Each solution is configured as close as possible to target total stripping of all HTML from the content.

You can view a sample result at http://gist.github.com/468426.

The results show that both Wibble and Htm

Truncated by Planet PHP.ie, read more at the original (another 2564 bytes)

Kae posted today about tackling this problem under Linux, but I’ve found that it’s quite a different task under OS X. Anyway here is how it’s done:

First you need to download the swatch package from here.

If you haven’t done it before you will have to set up cpan and it’s a lengthy process. Run cpan in the terminal. It’s safe to say yes to all the options. Once it’s installed you need to add some perl modules. In the cpan prompt add the following modules:

install Date::Calc
install Date::Format
install Date::Manip
install File::Tail

That’s the perl part finished anyway. Now change to the swatch directory and compile:

perl Makefile.pl
make
make install

Ok swatch should be installed now. So set up the configuration file: vim ~/.swatchrc Add the following lines to the file:

watchfor /PHP Parse error|PHP Fatal error/
bell 3

Save and return to the command line. Now to add swatch (and also optionally apache and mysql) as a startup item do as follows. Create a new directory named StartUpItems, then create the following files so your setup should look like this:

StartUpItems
StartUpItems/StartUpItems (a file with no extention)
StartUpItems/StartupParameters.plist

Now add the following contents to StartupParameters.plist:

< ?xml version="1.0" encoding="UTF-8"?>
< !DOCTYPE plist SYSTEM "file://localhost/System/Library/DTDs/PropertyList.dtd">

    
        Description
        Various Startup commands including Swatch, Apache and MySQL.

        OrderPreference
        Late

        Provides
        
                Starts up Swatch, Apache and MySQL.
        
    

That’s just a file required by OS X to recognise the startup item, don’t worry to much about it. Now to the StartUpCommands file which is a bash script:

#!/bin/bash
. /etc/rc.common

StartService(){
        ConsoleMessage "Starting Apache, MySQL and Swatch"
        apachectl start
        /usr/local/mysql/support-files/mysql.server start
        /opt/local/bin/swatch --daemon --config-file=/Users/conormacaoidh/.swatchrc --tail-file=/var/log/php/php-error_log --pid-file=/var/run/swatch-httpd-errors.pid
}

StopService(){
        ConsoleMessage "Stopping Apache, MySQL and Swatch"
        PID=`cat /var/run/swatch-httpd-errors.pid`
        kill -9 $PID
        apachectl stop
        /usr/local/mysql/support-files/mysql.server stop
}

RestartService(){
        RunService stop
        RunService start
}

RunService "$1"

The file

/etc/rc.common

is provided by apple and makes the process a lot easier, though it is not needed. The lines in StartService and StopService do the opposite of each other but are very similar. You may need to change the apache and mysql start/stop commands according to your own setup. If you are using MAMP this file will start apache and mysql:

/Applications/MAMP/bin/start.sh

As for the swatch line you will obviously have to change the config file location and the php error log location to what’s appropriate on your machine.

And that’s it really. Then copy the items to the correct directory for OS X to recognise them:

sudo cp -R StartUpCommands /Library/StartUpItems

It is vital that you change the permissions or else the startup item simply won’t be executed:

sudo chown -R root /Library/StartUpItems/StartUpCommands
sudo chgrp -R wheel /Library/StartUpItems/StartUpCommands
sudo chmod -R 755 /Library/StartUpItems/StartUpCommands

That’s it. You can test the configuration by running SystemStarter -n -D which will emulate what happens when the computer starts, it is useful for de-bugging.

So restart you computer, load a php file with an error in it and your machine will beep thrice!

Updated with more tests on 2010-05-16. Click here to jump to the 2010-05-16 update… The conventional wisdom always said that PHP’s include()/require() was quicker than include_once()/require_once(), but recently I came across an interesting post by Arin Sarkissian which suggests otherwise. Also I found more commentary on the performance benefit of using relative versus absolute paths in [...]