Stepping into the light - DWP contracts set to publish open performance information

Ever since Indus Delta started up, a major theme and recurring bugbear of the site has been the publication and analysis of welfare to work performance figures. After two years and a constant stream of Freedom of Information requests and leaked data, it appears the quest may be nearing its end.

Present Imperfect

Following the recent publication of New Deal and Programme Centre figures on the site, followed by New Deal Self Employment results and the latest Star Ratings and traffic light reports, a picture of delivery is beginning to emerge.

Unfortunately, it's still a fuzzy picture. Mostly, this is because the transition from locally measured and managed Jobcentre Plus-run contracts to a national DWP-run contracts framework happened after the current contracts were set up. Different parts of the country have been using different definitions of a job entry. The data itself is unreliable at the level of individual contracts, and is derived from self-reported figures (i.e. what the providers say they've got). Comparison of individual providers is a fruitless task. Given that many contracts don't have a single provider performing anywhere near the supposed targets, it's fair to assume that the contract targets aren't a reasonable measure of what's achievable in any case.

Just as importantly, the picture is still incomplete. Contract details on NDDP, Pathways to Work and other contracts are being withheld in the name of commercial confidentiality. Individual performance on Pathways to Work remains a mystery, despite performance league tables going out to the providers themselves on a monthly basis.

Open and shut

So why is everything looking so sunny, in spite of the remaining difficulties? The answer lies in a fundamental shift in the contracting approach, instituted by the DWP themselves in their Commissioning Strategy back in 2008. The current contracts were procured under the old regime, and are still being run with the old regime's secretive approach. This is perhaps unsurprising given just how ramshackle their performance data actually is.

Flexible New Deal, and all future contracts, are being procured under a much more open system. The Commissioning Strategy committed the DWP to openly publishing contract performance information, and all the signs are that they're following through on this promise:

• The FND specification and ITT instructions stated that contract targets and performance may be published openly
• With the introduction of PRaP, the management information should become far more reliable, and less open to differing local interpretations
• According to the Employment Minister, the DWP is planning to publish monthly performance tables on its website later in 2009

Just to give an added push, Serco have already published their FND subcontractor delivery targets, will publish their subcontractor and overall performance data on their website on a weekly basis from go-live, and have made clear that they would like to publish their contract performance targets. This last one is subject to DWP approval, although it's hard to see how commercial confidentiality could be used to enforce secrecy on this point once the the contracts have actually been signed. Other providers have also indicated a willingness to open up about their contract targets, in a rather more circumspect fashion.

What does it all mean?

Given the controversy over star ratings in hospitals, and league tables for schools, is open performance data really the best thing since sliced bread? I would argue yes:

• We'll finally know who the bad and good providers are. This will make it possible to reward good performance and force poor performers to improve
• Star ratings won't be undermined as easily by providers claiming that quirks in the system are misrepresenting their true performance
• As the MP expenses scandal illustrates, self-regulating systems or systems where the regulators have a cosy, behind-closed-doors relationship with the regulated are far more open to abuse
• I can spend my time doing more productive things than battling the civil service's amazing ability to obfuscate information it's not happy about.

Funnily enough, the schools league table provide an excellent example of what can go wrong when rolled-up headline figures are used instead of raw data. A key measure of performance in secondary schools is the percentage of students getting 5 grades A-C. In consequence, many teachers find themselves concentrating on pushing marginal D grade students up a grade, and paying less attention to the needs of other students. With the raw data, it will be much easier to identify and shame providers who respond to such perverse incentives

What could still go wrong?

One big thing that could throw a spanner in the works of opening up about provider performance is if the contract targets are kept secret. It's not at all clear at this stage that the DWP are intending to allow their publication. However, it's also difficult to see how publishing actual performance but hiding the targets would even be possible - some basic maths on the star ratings and the actual monthly performance would quickly reveal the contract targets.

A wider issue is that many of the recession-response and locally managed contracts (Future Jobs Fund, WNF etc.) do not appear to be following the Commissioning Strategy. Rumours abound that the £1bn Future Jobs Fund is likely to amount to a government grants scheme, without payment on outcomes and with limited measurement of impact. Given that, is it likely that they'll publish open performance information that holds delivery to account?

Putting these together with the still-to-be-replaced New Deal contracts, and the still-secretive Pathways to Work and other disability support contracts, the journey is far from over. But with FND at least, the promised land is in sight. Hallelujah.