Analysis of Pathways to Work performance (July 09 figures)

Update 5/11 - This analysis has been superseded by analysis of the figures released in October, available here

The DWP have released official statistics for Pathways to Work performance, available as a pdf on their website. These differ from monthly league tables by being more accurate and less up-to-date. Here are the headline figures:

Starts (to January 2009)
132,150 starts to Provider Led Pathways to Work of which:

• 42,930 mandatory starts in Phase 1 districts
• 19,960 mandatory starts in Phase 2 districts
• 38,050 voluntary starts in Phase 1 districts
• 31,220 voluntary starts in Phase 2 districts

Job Entries (to October 2008)
11,210 job entries of which:

• 1,670 mandatory job entries in Phase 1 districts
• 180 mandatory job entries in Phase 2 districts
• 6,700 voluntary job entries in Phase 1 districts
• 2,660 voluntary job entries in Phase 2 districts

Performance anxiety

Now, the Conservatives have been touting round a story that only 8% of people starting Pathways to Work have found jobs. Looking at the figures, there's a teeny tiny flaw in this interpretation. Starts go up to January, and job entries only up until October. As we've discussed previously, job entries lag behind starts for the fairly obvious reason that it takes time for someone starting provision to get help and move into a job. In other words, the headline figures are pointless.

Luckily enough, the report doesn't just have the headline figures. It also has the job entries and outcomes, broken down by month and district. This makes it possible, for the first time, to put together a reasonably accurate picture of how each provider is performing across the UK. Or, er, was performing. Eight months ago. It's been a while since October 2008, after all.

As it happens 27th October 2008 was the start date for Employment and Support Allowance. Not only this, but the operation of Pathways to Work has been reviewed and changed in a number of ways in 2009. Taking these together, the entire Pathways to Work regime is so different from the period in this report that drawing conclusions about present performance would be deeply unwise.

A rough comparison of provider performance

With the preceding caveats in mind, it's still possible to get some early comparisons of performance across different areas. To do this, I've used the following assumptions:

• It takes on average two months for someone starting Pathways to Work to get a job. This is an arbitrary figure, and could easily skew the overall results by a few percentage points, but it shouldn't favour one provider above another unless there are huge differences between rates of change in throughflow across different providers
• Job entries are a reliable measure of providing a good service. This is controversial, given that the DWP criteria for measuring and evidencing job entry are notably narrow, and don't leave leeway for e.g. placing someone in a low-hours role then gradually ramping them up as they improve in job readiness, or placing someone in appropriate training
• There are no differences in difficulty of getting people into work between different parts of the UK. This is almost certainly false, but since the contract targets haven't been published, and there isn't a standard model of 'labour market difficulty' yet, there's no way of accounting for this right now

Working on these assumptions, a simple comparison would be to take starts until August 2008 and compare them to job outcomes until October 2008. Unfortunately, Pathways phase 2 areas only started seeing reasonable numbers of mandatory customers from July 2008 onwards, so there are far too few customers to make any judgment whatsoever on performance. Phase 1 mandatory referrals kicked off properly in around March 2008, giving 6 months of starts to work from. Voluntary referrals on both were up and running by early 2008.

Here's the table of results on that basis. The results are grouped into starts, jobs and Job Entry Rate for mandatory referrals, then the same again for voluntary referrals. The gaps are due to phase 2 provisions having too little delivery time on mandatory referrals.

District	Provider	Mandatory starts	M jobs	M %JER	Voluntary starts	V jobs	V %JER
Bedfordshire and Hertfordshire	Shaw Trust				670	130	19
Birmingham and Solihull	WorkDirections	1,450	120	8	1520	310	20
Black Country	Seetec	1,120	60	5	1530	340	22
Cambridgeshire and Suffolk	Reed in Partnership				790	200	25
Central London	WorkDirections	1,200	70	6	950	110	12
Cheshire and Warrington	TNG AND Intraining				650	150	23
City and East London	WorkDirections	760	30	4	920	150	16
Coventry and Warwickshire	Working Links				890	180	20
Devon and Cornwall	A4e	1,460	170	12	3080	900	29
Edinburgh Lothian and Borders	WorkDirections	1,120	130	12	1580	470	30
Forth Valley Fife and Tayside	Triage Central	1,100	130	12	1840	550	30
Gloucester, Wiltshire and Swindon	Working Links				1010	170	17
Greater Manchester East and West	Shaw Trust	2,350	280	12	4170	1,200	29
Hampshire and Isle of Wight	Shaw Trust				760	170	22
Kent	RBLI				690	130	19
Lambeth, Southwark and Wandsworth	WorkDirections	990	70	7	1150	180	16
Leicestershire and Northamptonshire	Working Links				1380	230	17
Lincolnshire and Rutland	TNG	560	60	11	1390	260	19
Norfolk	Shaw Trust	500	50	10	1570	330	21
North and Mid Wales	A4e	670	70	10	1200	350	29
North and North East London	Reed in Partnership				720	120	17
North East Yorkshire and the Humber	Working Links AND Intraining				1130	230	20
Nottinghamshire	WorkDirections	1,280	130	10	1370	420	31
South East Wales	A4e	1,170	110	9	1590	420	26
South London	Reed in Partnership				1070	150	14
Surrey and Sussex	RBLI AND A4e				830	180	22
Thames Valley (Berkshire, Buckinghamshire and Oxfordshire prior to April 2009)	Shaw Trust				680	120	18
The Marches	Remploy				1590	200	13
West London	Reed in Partnership				750	120	16
West of England	Working Links				930	200	22
West Yorkshire	A4e	2,300	200	9	3180	730	23

You can sort the data by clicking on the column headings. I'd strongly suggest you do that as it's difficult to make sense of otherwise. Also note that certain districts operate under Customer Choice and are shared between two providers.

OK, so what do the results tell us? Well, not a huge amount, immediately. We can try rolling up the voluntary performance figures by provider, as follows:

Provider	Voluntary starts	Voluntary jobs	Total %JER
--Total	41580	9400	23
A4e	9050	2400	27
RBLI	690	130	19
RBLI AND A4e	830	180	22
Reed in Partnership	3330	590	18
Remploy	1590	200	13
Seetec	1530	340	22
Shaw Trust	7850	1950	25
TNG	1390	260	19
TNG AND Intraining	650	150	23
Triage Central	1840	550	30
WorkDirections	7490	1640	22
Working Links	4210	780	19
Working Links AND Intraining	1130	230	20

But even that is of limited use. The reason? The figures are for voluntary provision. In voluntary provision, percentage figures don't always work very well, as it's possible for providers to target who they deliver to. Is one provider better than another if it gets a higher percentage of customers into work, but only takes in job ready customers in the first place? It's also worth noting that each month's starts and jobs are rounded off to the nearest multiple of 10, which creates further inaccuracies.

Conclusion

Even with formally verified official performance statistics, it's still too early to identify good or bad provider performance. Both the provision and the benefit system have altered substantially since the period covered by the released data, and external factors such as the local labour market and demography are not allowed for in any way.

There's a formal report on Pathways to Work due later in the year, but in the meantime the main thing to note is that, while there have undoubtedly been huge problems with the provision, the 8% figure being bandied about has no basis in fact. I'll probably put together a report on the work of the Pathways to Work taskforce for next week, which will go into more detail on the exact problems the provision has faced, and what solutions have been implemented.

PS - The figures are available in Excel spreadsheet form below for ease of analysis, provided very promptly by the DWP. Thanks, DWP folks!