Can't see the wood for the trees? You need our free industry newswire
Is Ofsted working?
It's been a few years since I helped on an inspection visit from the then-ALI. Since then, performance statistics have become far more important, with class and one-to-one observations taking more of a back seat. Given the forthcoming introduction of star ratings and recent criticisms of Ofsted's school inspections as pointless, what's the purpose of Ofsted inspections nowadays? Does anyone find them useful?
Also: If you're looking for provider ratings on the Employment Zones, you could do worse than looking at their star ratings (pdf). The big story is that Reed in Partnership's results are absolutely atrocious and Working Links' are surprisingly good. I may publish some brief analysis of the results to see what I can make of them.
Also also: Ofsted staff, I know you subscribe to the newswire! Now's your chance to tell people what you really think.
Star ratings are only for DWP programmes, so Ofsted is still the arbiter of quality in LSC-funded provisions. Given FAM covering the administrative / paperwork side of things, and star ratings covering most of the other aspects of provision quality, it's difficult to see that Ofsted will still be coming into welfare-to-work provisions in a few years. We're still in a transition now though, and as you point out they've only done star ratings for a small subset of provisions. When are you updating the contracts database?
Hi Holmwood,
Good points. I'm still working on cleaning up the DWP contracts database - it's in terrible shape. I should be there shortly though, maybe even by the end of this week (As I've been saying for the past few weeks).
I am afraid the star ratings do not offer much insight into actual underlying performance.
The organisations are scored on performance against their contractual targets. These will have been derived from their performance offer in their original tenders - possibly renegotiated in the meantime. But those targets may vary wildly between areas and not necessarily in any sort of relation to local labour market conditions.
Unless DWP provide the targets alongside the table, it is impossible to use the star ratings for any sort of meaningful comparison. You cannot look at this table and conclude in any reliable way that one organisation is better or worse than another at helping long-term unemployed people find and keep work.
It is, of course, imperative that we move towards more open and transparent contracting. Performance on programmes must be out in the public domain and future contracting decisions must take account of an organisation's track record. Unfortunately, these star ratings are not, as they stand, a step in that direction.
Thanks for the response w2w. I agree that contracts and performance should be as transparent as possible. I suspect it's a far more important step than Flexible New Deal or any other whizzy model. How can public services be in any way accountable when their performance, payments, and contractual conditions are kept secret because of 'commercial confidentiality'? There isn't even a reliable list of who holds which contracts, for goodness' sake. Hopefully what I'm doing here will increase the transparency of welfare-to-work provision.
With regard to the usefulness of star ratings, I don't have personal experience of EZ target setting. I wasn't aware of the individual negotiations, but that does indeed mess with the validity of the results. Unfortunately, it also messes with the validity of Ofsted reports just as much, since they're based on performance against contractual targets as well.
I wonder if 'local labour market conditions' aren't something of an excuse for a lot of providers. Some of the highest performers have been in the most deprived areas of the country. The housing and childcare traps in London and the supply-side issues in rural ex-industrial areas (i.e. old mining towns) are the most obvious issues. However, if you have a proper competition for contracts based on prices and performance, then in theory at least the contracts will reflect the actual difficulty of local delivery. Do EZs in the same area ever have different targets? That would be wacky.
Hi
I would like to add that i work for a local authority and manage the DWP Workstep Programme, which is inspected by Ofsted. I don't wish to bore you about how inspections were managed in the early days of ALI with Workstep providers and how limitied the knowledge and experience of all ALI inspectors were, and even many years on, there is still a reluctance to understand the programme and the complexities of the client group we are supporting back into employment. However, during a recent inspection the Ofsted inspectors were clearly focussed on one area - progression. This has historically been a difficult area for all Workstep providers nationally. Yes, it is fair to state that some providers achieve higher progression targets than others, but generally the nationally achieved targets have been the same for many years on Workstep. My point really is that Ofsted seem to have their own agenda and considering the whole process is about transparency and accountability, this was not apparent in the inspection when inspectors were questioned on how they were measuring targets, how they were using evidence to make judgements etc. It was clear that Ofsted had been given a 'steer' from Government on what the Government expected in regard to outcomes from Workstep providers and this was eveident within the inspection process. When i questioned the inspectors on this issue, of course it was denied that targets had been set 'of the record' for Workstep Providers within an Ofsted inspection. Yet, the whole inspection process was geared around outcomes, outcomes and more outcomes. Whilst i appreciate that targets are important and funding should be 'outcome' based (proportionately), the Workstep programme is very unique to DWP and unless those that are managing, inspecting or have a financial input understand what the programme is really about - 'placing the most vulnerable people into employment,' then i feel that more and more providers will continue to be inspected by inspectors that do not understand 'employment,' but do understand 'education' and results will be outcome focussed rather than quality focussed. I have never known a Government that has been so focussed on outcomes and yet forget accountability apart form when it suits them.
There are two responses to this. First, I believe that Ofsted has been outcomes-focused for a couple of years or so now, with the consequence that everyone running under-target on job outcomes (i.e. most New Deal deliverers) gets a bad grade. This steer wasn't specific to any type of provision.
Second, rumour has it that the FAM teams are shutting down in July. No word on what's going to replace them, but the new PRaP IT system won't leave a paper trail to be inspected, and new contracts don't pay on attendance in any case, so it makes sense. That leaves Ofsted, Star Ratings, and the DWP contract management process as the primary monitoring and management streams, with external quality and financial accreditations (Matrix, the provider financial assurance check etc.) as secondary ones.
I think it has been generally accepted in the Self-employment sector that compliance (FAM) measurements often conflict with quality (ALI/Ofsted)delivery because the prescribed procedures are often inappropriate to how Client progress should be managed. This is not helped by Ofsted seeing its role as simply "Inspection" rather than Quality Assurance, seeing Specification issues as beyond their remit. We asked them!
Also, by their own admission, there are clearly problems with the Equality and Diversity definitions. Helping someone start a niche business is obviously exclusive -and ensuring someone else does not exclude possible (and wanted) customers is about being inclusive. Unfortunatly, simply helping Clients in these ways does not appear to be recognised as ticking the right boxes.
It can be argued that their reliance on Ofsted amounts to an abdication of quality responsibility by JCP - effectively replaced by the outcome-counting. And that raises interesting questions about current contracting policies.