2020-2021 PSBAR Monitoring Review
Introduction
The Central Digital and Data Office (CDDO) monitors compliance with the Public Sector Bodies (Websites and Mobile Applications) (No.2) Accessibility Regulations 2018 and has recently released their report containing findings from the monitoring between February 2020 and November 2021.
Having been following the progression of the regulations through our own research regarding accessibility statements, disproportionate burden and more recently an in-depth look at FE colleges, we were excited to compare our findings with those of the monitoring body.
In this review of the CDDO report we will go through each section and pick out key findings which we feel are worth drawing attention to.
What CDDO monitored
In section 3 of the report, the CDDO outline what has been monitored over the February 2020 to November 2021 period.
The CDDO have had the task of starting up a team to complete monitoring of the UK for compliance with these regulations. Building the capability and sourcing the expertise to achieve this takes time. What the CDDO have achieved over the reported period does not match with what was committed to in the Monitoring and Enforcement Memorandum.
What they said
"From 2020, this body will be responsible for monitoring the compliance of a sample of websites and mobile applications based on population size (2056 websites with a simplified method, 113 with an in-depth method, and 72 mobile applications each year based on June 2018 ONS data)."
What they did
612 websites have been tested (593 by the simplified testing method and 19 by the detailed testing method). 2 mobile apps have been tested.
42 website cases were closed during the process, due to being exempt, as private sector, or organisations that were closing or merging. This brings the final number of websites tested to 570.
The MOU committed the CDDO to monitoring 2056 websites per year using the simplified testing. Over a period of 22 months they have achieved monitoring of 612 website where they should instead be closing on the 4000 mark to say nothing of the 19 detailed tests (of which only 4 are actually complete), or 2 mobile apps tests where after 5 months since the June 2021 deadline for mobile apps they should have tested 30. There are likely many factors to this outcome including recruitment times, developing processes, populating monitoring lists etc. but from our experience with the monitoring process in supporting our customers, we are sure CDDO would benefit from increased resources to help them meet their committed targets.
Another factor that obviously has some impact to the delivery has been Covid-19. The CDDO makes multiple mentions of the impact Covid-19 has had on the monitoring process which has clearly limited their ability to engage with certain organisation types.
"...due to the burden of extra coronavirus-related work and smaller organisations having problems making changes to their sites easily, we have focussed on larger public sector organisations..." (Section 3)
"Willingness and cooperation is usually dependent on organisation size and resources available. The pandemic has inevitably affected this." (Section 7.1)
There are some inconsistencies within the report when the CDDO refer to the number of mobile apps tested. On occasion they mention 4 mobile apps audits, and in others that only 2 apps were tested. It appears that 2 apps produced by the same supplier were tested across both iOS and Android.
Finally a worrying result given the CDDO are the monitoring body for the regulations and should quite accurately be able to define what is and what is not in scope of the regulations is the 42 closed cases due to being exempt, as private sector, or organisations that were closing or merging . This is especially concerning given the CDDO have focussed on "central government (including agencies and arm’s length bodies), larger local government and large and central health organisations" over this period, which arguably cover groups most clearly within scope of the regulations. It will be interesting to see if the CDDO produce any updated guidance on scope identification to help organisations following this experience.
What CDDO found
The CDDO gave useful breakdowns on the results of their monitoring for the simplified, detailed and mobile testing along with information on regularly occurring WCAG issues. Interesting to note are the following points made:
Simplified tests
A. "We found no accessibility issues for 8 websites, of which 5 had compliant accessibility statements. Approximately half of the websites tested had less than 5 issues, 75% of sites had less than 9 issues, and we found at most 26 issues on one site. As mentioned in the testing methodology, simplified tests only cover parts of WCAG and test on a small number of pages."
B. "...after 12 weeks, 229 websites (59%) had fixed the issues found or had a short-term roadmap to fix remaining issues, and 159 (41%) still had some issues remaining."
Section 4.1
Our comments
A. We want to reiterate here the final point that simplified testing only covers a tiny number of pages and mainly relies on automated testing and very basic manual tests. The relatively small number of issues including those with no issues is in great part due to small sample sizes. The more telling results are those of the detailed testing in the report.
B. Given the small sample sizes and small number of issues (79% having less than 9) it is concerning that 41% did not even have a roadmap after 12 weeks.
Detailed tests
"All four public sector websites [that were tested in detail] had non-compliant accessibility statements when they were initially tested. All had accessibility issues missing in the statement, and two had mandatory wording that was missing. "
Section 4.2
Our comments
As a reminder, the CDDO updated the required statement template in April 2020. A significant number of organisations use the outdated template and are missing mandatory wording changes.
Accessibility statements
CDDO said
"At time of initial testing, 39 sites (7%) had a compliant accessibility statement, 461 sites (83%) had a statement but with some mandatory information missing, and 55 (10%) had no statement.
After monitoring and correspondence, 319 sites (80%) had a compliant accessibility statement, 80 sites (20%) still had some mandatory information missing, and 2 (0.5%) had no statement."
Section 5
Our comments
It is interesting to see the CDDO figures in comparison to our own research. The compliance figure is slightly lower than our findings from 2020 which is concerning but coupled with the 83% from the CDDO findings showing statements missing information, perhaps this does show improvement (depending on how much is missing).
Another possible reason for the lower compliance scores is the CDDO counts a statement as non-compliant if it does not include issues found during their monitoring. This could suggest that organisations are not self-assessing their websites to an appropriate level, relying on automated tools etc.
The positive outcome is that the monitoring process yielded significant increases in the level of compliant statements, but shows that the problem still persists and only tough action is likely to change organisations' behaviours.
The CDDO found most common issues with accessibility statements include:
mandatory wording missing, including the wording to specify what the accessibility statement applies to
issues being found in CDDO tests which are not included in the statement
old versions of an accessibility statement still present that have not been updated since the regulations came into force, which do not resemble the required legislative format
organisations directly copying and pasting text from the sample statement template, and not replacing with their own organisation’s information
It is great to see the monitoring body identifying these common issues, which we have been on about for years and closely match issues which dictate our own All Able grading methodology.
The CDDO also said:
"It is not always clear where accessibility issues are found, and what the problems are. This means that users cannot easily understand where and how they may have issues using the website."
This is one of the biggest problems we see with most accessibility statements that are in the right format. Issues often say "some pages" or "some images" have an issue. Going into more detail on what aspects of the website and what user journeys are disrupted, how much and for what users, is something we all need to do to give better guidance.
All Able provide comprehensive support on testing your websites and writing compliant and useful accessibility statements. Our advanced support includes the ASPIRE review process provides comprehensive guidance to help institutions write a quality accessibility statement. We can help you at every step of the process through transparent scoring criteria, practical guidelines, and consulting services if a little extra help may be required.
Disproportionate burden
CDDO said
"Lack of time or knowledge does not constitute a disproportionate burden..."
"32% of websites have claimed disproportionate burden..."
Section 6
Our comments
We are glad to see this clarification from the CDDO. We often see organisations thinking about claiming for the difficulty of finding out whether they are breaking the law or not, which we have always maintained is not appropriate.
More shocking is the escalation of disproportionate burden claims. In our 2020 accessibility statements research we were aware of around 300 claims across 1800 orgs. For a third of those monitored by the CDDO to have made a claim is shocking given the ongoing lack of evidence supplied with claims.
The CDDO identified the following common issues when looking at disproportionate burden claims:
organisations copying the disproportionate burden claim directly from the sample accessibility statement
organisations claiming disproportionate burden without having carried out an assessment beforehand
misunderstanding of when the disproportionate burden exemption applies, as opposed to exemptions for certain pages and information from the accessibility regulations
These findings match those from our own disproportionate burden research and show that there is a persistent problem with the CDDO sample statement in which people see the example text, assume that it is a blanket exemption and copy it over. We have repeatedly pointed this out and encourage the sample statement to be updated with clearer guidance to stop this issue.
What was great to see is the list of things CDDO advise organisation's consider when making a claim. These match up well with advice we have been giving for some time, and All Able have helped many organisations with disproportionate burden claims as part of the monitoring process over the past year.
Lessons learnt
CDDO said
"The majority of organisations acknowledge our report within a month, but some organisations will not reply to us until after we inform them we are sending their information to the enforcement bodies"
"A minority (roughly 20%) never respond to our report or subsequent emails. "
Section 7.1
Our comments
Organisations are required to respond within 7 days of receiving the monitoring report.
That (approximately) 120 organisations completely ignored contact from the CDDO is staggering, and arguably the worst possible decision that could be taken.
These results together may show that organisations are not taking this as seriously as they should. The consequences of which is greater attention from the enforcement body later on.
It was quite odd to see the lessons learnt section so early on in the report. We would have expected this to be later on with so much to learn from the feedback, complaints and comms sections after this point of the report. This section was a missed opportunity as it only outlined an account of what happened. It did not specify any particular issues, did not dissect them and certainly did not offer any future actions that the CDDO will be taking to improve the situation.
Feedback
CDDO asked for feedback on the process from all organisations that were monitored. One result was the 10% of organisations said being monitored created 'unnecessary work'.
“It impacted ‘business as usual’ work and took valuable resources away from priority pandemic-related work.” (Section 8)
The pandemic should not be used to exclude disabled users and priority pandemic content should also be accessible. Trading one for the other is not an option. Lawsuits have already been brought over inaccessible Covid-19 information and many that are 'vulnerable' have disabilities meaning those that the 'priority pandemic-related work" affects the most are also the most likely to need accessible services. The pandemic has also led to more people accessing information digitally, which makes creating accessible digital content even more imperative.
Excluding people is not acceptable and accessibility should be seen as part of BAU work. The opposite, where organisations have shown a lack of engagement, suggests a lack of organisational maturity and points to discriminatory practice.
The CDDO also released these figures from their feedback where public sector organisations reported that they are dealing with the following barriers to meeting the accessibility regulations:
lack of awareness about the regulations (55%)
funding issues (53%)
lack of accessibility training (52%)
needing more accessibility guidance (45%)
difficulties collaborating with contractors (34%)
The lack of awareness and requirement for more guidance are two areas that we talk about further in the Comms section of this article, but are very concerned that organisations are still reporting high lack of awareness of the regulations. The regulations have been around for 3 years and this monitoring period has primarily focussed on groups which the CDDO have targeted with the most awareness raising activity. There is clearly still lots that needs to be done to better raise awareness and also to ensure that guidance is easier to find.
Collaborating with contractors is a consistent difficulty for many public sector bodies. Knowing the right questions to ask during procurement, how to check answers from suppliers, what content to have in contracts, and how to engage with suppliers successfully to make accessibility improvements are all challenging without the right expertise. All Able have a lot of expertise in supporting accessible procurement practices, working with suppliers and advising on 3rd party exemption options. You can also find many supplier accessibility statements on searchBOX.
Likewise, building that expertise across staff can be very time consuming but also a valuable key step in improving accessibility outcomes. There are lots of aspects of digital accessibility that training can focus on. We have talked about our priority areas for training in our training section of the website.
Complaints, compliance and enforcement
The CDDO have made it clear that all cases that were sent to them by EASS or ECNI did go on to be monitored. They also said:
"In one case, a complaint came through to the organisation through multiple channels and they commissioned an audit from an independent accessibility consultant. This audit was reviewed and we used this audit as part of our detailed testing. "
This is useful knowledge and shows that having useful evidence such as comprehensive audit reports can be accepted by the CDDO as part of the monitoring.
Further to this, the report also shows for the first time, more information on the enforcement processes. For example the two stage approach of the EHRC which starts off with more cordial contact and moves on to binding agreements. Also of keen interest to note is that EHRC have secured compliance without need for progressed enforcement action to date. This means that EHRC has been active, which is a question many have been asking.
Comms
What they said
"Government Digital Service (GDS), and then CDDO, completed a communications campaign from 2019 to tell the public sector about the new accessibility regulations, and to promote the published guidance to help the public sector to comply.
The campaign won a bronze award in the digital category of the Public Service Communications Awards 2020."
Section 10.4
What they did
CDDO monitoring feedback specifically showed organisations struggled with the following:
lack of awareness about the regulations (55%)
needing more accessibility guidance (45%)
This disconnect between the internal recognition for their campaign and results from the monitoring where near half or the respondents claim lack of awareness about the regulations and needing more guidance as barriers is especially striking given that the CDDO targeted monitoring at larger organisations matching those they aimed the most awareness raising activity toward.
Our most recent report on Further Education (FE) colleges with Thomas Pocklington Trust is the latest in a long line of attempted engagement with FE colleges in the UK. The FE sector has the worst results for accessibility statements by far with only 39 compliant statements out of 424 organisations (9%). In our research and previous experiences including personally emailing all college principles in the UK without a statement before the regulation deadlines, colleges are comparatively almost completely unaware of the regulations especially at leadership levels.
Going forward there are easy win opportunities for the CDDO to take more direct contact approaches to place the regulations in the awareness of public sector bodies' leadership teams. In addition to this, there are further opportunities for CDDO guidance on many challenging parts of compliance such as universities making captions 100% accurate. Presenting a centralised source of guidance on regulation challenges and practical steps towards compliance based on feedback in this CDDO report would be of great value to public sector bodies.
Summary
In conclusion there are several areas where the monitoring, support and behaviour of public sector organisations needs to improve.
There are opportunities for the CDDO to provide valued change off the back of this report:
A more detailed lessons learnt report and how the CDDO are going to improve the monitoring process would be highly relevant to public sector bodies forward planning in anticipation of monitoring activity.
Updated guidance on regulation scope, accessibility statement compliance, disproportionate burden and many other areas based on the findings of this report would help to better inform practical actions from public sector bodies.
More direct communications and awareness raising activity could lead to improved compliance across those parts of the sector which are lagging behind. Engagement across the accessibility community could help to further spread the word.
Public sector organisations also have opportunities to act to improve accessibility practices:
Update accessibility statements based on the current template
Improve practices for disproportionate burden claims and assessments
Consider staff skill gaps and accessibility training to improve services
Engage with the CDDO early on to remediate problems to avoid EHRC enforcement later down the line
A big thank you to the CDDO for publishing these findings. It helps us better understand where the digital accessibility community need to focus our efforts in support of improving services for end users and where we can help organisations better meet legal requirements, prepare for monitoring and better plan their own accessibility improvement activities.