This report, Threlfall,J., Nelson,N. and Walker,A. (2007). Report to QCA on an investigation of the construct relevance of sources of difficulty in the Key Stage 3 ICT tests, was published in 2007 and used to be available from http://www.naa.org.uk/libraryAssets/media/Leeds_University_research_report.pdf. Unfortunately, this link is now broken and it does not seem to be possible to find the original report on the NAA website and I am therefore making it available here.
The work was originally commissioned by QCA in the context of a plan to make testing of ICT skills compulsory for all 14 year old students in England from 2008. The tests were piloted but not, in the end, implemented. The Leeds team was commissioned to explore the question of whether on-screen testing might involve non-construct relevant sources of difficulty that are different from those in paper tests. Our conclusion was that sources of difficulty are different on on-screen and paper-based tests and we proposed frameworks to illustrate the sources of difficulty in on-screen tests. It is important to note that, although this research was conducted on the KS3 ICT pilots, the findings and frameworks apply to on-screen testing in general rather than the KS3 ICT tests alone.