Introduction

I attended the meeting of the Forensic Digital Media Analysis Working Group, organized and presented by the Chartered Society of Forensic Sciences, in Birmingham, UK on October 21, 2019.  One of the topics discussed was the state of general image comparison evidence in the UK in light of the FSR’s Regulatory Notice 01/2019 entitled Image Enhancement and Image Comparison: Provision of Opinion, which took effect in July 2019. 

This article must be read in conjunction with the following articles published in the journal Forensic Science International: Synergy in July, September and October 2019 respectively, all of which are in earlier articles on this website:

• Evaluation of the Forensic Science Regulator’s recommendations regarding image comparison evidence. Forensic Science International: Synergy 2019; 1(1).

• Commentary on: Hak. Evaluation of the Forensic Science Regulator’s recommendations regarding image comparison evidence. Forensic Science International: Synergy 2019; 1(1).

• Response to “Commentary on: Hak. Evaluation of the forensic science regulator’s recommendations regarding image comparison evidence. Forensic Science International: Synergy 2019; 1(1).”

As a result of the FSR Regulatory Notice, the three articles published in FSI: Synergy, the Birmingham meeting, and my discussions with the FSR, I believe the following accurately sets out the UK position moving forward regarding image analysis and comparison.

Image Comparison Evidence Problems Identified

The FSR seeks to address a series of problems regarding expert image comparison evidence that have been noted anecdotally.  These problems include the following:

• Conclusions being reached on image interpretation when the quality of the images does not support such conclusions.
• Cognitive bias impacting image interpretation.
• Lack of competence, insufficient training and qualifications.
• Peer review not being utilized.
• No meaningful proficiency testing or calibration of expertise.
• Interpretation of the meaning of image comparisons using an inappropriate scale.  [It should be noted that the FSR acknowledged that there is not an ideal scale for image comparison at the moment.]
• Not considering alternative explanations.
• Measurement uncertainty not being validly assessed and incorporated.
• Using methodology that has not been scientifically validated.

Acceptable Parameters for Image Interpretation

According to the FSR, a properly trained and qualified imagery expert is capable of providing expert evidence on the following topics:

• Guiding the court through CCTV images and explaining what is being shown.
• Interpreting technical aspects of images such as image quality, compression artefacts, resolution, frame rate, aspect ratio, and image enhancement.
• Highlighting individual similarities and differences on objects shown in the images, such as damage to a motor vehicle, though not if you are unaware of any linkage between the images.
• Explaining why objects may look different among various CCTV images.  For example, camera placement, image resolution, aspect ratio, compression, and infrared light may cause an object or scene to look different even though the same subject matter is being shown.
• Highlighting the relevance of shadows, the shape of objects, shading, and surroundings shown.
• Commenting on the apparent size of objects (including height determination) seen within an image but only if there is a valid and scientific method for estimating a range of uncertainty in those measurements.  An arbitrary error rate would not suffice.  

Scientific Comparison and Evaluation

The FSR states that for balance, the expert must address the findings (as found by the expert) given each of at least two competing propositions.  The propositions must be mutually exclusive.  Propositions are generally considered in pairs, where one represents the prosecution theory and the other, the defence theory.  The expert evaluates the probability of the findings given the propositions, not the probability of the propositions.  An example of such propositions:

• Proposition 1: The vehicle in the questioned CCTV images is the same vehicle seen in the known images (prosecution theory).
• Proposition 2: The vehicle in the questioned CCTV images is some other unknown vehicle (defence theory).

The expert then looks at the probability that these observations would have been made if the first proposition was true.  Secondly, what is the probability that these observations would have been made if the second proposition was true.

Assuming there were no differences that could not be accounted for by imagery artefacts, but little detail was visible because of poor image quality, then regarding the first proposition, is what is seen close to what would be expected if it was in fact the same vehicle?  Regarding the second proposition, is what is seen close to what would be expected if it was in fact a different vehicle?  The expert must then consider whether he/she has sufficient expertise to identify subtle differences in the vehicles and whether he/she has the expertise to assess whether they are actually different vehicles.

The FSR was critical of the “exclude/cannot exclude” approach to comparison work because it results in a statement that is often stronger than the evidence actually suggests and can lead to bias.  

Factors for the Court to Consider in Determining the Reliability of Expert Evidence

Pursuant to section 19.2 of the Criminal Procedure Rules, an expert is obliged to provide opinions that are objective, unbiased and within the expert’s area of expertise.  The expert is required to define his/her area or areas of expertise within the expert report and when giving evidence.  When giving evidence, the expert is required to draw the court’s attention to any question the answer to which would stray beyond the noted area/areas of expertise.  The FSR cautioned as to the importance of being very specific as to the expert’s area/areas of expertise within image analysis as the expert may not be an expert in all areas within the broader field. 

1. Criminal Practice Directions, Part V, 19A.5 (CPD)

Therefore, factors which the court may take into account in determining the reliability of expert opinion, and especially of expert scientific opinion, include: [Note: All comments in italics are based on annotations by the FSR]

(a) the extent and quality of the data on which the expert’s opinion is based, and the validity of the methods by which they were obtained;

The science used must be robust and validated.

(b) if the expert’s opinion relies on an inference from any findings, whether the opinion properly explains how safe or unsafe the inference is (whether by reference to statistical significance or in other appropriate terms); 

The interpretation must be balanced and clearly reported.

(c) if the expert’s opinion relies on the results of the use of any method (for instance, a test, measurement or survey), whether the opinion takes proper account of matters, such as the degree of precision or margin of uncertainty, affecting the accuracy or reliability of those results; 

Robust science requires validation, particularly when considering measurements of height and size, and also factoring in uncertainty due to poor image quality.

(d) the extent to which any material upon which the expert’s opinion is based has been reviewed by others with relevant expertise (for instance, in peer‐reviewed publications), and the views of those others on that material; 

This is part of robust science.

(e) the extent to which the expert’s opinion is based on material falling outside the expert’s own field of expertise; 

The expert must clearly define areas of expertise and support such expertise with sufficient evidence.

(f) the completeness of the information which was available to the expert, and whether the expert took account of all relevant information in arriving at the opinion (including information as to the context of any facts to which the opinion relates); 

The interpretation must be balanced.

(g) if there is a range of expert opinion on the matter in question, where in the range the expert’s own opinion lies and whether the expert’s preference has been properly explained; and 

This requires clear reporting including any limitations on the opinion and weaknesses in the evidence.

(h) whether the expert’s methods followed established practice in the field and, if they did not, whether the reason for the divergence has been properly explained.

Robust science requires clear reporting.

2. Criminal Practice Directions, Part V, 19A.6 (CPD)

In this section of the CPD, factors that suggest opinion unreliability are noted.

In addition, in considering reliability, and especially the reliability of expert scientific opinion, the court should be astute to identify potential flaws in such opinion which detract from its reliability, such as: 

(a) being based on a hypothesis which has not been subjected to sufficient scrutiny (including, where appropriate, experimental or other testing), or which has failed to stand up to scrutiny; 

(b) being based on an unjustifiable assumption; 

(c) being based on flawed data; 

(d) relying on an examination, technique, method or process which was not properly carried out or applied, or was not appropriate for use in the particular case; or 

(e) relying on an inference or conclusion which has not been properly reached.

3. Declarations

All expert reports must include the declarations of truth and compliance as required by section 19B CPD, as follows:

19B.1 The statement and declaration required by CrimPR 19.4(j), (k) should be in the following terms, or in terms substantially the same as these: 

‘I (name) DECLARE THAT: 

1. I understand that my duty is to help the court to achieve the overriding objective by giving independent assistance by way of objective, unbiased opinion on matters within my expertise, both in preparing reports and giving oral evidence. I understand that this duty overrides any obligation to the party by whom I am engaged or the person who has paid or is liable to pay me. I confirm that I have complied with and will continue to comply with that duty. 

2. I confirm that I have not entered into any arrangement where the amount or payment of my fees is in any way dependent on the outcome of the case. 

3. I know of no conflict of interest of any kind, other than any which I have disclosed in my report.

4. I do not consider that any interest which I have disclosed affects my suitability as an expert witness on any issues on which I have given evidence. 

5. I will advise the party by whom I am instructed if, between the date of my report and the trial, there is any change in circumstances which affect my answers to points 3 and 4 above. 

6. I have shown the sources of all information I have used. 

7. I have exercised reasonable care and skill in order to be accurate and complete in preparing this report. 

8. I have endeavoured to include in my report those matters, of which I have knowledge or of which I have been made aware, that might adversely affect the validity of my opinion. I have clearly stated any qualifications to my opinion. 

9. I have not, without forming an independent view, included or excluded anything which has been suggested to me by others including my instructing lawyers. 

10. I will notify those instructing me immediately and confirm in writing if for any reason my existing report requires any correction or qualification. 

11. I understand that:

(a) my report will form the evidence to be given under oath or affirmation;

(b) the court may at any stage direct a discussion to take place between experts; 

(c) the court may direct that, following a discussion between the experts, a statement should be prepared showing those issues which are agreed and those issues which are not agreed, together with the reasons; 

(d) I may be required to attend court to be cross-examined on my report by a cross-examiner assisted by an expert. 

(e) I am likely to be the subject of public adverse criticism by the judge if the Court concludes that I have not taken reasonable care in trying to meet the standards set out above. 

12. I have read Part 19 of the Criminal Procedure Rules and I have complied with its requirements.

13. I confirm that I have acted in accordance with the code of practice or conduct for experts of my discipline, namely [identify the code] 

14. [For Experts instructed by the Prosecution only] I confirm that I have read guidance contained in a booklet known as Disclosure: Experts’ Evidence and Unused Material which details my role and documents my responsibilities, in relation to revelation as an expert witness. I have followed the guidance and recognise the continuing nature of my responsibilities of disclosure. In accordance with my duties of disclosure, as documented in the guidance booklet, I confirm that: 

(a) I have complied with my duties to record, retain and reveal material in accordance with the Criminal Procedure and Investigations Act 1996, as amended; 

(b) I have compiled an Index of all material. I will ensure that the Index is updated in the event I am provided with or generate additional material; 

(c) In the event my opinion changes on any material issue, I will inform the investigating officer, as soon as reasonably practicable and give reasons. 

I confirm that the contents of this report are true to the best of my knowledge and belief and that I make this report knowing that, if it is tendered in evidence, I would be liable to prosecution if I have wilfully stated anything which I know to be false or that I do not believe to be true.’

4. FSR Codes of Practice and Conduct – Appendix: Digital Forensics – Video Analysis (Issue 1)

Further instruction is as follows:

7.5.1 Providers who undertake image comparison must do the following. 

1. Demonstrate the appropriate competence in relation to the image-based processes that have been undertaken in addition to demonstrating competence in comparison. 
2. To reduce the risk of confirmation bias, incident footage containing unknown persons or objects of interest shall be analysed to identify distinguishing features before known footage of the suspect objects of interest is viewed and information revealed to the analyst.
3. Ensure that all relevant information in relation to image processing undertaken by a third party is communicated to the person undertaking the comparison. 
4. Demonstrate the decision process and basis for critical findings. 
5. Demonstrate that the methods used for comparison are appropriate, through validation, for the image characteristics of the case material. For example, methods developed for high quality recordings may not be valid for low quality CCTV images. 

8.1 Validation Introduction

8.1.1 The method shall be validated, or any existing validation to be verified, as laid out in the Codes. The functions used in hardware and software tools where operation has an impact in obtaining results are to be validated as part of that validation of the method.

8.3 Image Comparison

8.3.1 All methods designed for image comparison require validation, where the comparison uses proportional relationships and/or metrics the validation shall include an appropriate, robust and repeatable method for quantifying the associated uncertainties (see 9.1 Photo/Videogrammetry).

The FSR noted that validation includes the evaluation of the underlying science, model or theory.  A literature search is usually involved.  Any limitations or risks must be disclosed.  If limitations have been addressed via the methodology used and have been validated, that must be stated.

Producing Expert Reports and Providing Testimony in Court

Based on all of the material discussed in this article, there are four scenarios under which an expert may be called to give expert testimony.  Each scenario has different parameters and limitations.  An imagery expert must give due consideration as to which scenario applies to a given case.

Scenario 1 – The imagery expert is also an expert in the subject matter being compared

This scenario represents the exception rather than the rule as it would be uncommon for an imagery expert to also be an expert in the subject matter being compared.  However, if the expert is so qualified and has properly set out the qualifications for claiming expertise in both areas, then the expert is entitled to provide opinion evidence on image comparison.

Scenario 2 – The imagery expert claims subject matter expertise as a result of case-specific research

Cases may arise wherein the expert did not have expertise in the subject matter being compared prior to commencing the analysis but as a result of extensive research, the expert is of the view that such a level of expertise has been achieved.  There is no set standard to determine when expertise can be claimed as this is a subject assessment based upon the expert’s education, training, skill, and experience.  Suffice it to say that if an expert claims expertise in the subject matter being compared, that must be objectively reasonable and supportable in order for the opinions to have any merit and thus carry any weight.

Because a court may disagree with the expert’s claim of subject matter expertise, an imagery expert is well advised to have a Plan B to fall back upon, those being Scenarios 3 or 4 as appropriate. 

Scenario 3 – The imagery expert provides a comparison opinion on subject matter where no expertise is claimed

It is this scenario that is at the heart of the disagreement between the FSR and me.  The FSR has stated that an expert is not qualified to give any opinion evidence where there is no subject matter expertise.  As noted in the above articles published in FSI: Synergy, I am of the view that a properly qualified imagery expert can make a meaningful contribution by conducting a sound, forensic comparison of questioned to known images even without expertise in the subject matter being compared.

The FSR has stated that if an expert provides a comparison opinion in such circumstances, the expert must do the following:

• State what qualifications and experience are being relied upon (s. 19.4(a) CPR).
• State that the expert is operating contrary to the FSR’s Regulatory Notice (s. 19.2(3)(d)(ii) CPR).
• Advise whether the expert has a lack of commitment to prescribed standards where that might be expected (CPD, Part V, Section 19A.7(h)).
• Advise whether the expert has a history of failure or poor performance in quality or proficiency assessments (CPD, Part V, Section 19A.7(i)).

The FSR’s role is to make recommendations and set standards.  These do not yet have the force of law.  As a result, the failure to follow the FSR’s directions do not equate with legal inadmissibility.  Only a judge can decide that.  To be clear, I am not suggesting that experts should ignore the Regulatory Notice.  Rather, I am suggesting that if you decide to give comparison opinion evidence on subject matter in which you have no expertise, this is the approach that the FSR recommends.  Some experts do provide expert comparison opinions on subject matter in which they claim no expertise but then clearly state in the report the qualification that while they are imagery experts, they do not claim expertise in the subject matter being compared.  That is then something that must be weighed by the court when considering the evidence.

Scenario 4 – The imagery expert does not provide a comparison opinion on subject matter where no expertise is claimed

This is the FSR’s recommended approach.

The expert is entitled to provide a full analytical opinion of all technical aspects of the images, as outlined earlier in this article.  This is properly expert opinion testimony.  Further, the expert can point out all observations made regarding the questioned and known images.  This would constitute factual testimony, not expert opinion, because the imagery expert is not an expert in the subject matter contained in the images.  I note parenthetically that the distinction between fact and opinion is often difficult to discern.  Using this approach, the expert does not provide any opinion as to the relationship between the questioned and known images.

This approach has merit.  I have always been of the view and have long espoused in LEVA training that it is not the role of a forensic video analyst to solve the case.  The role of the expert is to fully interrogate and interpret the images and provide opinions that are fair, reasonable and within the expert’s qualifications.  It is then up to counsel to use that evidence as building blocks in arguing the case before the court and in turn, for the court to use in deciding the case.  Under this scenario, the work of the imagery expert remains very important in the analysis of the video images.   

Conclusion

The UK has experienced challenges in the provision of forensic science services in relation to image analysis, interpretation and comparison.  In accordance with her mandate, the FSR has established guidelines and standards for the provision of such services.  It is of critical importance that forensic service providers be aware of these and strive to comply where feasible.  This article is intended to provide assistance in achieving this goal.