The Quality Assessment of LLM in Digital Forensics

The utilisation of Artificial Intelligence in investigative contexts facilitates the optimisation of the analysis of digital artefacts, which can be collected in a variety of formats, including text, printouts, images, and videos, as well as from diverse sources, such as datasets, networks, and devices. More recently, while the adoption of LLM (Large Language Models) appears to hold significant potentials, it is also accompanied by emerging concerns relating to its technological limitations, including distortions and hallucinations, which can hinder law enforcement and compromise the balance between the rights of the accused and the power of the judicial authorities. Notwithstanding the activities prohibited by Art. 5 of the AI ACT, the fact is that LLM technologies are already assuming a pivotal role in investigations, yet are still inadequate the principles, methodologies and tools allowing courts to assess accuracy and precision of the language models before implementing them. The objective of this research project is to develop a methodology that can be adopted for the assessment and comparison of LLMs in performing analytical processes. To this end, a synthetic dataset of electronic evidence is generated from a fictitious criminal scenario and fed to the agent with engineered prompting techniques.