Quality Assurance for Incident Response in the Age of AI

Incident investigation reports support critical decisions on system recovery, recurrence prevention, stakeholder communication, and public disclosure. However, victim organizations often lack the expertise to judge whether an investigation was sufficient or whether its conclusions are supported by evidence. JPCERT/CC worked with 13 Japanese incident response providers, corporate CSIRTs, and SOC teams to develop two checklists and a sample report for improving report quality. We then built an LLM agent that uses these materials to review incident investigation reports. It checks whether investigation objectives are answered, whether the investigated systems, logs, and time periods are clearly described, and whether major conclusions are supported by evidence IDs or log excerpts. This session presents the quality assurance model, the LLM-assisted review method, its limitations, and the incident response tasks that should remain under analyst control.

Location :
- Track 1(HALL B)
Category :
- General
Share :

Speakers

Shusei Tomonaga

朝長秀誠

Shusei Tomonaga is a CTO of JPCERT/CC. Since December 2012, he has been engaged in malware analysis and forensic investigation. In particular, he spearheads the analysis of targeted attacks affecting critical Japanese industries. In addition, he has written blog posts on malware analysis and technical findings (https://blogs.jpcert.or.jp/en/). He has presented at CODE BLUE, BsidesLV, Botconf, VB Conference, Hitcon, PHDays, PacSec, FIRST Conference, DEF CON, BlackHat ASIA, USA Arsenal and more.

Shusei Tomonaga is a CTO of JPCERT/CC. Since December 2012, he has been engaged in malware analysis and forensic investigation. In particular, he spearheads the analysis of targeted attacks affecting critical Japanese industries. In addition, he has written blog posts on malware analysis and technical findings (https://blogs.jpcert.or.jp/en/). He has presented at CODE BLUE, BsidesLV, Botconf, VB Conference, Hitcon, PHDays, PacSec, FIRST Conference, DEF CON, BlackHat ASIA, USA Arsenal and more.
Yuki Yano

矢野雄紀

Yuki Yano is a member of the Incident Response Group at JPCERT/CC. Prior to joining JPCERT/CC in July 2025, he worked on cybercrime investigations at a prefectural police department and later served as an Associate Professor at the National Police Academy, where he was responsible for training cybersecurity professionals. At JPCERT/CC, he is engaged in incident report handling, incident response support, malware analysis, and digital forensic investigations. He has presented at BlackHat USA Arsenal.

Yuki Yano is a member of the Incident Response Group at JPCERT/CC. Prior to joining JPCERT/CC in July 2025, he worked on cybercrime investigations at a prefectural police department and later served as an Associate Professor at the National Police Academy, where he was responsible for training cybersecurity professionals. At JPCERT/CC, he is engaged in incident report handling, incident response support, malware analysis, and digital forensic investigations. He has presented at BlackHat USA Arsenal.

Back to Time Table