Research outline
Evaluating AI-assisted oncology records.
A computational research outline in bioinformatics and molecular oncology knowledge representation, evaluating AI extraction from synthetic oncology records and the corrections users make when checking the results.
Research question
How can gene variants, molecular targets and signaling pathway references be represented alongside longitudinal oncology records? How accurately can AI structure fragmented records, and how do user verification and correction affect field accuracy, temporal consistency and checking effort?
The scope is faithful information extraction and record organization. Diagnosis, treatment recommendations and clinical outcomes are outside this evaluation.
Data and reference annotations
The evaluation design uses completely fictional oncology narratives and reports. Cases cover multiple treatment stages, missing dates, numerical values and units, reported gene variants, explicit negatives, repeated reports and conflicting information.
Reference annotations are defined before inspecting model outputs. They record the fields supported by each source, the correct treatment stage and information that should remain unknown. The annotation process and contributors will be documented with the dataset.
Comparison workflow
- Fix the source texts, field definitions, model version, prompts and run settings.
- Save the initial AI extraction. The person operating MyOncode checks it against the synthetic source, makes corrections and records the checking time.
- Compare both the initial and corrected records with the reference annotations, separating omissions, incorrect values, unsupported additions and treatment-stage errors.
- Keep development cases separate from the evaluation set and record changes to prompts, fields and model versions.
Measures and reporting
- Field-level precision, recall and F1, with explicit matching and missing-value rules.
- Unsupported additions and assignment to the wrong treatment stage.
- Fields corrected by users, time spent checking and errors remaining after correction.
- Failure cases grouped by input difficulty, with the dataset size and evaluation settings reported alongside the results.
- Source provenance for molecular reference content, distinguishing cited relationships from interpretations.
Sample sizes, model configurations and the first evaluation version will be fixed before the evaluation run. Results will be reported for the tested synthetic cases.
User verification and educational content
Patients and caregivers check the correctness of their own AI-extracted information in the product. The evaluation measures this user verification workflow.
Totoro, Founder & Research Lead at MyOncode, reviews the project's educational content on gene variants, molecular targets and signaling pathways. Educational content review is distinct from checking a user's record.
Expected outputs
The 6–12 month research plan is organized around an evaluation protocol, an annotated synthetic benchmark, reproducible scripts and open-source workflows with a technical report. Google Antigravity is the intended workspace for evaluation code, failure-case analysis and development of extraction and molecular reference workflows.