PatoScan

Prostate cancer in data, not on paper. The system converts medical reports and diagrams into data to enable a comparison of histology with magnetic resonance imaging.

PatoScan converts paper-based prostate cancer reports into structured data. Instead of manually transcribing the information, the doctor reviews a pre-filled draft — with the potential to reduce case processing time from 30 to 3 minutes. The result is data that enables the systematic comparison of histology with magnetic resonance imaging.

How it works The user uploads a histology report and the corresponding diagram in JPG, PNG or PDF format, either individually or in batches. OCR scans the documents and, based on the recognised identifiers, the system verifies that they are correctly paired.

The image processing pipeline normalises the lighting and uses the SIFT algorithm to align the scan with the reference template. By suppressing the pre-printed background, it highlights the hand-drawn markings. Geometric analysis then determines their overlap with the 41 anatomical segments of the COMPARE registry.

In a single structured request, we pass the entire OCR text, the original diagram, the aligned image with numbered segments and the geometric map to the multimodal AI. The model links the text to the diagram and suggests data on the tumour, Gleason scores, dimensions and localisation. The response must adhere to the prescribed JSON schema; the backend then verifies citations, values and localisation rules.

A key challenge was the inconsistency of the inputs: the orientation of the right and left sides may vary between views, the text may not correspond to the diagram, and markings are often faint or overlap. We therefore determine the affected regions based on the shared context of the text and the image. A coloured marker alone is not evidence of a tumour. We distinguish between documented data and inferred suggestions, and leave any discrepancies visible for verification.

What data did we use? Anonymised histological reports, 40 scanned prostate diagrams and the COMPARE data dictionary provided by the Masaryk Memorial Cancer Institute. We utilised pre-trained OCR and AI models; our own application was developed during a hackathon.

What we achieved In 24 hours, we created an integrated prototype, from document upload through OCR, image processing and AI evaluation, right through to the backend API and a clear, intuitive interface. Users can compare the report, diagram and results side by side, view source citations, correct data and regions, and export the results to JSON.

We aim to verify the time savings in real-world use. The benefits include less manual transcription, more consistent data and traceable sources for individual data points. The final decision remains with the doctor.

What’s next Expert validation with pathologists, measuring accuracy and time savings, and completing the direct import of approved data into REDCap. We are currently exporting to a file. The next step is also to verify operation within the hospital’s infrastructure; the current AI evaluation utilises the Azure API.