PillGaze

Patients carry inaccurate lists of their medicines. With our service, they simply take a photo of their medicines and the list is securely transferred to the doctor’s computer, where it can be edited.

“A white tablet like that, in the morning.” That’s how the mystery begins – one that steals doctors’ time every day. PillGaze reduces it to a photo and a four-digit PIN.

The patient takes a photo of the boxes or a list of medicines. The service first uses a simple, fast AI model to transcribe the names, strengths and dosages into a clear list. The patient checks it and, using their PIN, sends it to the doctor’s computer. The transfer is sufficiently secure and does not require the patient to enter their name or national insurance number. The doctor edits the list and copies it into the patient’s records.

The second service utilises a team of AI agents that check for possible interactions between the patient’s medicines. They simultaneously search for information on Drugs.com, RxList and Medscape and prepare a summary of their findings for the doctor, complete with links to the sources.

We draw data from the SÚKL’s open catalogue for names, strengths and active ingredients, and from HemOncKB for the composition of cancer treatment regimens. For development and testing, we had photographs of packaging and sample lists from the brief, including errors and incomplete data.

For the hackathon, we built a workflow: photo → digital list → PIN → doctor’s computer, including both AI services and an editable medical interface. Next, we want to verify the accuracy and time savings in outpatient clinics, clinically evaluate the results, and finalise the PIN expiry and automatic data deletion features.

The patient takes a photo. Swarm searches. The doctor decides.