Introduction
Access to accurate and user-friendly medication information remains a significant challenge in many parts of the world including India among other countries where this is common. This issue is compounded by linguistic diversity and limited health literacy, especially in rural areas.
Most individuals rely on standard drug labels printed in a single language, which can be difficult for non-native speakers or those with limited education to understand. This often forces them to depend on others for critical information. In such populations, people frequently identify medications by their physical packaging, a risky practice as healthcare increasingly shifts to digital platforms.
For example, a study of Hispanic parents in the U.S. with limited English proficiency and health literacy found that they were twice as likely to make errors in dosing children’s liquid medications. This highlights the urgent need for accessible healthcare information tailored to diverse linguistic and educational needs.
To address this problem, we analyzed consumer behavior in India throughout the medication journey— from obtaining a prescription from a doctor to purchasing medicines and adhering to prescribed dosages. Our observations revealed that elderly individuals and non-English speakers often struggle with understanding dosage instructions and rely heavily on external assistance. Many also purchased medications directly from local pharmacies using only the packaging as a reference.
In response, we proposed a solution for healthcare assistants and consumers that leverages Llama models available through AWS Bedrock. Specifically, we utilize the Llama 3.2 vision model for its advanced vision and multilingual capabilities, enabling the extraction of critical information from invoices and medicine wrappers accurately. This enables us to guide users on dosage instructions in their preferred language, providing accessible, easy-to-understand medication guidance and empowering users to follow prescriptions confidently and independently.
Prescription and Medicine Wrapper Analysis
Our application, as seen in Figure 1, leverages AWS Bedrock to upload photos of prescriptions or medicine wrappers through WhatsApp, enabling advanced analysis in their preferred language. With AWS Bedrock’s serverless infrastructure, we utilized the state-of-the-art open-source Llama 3.2 model to extract key information such medicine names, dosages, and instructions directly from images.
This extracted information is cross-referenced with medically verified data securely stored in a Qdrant vector database on AWS EC2. The process ensures high accuracy and reliability. AWS Bedrock enables a clear, simplified process, where the Llama 3 language model further refines and translates the response into the user’s desired language, ensuring a friendly, linguistically adaptable experience that eliminates prompt complexity. AWS Bedrock’s integration here is key, making it possible to efficiently handle both image and text analysis with sophisticated, scalable AI.
Medicine Ordering Solution Using Prescription and Medicine Wrapper
For ordering medications, AWS Bedrock remains a cornerstone, managing both image and language inputs. Users can specify quantities and provide additional details in their native language, which the application processes via Bedrock’s Llama 3.2 model. The system extracts and interprets necessary details with precision, verified against medical data stored in Qdrant on AWS.
Once verified, the application automates the ordering process by making API calls to online pharmacies via function calls enabled on AWS Bedrock. Alternatively, users can choose to place orders with registered local pharmacists stored in a relational database. This flexibility ensures that users can access medications in a way that best suits their needs.
After the order is confirmed, the Llama models enable smooth interaction by sending an order confirmation message via WhatsApp. Payment are seamlessly handled through WhatsApp Pay. This Bedrock-powered flow not only makes prescription management and ordering intuitive but also ensures a secure, reliable, and language-friendly experience for users.
By leveraging Llama models through AWS Bedrock, our solution simplifies prescription analysis and medication ordering into an intuitive and accessible process. The advanced vision and multilingual capabilities of Llama 3.2 enable accurate extraction of medication information from images, while providing dosage instructions in users’ preferred languages.
Coupled with automated ordering via pharmacy APIs or local pharmacists and integrated payment through WhatsApp Pay, this solution addresses accessibility challenges faced by elderly and non-English-speaking users. It empowers individuals to confidently manage their medications, bridging critical gaps in language, healthcare literacy, and technology by introducing AI in healthcare. To learn more about our innovative solutions impacting healthcare, contact us.
