AIF-C01 · MLA-C02
RAG vs fine-tuning vs prompt engineering
When to use prompt engineering, retrieval augmented generation or fine-tuning to adapt a foundation model, with cost and effort trade-offs for AWS AI exams.
Three ways to make a model fit your task
A foundation model knows a lot in general but nothing about your company, and it may not answer in the style you need. There are three common ways to close that gap: change the prompt, give the model relevant documents at query time, or change the model's weights.
Those are prompt engineering, retrieval augmented generation (RAG) and fine-tuning. They are listed in order of effort and cost, and the usual advice is to try them in that order.
Prompt engineering
Prompt engineering means writing better instructions. You set a role, give context, show examples and specify the output format. Nothing about the model changes.
Common techniques include zero-shot prompting (just the instruction), few-shot prompting (a handful of worked examples in the prompt) and chain-of-thought prompting (asking the model to reason step by step). Inference parameters such as temperature also shape the output: lower values give more predictable answers.
Prompt engineering is cheap and fast to change. Its limit is the context window and the model's existing knowledge. A prompt cannot teach a model about a product launched after its training cutoff unless you paste the facts in.
Retrieval augmented generation (RAG)
RAG adds a search step before the model answers. Your documents are split into chunks, turned into vector embeddings and stored in a vector database. When a user asks a question, the system finds the most relevant chunks and adds them to the prompt. The model then answers using that text.
On AWS, Amazon Bedrock Knowledge Bases handles this pipeline: it connects to data sources such as Amazon S3, creates embeddings, stores them in a vector store such as Amazon OpenSearch Serverless, and retrieves passages at query time.
RAG is the right choice when answers depend on private or frequently changing information. An HR assistant answering from the current leave policy is a classic example. Update the policy document and the next answer reflects it, with no retraining. RAG can also cite sources, which helps users check answers and reduces hallucinations.
Fine-tuning
Fine-tuning continues training a pre-trained model on your own labeled examples, such as prompt and ideal-response pairs. The model's weights change, so the new behavior is built in.
It suits teaching a consistent style, tone or format, or improving performance on a narrow task like classifying support tickets into your categories. Amazon Bedrock supports fine-tuning for some models. A related method, continued pre-training, uses unlabeled domain text (for example, a large set of legal documents) to make a model more familiar with a field.
Fine-tuning costs more. You need good training data, compute for training, and often dedicated capacity to host the custom model. And because knowledge is frozen into the weights, the model goes stale when facts change and must be retrained.
Side-by-side comparison
A quick way to remember the differences:
- Prompt engineering: no training, lowest cost, changes take seconds. Best for format, instructions and simple tasks.
- RAG: no training, moderate cost for the retrieval pipeline. Best for up-to-date or private knowledge and answers with sources.
- Fine-tuning: training required, highest cost and effort. Best for specialized behavior, style or domain language that prompts cannot achieve.
- Training from scratch: rarely the right answer on these exams; it needs huge datasets and budgets.
Combining them
These are not exclusive. A support chatbot could use a fine-tuned model that writes in the company's voice, RAG to pull the latest product manuals, and a carefully written system prompt that tells it to say "I don't know" when the retrieved text does not cover the question.
On the exams, read for the requirement that rules options out. "Information changes daily" rules out fine-tuning alone. "Lowest effort" or "no additional training" points to prompt engineering or RAG. "Must respond in a specific style across thousands of requests" or "improve accuracy on a specialized task" points to fine-tuning. "Reduce hallucinations by grounding answers in company documents" points to RAG.
Practice questions
An online tutoring company has an AI model in production that answers student questions. The operations team already knows how accurate the answers are and now wants to track how efficiently the deployed model performs while it serves live requests. Which metric should the team track?
- Average response time
- Training time for each epoch
- Customer satisfaction score (CSAT)
- Number of training instances
Answer
A. Efficiency at run time is about how fast the running model answers, so average response time is the direct measure. Training time per epoch is a speed figure too, but it describes the training phase, and so does the number of training instances. Customer satisfaction is a business outcome.
AIF-C01 question 15
A farm equipment maker wants operators of its tractors and harvesters to ask a language assistant about fault codes and machine settings while they work in fields that have no mobile coverage. The operators need answers almost instantly. Which approach meets these requirements?
- Incorporate a centralized small language model (SLM) API for asynchronous communication with the vehicles
- Deploy optimized small language models (SLMs) on the vehicles' onboard hardware
- Incorporate a centralized large language model (LLM) API for asynchronous communication with the vehicles
- Deploy optimized large language models (LLMs) on the vehicles' onboard hardware
Answer
B. No coverage in the field rules out both centralized APIs, since they fail as soon as the machine is out of range. The model has to live on the vehicle. An optimized small language model fits the limited onboard hardware and answers with the lowest latency, while an optimized large language model would need far more memory and compute and respond more slowly.
AIF-C01 question 10
A roadside assistance provider is adding a large language model (LLM) assistant for its phone dispatchers. Today a dispatcher works through several systems during each call to find coverage details and the nearest tow partner, and the assistant is meant to remove most of those steps. Which business objective should the provider use to judge whether the assistant is working?
- Average call duration
- Website engagement rate
- Regulatory compliance
- Corporate social responsibility
Answer
A. Fewer steps per call should show up as shorter calls, so average call duration is the business measure to watch. Website engagement rate is easy to collect, but it describes how visitors use the website and has no link to how fast dispatchers handle calls.
AIF-C01 question 42
A veterinary medicine company built an assistant on a foundation model (FM) from Amazon Bedrock. The assistant answers staff questions by drawing on a large library of pharmacology research papers. Its answers are poor because the papers are dense with complex scientific terms, and many rounds of prompt revisions have not helped. What should the company do to improve the assistant?
- Use domain adaptation fine-tuning to adapt the FM to complex scientific terms.
- Change the FM inference parameters.
- Use few-shot prompting to define how the FM can answer the questions.
- Clean the research paper data to remove complex scientific terms.
Answer
A. Prompt engineering has already failed, and few-shot prompting is just more of it. Domain adaptation fine-tuning continues training the FM on text from the field, so it learns the scientific vocabulary prompts couldn't teach. Stripping the scientific terms out of the papers would remove the very information staff are asking about.
AIF-C01 question 64