You already write Java for a living. You can build AI features without leaving it. Here is the plan.

This series is for working Java developers. You know Spring Boot and microservices. You know little or nothing about AI, and that is fine. We start from zero on the AI side and use what you already know on the Java side.

The promise is small and honest. You will get a clear path in four stages. You will build something at every stage. By the end you will have one project you can show to a hiring manager or to your own team.

The whole path takes about 28 weeks if you follow the pace we suggest. That is a suggestion, not a rule. You may go faster or slower, and both are fine.

This is Part 1 of the series. It is the map. Every later part links back here, so you can always see where you are.

The roadmap at a glance

There are four stages. Each one has a short list of things to learn and one thing to build. The weeks are suggested pacing for someone with a full-time job. They are not a promise.

Stage	Suggested weeks	What you learn	What you build
1. Foundations	1 to 6	How LLMs work, embeddings, just enough Python and maths	A program that calls one model from Java and Python and logs tokens and cost
2. LLM apps and RAG	7 to 14	Prompting, structured output, streaming, retrieval	A document Q&A service in Spring Boot that shows its sources
3. Agents and MCP	15 to 22	Tool calling, the agent loop, MCP, human approval	A ticket agent that drafts replies and waits for approval
4. Production	23 to 28	Evals, tracing, cost budgets, guardrails	Evals and tracing for your agent, plus a short write-up

The total is about 28 weeks. Treat that number as a guide.

How should you use the table? Read the stage you are in, and ignore the rest for now. Each stage ends with a short list called "You are done when". Move on only when you can tick those lines. If a stage takes longer than the suggested weeks, that is normal. The order matters more than the speed.

Figure 1: The four stages, with suggested weeks under each one.

A few new words showed up in that table. LLM means large language model, the kind of model behind chat assistants. RAG means retrieval-augmented generation. In plain terms, your app looks up your own data first, then asks the model to answer using it. Do not worry about them yet. Each one gets a plain explanation when we reach it.

Why Java developers have a head start

AI work looks new, but a lot of it is normal software work. Three parts of your background carry over directly.

You already work with typed APIs. An AI model is a service you call over HTTP. You send a request and you get a response. The twist is that the response is text, and text is messy. A big part of AI engineering is turning that text into typed data your code can trust. You already think in types, so this will feel natural.

You already test code. AI features fail in odd ways, so testing matters even more here. The habit of writing a test before you trust a change is what separates a demo from a product. You have that habit.

You already run services in production. You know about timeouts, retries, logs, metrics and on-call. Model calls are slow, they cost money and they sometimes fail. You have solved those problems before with other services.

Now the honest part. Your head start is real, but it does not cover everything. You still need to learn how models behave, because they are not deterministic. The same input can give different output. You need to learn how to search by meaning instead of by keyword. And you need to learn how to measure quality when there is no single right answer. That is new, and it takes practice.

Tip

Do not rush to learn a lot of Python. You need just enough to read AI examples and notebooks. Your main language stays Java.

What an AI engineer actually does

An AI engineer builds software that uses AI models. That is the whole definition.

It helps to say what the job is not. An AI engineer does not train models from scratch. They do not do research on new model designs. Those jobs exist, but they are different jobs with different skills.

The daily work looks like this:

• Calling models from code and handling the answers.
• Retrieving data so the model can answer from facts you control.
• Giving models tools, so they can look things up or take actions.
• Testing the results, and watching them in production.

Notice how much of that is ordinary engineering. You design interfaces, handle errors and measure what happens. The model is one component in a bigger system.

Here is a small example. A company wants a helper that answers customer questions from its own manuals. The AI engineer does not build a new model. They choose an existing one, find the right manual pages for each question, and send those pages with the question. Then they check the answers, log what happened and keep an eye on cost. That is the job.

Before you start

You need a small setup. Nothing here is exotic, and you may already have most of it.

• A current long-term support (LTS) release of Java.
• Spring Boot, with a project you can build and run.
• Python, only for experiments and notebooks.
• Git.
• An account with one hosted model provider, or a local model runner such as Ollama. A model runner is a tool that downloads a model and runs it on your own computer.
• A Postgres instance. A Docker container is enough.

Why Python at all? Most AI tutorials, notebooks and research code are written in it. You will read a lot of it. You will not need to write much. Why Postgres? It is a database you probably know already, and it can store vectors for search by meaning once you add an extension.

We do not name specific models in this series. They change too fast. When we say "a hosted model", we mean any model you reach through a provider's API. When we say "a small model", we mean one small enough to run on your laptop.

Check versions

This area moves fast. Read the current docs before you copy anything. Here is what was true when this article was written.

• Spring AI 2.0.0 reached general availability on 12 June 2026. It is designed for Spring Boot 4.0 and 4.1 and Spring Framework 7.0, and it ships with MCP Java SDK 2.0.0. See the release announcement.
• The Spring AI reference docs still list older 1.x versions. If your team is on an older Spring Boot, check which Spring AI version matches it in the reference docs.
• The current MCP specification revision is 2026-07-28. Its main change is a stateless core. See the versioning page and the changelog.
• Libraries can trail the spec. The Spring AI announcement says its release follows the 2025-11-25 revision. Always check which revision your library supports.

Stage 1: Foundations (suggested weeks 1 to 6)

This stage is about vocabulary and intuition. You are not building anything big yet. You are learning enough to know what is happening when you call a model.

What you will learn

• Tokens. Models do not read characters or words. They read tokens, which are small chunks of text. Pricing and limits are counted in tokens.
• The context window. This is the maximum amount of text a model can look at in one call. Anything outside it does not exist for the model.
• Sampling and temperature. A model picks its next token from a list of likely options. Temperature controls how adventurous that pick is.
• Embeddings. An embedding turns text into a list of numbers. Texts with similar meaning get similar numbers. That is how search by meaning works.
• Vector similarity. To compare two embeddings, you measure how close the two lists of numbers are. Close means similar meaning.
• Just enough Python and maths. You need to read Python notebooks. You need the intuition behind vectors and probability, not formal proofs.

What to skip for now. Skip the inside of neural networks. Skip the calculus behind training. You do not need them to build with models, and they can eat weeks. If you get curious later, come back.

Java ideas you already know

• A token is like a token from a lexer. A lexer splits source code into small pieces before parsing. A model splits text into small pieces before reading.
• The context window is like a stateless request body. The model remembers nothing between calls. Everything it needs must be in the request.
• An embedding is like coordinates. Two points that are close on a map are close in meaning.
• Temperature is a randomness knob. At zero you get the most likely answer each time. Higher values add variety.

Tools to use

• Java, with a Spring Boot project.
• Python and Jupyter. Jupyter is a notebook tool where you run code in small steps and see the output right away.
• A hosted model API, or a small model running in Ollama.

For the Java side, Spring AI gives you a ChatClient. It is a small fluent API for talking to a model, with both a synchronous and a streaming mode. See the ChatClient docs. Here is the shape of a call:

@RestController
class AskController {

    private final ChatClient chatClient;

    AskController(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }

    @GetMapping("/ask")
    String ask(@RequestParam String question) {
        return chatClient.prompt()
                .user(question)
                .call()
                .content();
    }
}

This compiles against Spring Boot 4.0.0 and Spring AI 2.0.1 once you add a model starter for your provider. You also need that provider's settings, such as an API key. Check the docs for your versions.

Build this

Call one model from Java and from Python with the same prompt. Compare the answers. Then log the tokens and the cost for each call.

The comparison teaches you something quickly. The same prompt can give different answers on different runs. Logging cost teaches you that every call has a price.

You are done when

• You can explain tokens, the context window and temperature to a colleague without notes.
• Your program calls a model from both Java and Python and prints the token count for each call.
• You can say why two similar sentences get similar embeddings.

Stage 2: LLM apps and RAG (suggested weeks 7 to 14)

Now you build real features. This is the longest stage with the most new ideas, so give it room.

What you will learn

• Prompting that works. A prompt is the text you send to the model. Good prompts are specific, give examples and say what format to return.
• Structured JSON output. You ask the model for JSON and map it to a Java record. Then you validate it, because models sometimes break the format.
• Streaming. The model sends its answer in pieces as it writes. Your app can show text as it arrives instead of waiting.
• Retries, timeouts and cost limits. Model calls are slow and can fail. You set a time limit, retry safely and cap spending.
• RAG basics. Your app searches your own data first, then gives the best passages to the model. Chunking means splitting documents into passages. Embeddings turn those passages into vectors. Vector search finds the passages closest to the question.
• Better retrieval. Hybrid retrieval mixes keyword search with vector search. Reranking is a second step that sorts the results by how well they fit the question. You also learn how to check that retrieval is good.

What to skip for now. Skip fine-tuning, which means training a model further on your own data. Most apps never need it. Skip comparing five vector databases too. One Postgres with pgvector is enough to learn the ideas.

Java ideas you already know

• Structured output is like JSON deserialisation with validation. You map text to a typed object, and you reject bad input.
• Retries, timeouts and limits are the same patterns you use for any remote service. A circuit breaker still applies.
• A vector store is like a database index. It exists to find rows fast, only here the key is meaning.
• A chunk is like a page in a paged query. You cannot send everything, so you send the part that matters.

Tools to use

• Spring Boot.
• Spring AI or LangChain4j. LangChain4j is another Java library, built to simplify using LLMs in Java applications. Pick one and stay with it for the series.
• Postgres with pgvector. pgvector is an open-source extension that adds vector similarity search to Postgres. Spring AI has a pgvector store that works with it.

Build this

Build a document Q&A service in Spring Boot. It answers questions from your own files and shows which passages it used.

The last part matters most. Showing the passages lets a user check the answer. It also lets you check your own retrieval. When an answer is wrong, you can look at the passages and see whether search or the model was at fault.

You are done when

• Your service answers questions from at least a few of your own documents.
• Every answer lists the passages it came from.
• You have a small set of test questions, and you can say how often the right passage comes back.

Stage 3: Agents and MCP (suggested weeks 15 to 22)

Here the model stops only answering and starts doing. This is where design choices start to matter a lot.

What you will learn

• Tool calling. You describe a Java method to the model. The model asks to run it, and your app runs it. Spring AI explains this in its tool calling docs. The model never touches your systems directly. Your code does.
• The plan, act, observe loop. An agent is a program that lets a model decide the next step. It plans, calls a tool, reads the result, and repeats until the job is done.
• MCP. The Model Context Protocol is an open standard for connecting AI applications to tools and data. Servers offer tools. Clients call them. The MCP intro compares it to a USB-C port for AI applications.
• Memory and state. The model forgets between calls, so your app stores what it needs and sends it back.
• Multi-step workflows and routing. Some tasks need several steps. You can send easy tasks to a small model and hard ones to a larger one.
• Human approval. For risky actions, such as sending an email, a person must say yes first.

What to skip for now. Skip teams of many agents that talk to each other. They are hard to debug, and one agent with good tools teaches you more. Skip large agent frameworks until you have built the loop by hand once.

Java ideas you already know

• Tool calling is like exposing a service interface. The model sees the method name, the description and the parameters. That is your API contract.
• The agent loop is like an event loop or a state machine. It repeats until a stop condition is met.
• An MCP server is like a REST service with a standard shape. An MCP client is the matching client, so any AI app can use the same tools.
• Human approval is like a manual review step in a deployment pipeline. Nothing ships until someone signs off.

Tools to use

• Spring AI tool calling and its MCP support, with client and server starters.
• The MCP Java SDK, the Java library that Spring AI builds on.
• LangChain4j, if you picked it in Stage 2.

Build this

Build an agent that reads support tickets through an MCP server and drafts replies. It waits for a human to approve each reply before anything is sent.

Keep the first version small. One tool that lists tickets and one that reads a ticket is enough. Add the approval step before you add anything clever.

You are done when

• Your agent reads tickets through an MCP server, not through code baked into the agent.
• No reply is sent until a person approves it.
• You can explain each step the agent took on a real ticket, from a log.

Stage 4: Production (suggested weeks 23 to 28)

This stage is where you earn the title. Many people can build a demo. Fewer can show that it works, and keep it working.

What you will learn

• Evaluation sets and regression tests. An evaluation set is a list of inputs with the answers you expect. You run it after every change to see if quality went up or down. Spring AI has evaluator tools for this.
• Tracing. A trace shows the path of one request through your system. OpenTelemetry is a vendor-neutral standard for traces, metrics and logs.
• Cost and latency budgets. You decide how much a request may cost and how long it may take. Then you alert when it goes over.
• Guardrails. These are checks that keep your app safe. They cover personal data, prompt injection and limits on what tools can do. Prompt injection is when text in the input tricks the model into ignoring your instructions.
• Audit logs and approvals. You keep a record of what the agent did and who approved it.

What to skip for now. Skip building your own evaluation platform or dashboard. A plain test class, a file of questions and a CI job are enough. Fancy tooling can wait until you know what you need.

Java ideas you already know

• An evaluation set is like a regression test suite. The difference is that results are scored, not just pass or fail.
• Tracing is like distributed tracing across microservices. A model call is one more hop.
• A cost budget is like a rate limit or a quota. You enforce it in code.
• Guardrails are like input validation and least-privilege access. Never trust input, and never give a component more power than it needs.

Tools to use

• Spring Boot.
• Micrometer, the metrics facade you may already use. Spring AI adds metrics and tracing through it.
• OpenTelemetry.
• A CI pipeline that runs your evaluation set.

One caution. Spring AI keeps prompt and completion logging off by default, because they can hold private data. Turn them on only when you have thought about who can read the logs.

Build this

Add evaluation and tracing to your agent. Then publish a short write-up of what you measured and what you fixed.

This is your portfolio piece. It shows the thing most candidates cannot show: evidence that you measured quality and acted on it.

You are done when

• A CI job runs your evaluation set and fails when quality drops.
• You can open a trace for one request and see every model and tool call in it.
• Your write-up lists what you measured, what was wrong and what you changed.

The one project that grows

You may have noticed that the four builds are not four separate projects. They are one project, a support helper, that gets bigger each stage.

In Stage 1 it is a command-line program that calls a model. In Stage 2 it becomes a service that answers questions from documents. In Stage 3 it becomes an agent that handles tickets with human approval. In Stage 4 it gets evals and tracing.

Figure 2: One project that grows. Each stage adds a layer on top of the last one.

There are good reasons to work this way. You never start from a blank page. Each stage builds on code you already understand. And at the end you have one coherent project to show, not four half-finished ones.

It also teaches a real lesson. Real systems grow in layers. When you add the agent, you will find gaps in your retrieval. When you add evals, you will find bugs in your agent. That is normal, and it is how the work goes on a real team.

Mistakes to avoid

Five mistakes show up again and again.

Skipping tests and evaluations. It is tempting to try a few prompts by hand and call it done. Then a small change breaks something and you do not notice. Build the evaluation set early, even if it has only ten questions.

Stuffing everything into the prompt. The context window is large, but it is not free. Long prompts cost more, run slower and can make answers worse. Retrieve what matters and send only that.

Trusting model output without checking it. A model can sound sure and be wrong. Validate structured output. Check facts against your sources. Show users where an answer came from.

Building agents before you can do retrieval. Agents lean on retrieval, tools and good prompts. If those are weak, the agent will be weak too, only harder to debug. Do the stages in order.

Chasing every new framework. New libraries appear every week. Pick one stack, learn it well and finish your project. The ideas in this series carry over from one tool to another.

Warning

An agent that can send email, change records or spend money needs limits and a human check. Add them before you add more features, not after something goes wrong.

How to keep going

A good rhythm is 6 to 8 hours a week. That is a suggestion. Adjust it to your life. Some weeks you will have two hours, and that still counts.

Use a simple weekly loop. Learn something. Build something small with it. Then write down what you learned, even if it is only a few lines in a notes file.

The writing step feels optional. It is not. Writing shows you what you did not understand. It also gives you material for your portfolio later.

If you fall behind, do not start over. Pick up where you stopped. Skip the nice-to-have items and finish the "You are done when" lines for your stage. Those lines are the real goal. The week numbers are not.

Your portfolio and interviews

Your last stage produces the portfolio piece. Put the project in a public repo, with these parts:

• A README that says what the project does, how to run it and what it cannot do.
• An architecture sketch. One diagram is enough.
• Evaluation results, with the questions, the scores and what changed over time.
• A short write-up of what you measured and fixed.

Interviewers at this level tend to ask about the same few things. Expect questions like these:

• How does a model turn text into an answer, in simple terms?
• How would you build a Q&A feature over private documents?
• How do you know your retrieval is any good?
• What can go wrong when a model can call tools, and how do you limit it?
• How do you test and monitor something that gives different answers each time?

You do not need perfect answers. You need honest ones that point to something you built. "Here is what I measured and what I changed" is a strong answer.

What comes next

This series has ten parts. Parts 1 and 2 are published. The rest are planned, and the order may change.

1. Java Developer to AI Engineer: The Complete Roadmap (published)
2. How LLMs work, for Java developers (Stage 1, published)
3. Call your first LLM from Spring Boot (Stage 1 to 2)
4. Structured output and streaming (Stage 2)
5. RAG with Spring AI and pgvector (Stage 2)
6. Agents: the plan, act, observe loop (Stage 3)
7. MCP servers and clients in Java (Stage 3)
8. Evaluating and tracing LLM apps (Stage 4)
9. Guardrails, approvals and cost control (Stage 4)
10. Your portfolio and interviews (finish)

To follow along, subscribe to the blog feed in your RSS reader. You can also find us as @agentworkshq on X and Instagram.

Next, you will learn how LLMs work, in terms a Java developer will find familiar.