Journal · Hiring

Hiring AI developers: what to look for before you hire

ArmTri Team · Oct 16, 2026 · 4 min read

What separates AI developers who ship dependable features from those who build impressive demos, and how to tell them apart before you hire.

Before you hire AI developers, define the problem

Many companies decide to hire AI developers before they decide what the AI should do. That order is expensive. The word AI covers very different work: adding a chat assistant to a support page, extracting data from documents, ranking products, forecasting demand or training a model on your own data. Each needs a different kind of person.

Write one sentence that describes the outcome for the user, for example 'support agents get a suggested reply drafted from our help center and past tickets'. That sentence tells you whether you need someone who builds on existing models or someone who trains new ones.

AI engineer or machine learning engineer?

An AI engineer usually builds products on top of existing models through APIs. Their strengths are prompt design, retrieval from your own data, evaluation, cost control and integration into a real application. Most companies building LLM apps today need this profile first.

A machine learning engineer works deeper in the stack: preparing training data, training and fine-tuning models, and running them in production. You need this profile when off-the-shelf models cannot do the job, when data must stay on your own infrastructure, or when your product is the model itself.

Hiring the second when you need the first often leads to over-engineered solutions. Hiring the first when you need the second leads to a demo that never becomes reliable.

What good looks like in LLM apps

A demo of an LLM feature can be built in an afternoon. A dependable feature takes careful work, and that work is what you are hiring for. Look for people who talk about evaluation without being asked: how they test whether answers are correct, how they catch regressions when a prompt or model changes, and how they track quality over time.

Good AI developers also think about failure. What happens when the model is unsure? When the source data is missing? When a user asks something out of scope? Ask them to walk you through how they would handle these cases in your product.

Finally, they think about cost and speed. Every model call has a price and a delay. A strong candidate can explain the trade-offs between a larger model, a smaller one and caching, in plain language.

Questions that separate builders from demo makers

Ask for a project they took to production, not a prototype. Then ask: how did you know it worked? What broke after launch? What would you build differently now?

Ask how they handle private data. Where does it go, what gets logged, and how would they keep customer information out of places it should not be? In regulated industries this question alone narrows the field.

Ask them to explain retrieval, embeddings or fine-tuning as if you were a new team member. Clear explanations are a strong sign of real understanding, and you will rely on that clarity in every planning meeting.

The rest of the team still matters

An AI feature is still a software feature. It needs a backend that stores conversations and results, a frontend that shows uncertainty honestly, and someone who decides what the product should do. A brilliant AI engineer working alone often ends up building the surrounding product too, slowly.

This is where matching matters. At ArmTri we assemble AI engineers together with the backend, frontend and product people around them, chosen for skill, personality and knowledge of your industry, so the AI work has a product to live in from the start.

The short version

Before you hire AI developers, write down the user outcome. Choose an AI engineer for products built on existing models and a machine learning engineer when you need custom models. Hire for evaluation, failure handling and cost awareness, and ask about production experience rather than demos. Then make sure the rest of the team is in place to turn the feature into a product.

If you want to see which mix of AI and product people fits your idea, describe it and we will assemble the team.