Choosing a company to build your AI-based application is a bit like picking a contractor for a complicated installation in a building. The impressive drawing on paper is not what counts. What counts is whether the whole thing works, can be maintained and does not fall apart under the first serious load. We are the Web Systems team, a software house from Łódź operating since 2006, and we have seen dozens of projects in which AI was supposed to be a magic button. And turned out to be a complex engineering undertaking. In this guide we show what to actually look at before you sign a contract, and how to tell a team selling promises from one that will simply deliver a working solution.
Spis treści
Why you need an AI vendor and what you are really looking for
The first misconception we run into with clients? The belief that rolling out artificial intelligence comes down to plugging in a ready-made language model. In reality it is an engineering project in which the model itself is often the least troublesome part. The difficulty lies elsewhere: in integration with existing systems, in preparing the data, in handling edge cases and in maintaining the whole thing once it goes live. A model without the context of your company generates answers that sound nice but are useless. Only combining it with your data, processes and business rules creates value.
A client looking for an AI application vendor is essentially looking for a partner who will deliver something that works. Not another marketing deck with a flashy demo. A demo can be thrown together in a day, because it runs on hand-picked data and pre-planned questions. Production is a different league. That is where real users appear, along with unusual queries, faulty input data and a requirement for repeatable quality. And it is exactly that difference between two worlds that you have to sense in a conversation with a prospective vendor.
As a software house from Łódź we look at AI from the perspective of a builder who knows the real constraints of a project. We know what it costs to maintain an integration when an external API suddenly changes its response format. We know how a system behaves when the knowledge base swells from a thousand to a million documents. That knowledge does not come from theory. It comes from years of building web applications, B2B systems, integrations and automations. We added AI to that set of skills as one more tool, not as a fashionable add-on detached from engineering.
The most important difference you should pick up on has to do with language. A company selling hype talks about revolution, transformation and unlimited possibilities. A team that understands the subject talks about data, architecture, token costs, latency and ways of measuring quality. The latter will ask you uncomfortable questions before promising anything: where does the data come from, who owns it, which systems have to be integrated, what answer accuracy is acceptable to you. No such questions? That is the first warning sign.
It is also worth answering honestly what you actually need. Sometimes the answer is an assistant based on a language model with search over a knowledge base. Other times a simple automation or a classic algorithm will do the job more cheaply and more reliably than a generative model. A good vendor does not force AI into places where a plain business rule works fine. If a team proposes artificial intelligence as the answer to every problem, treat that as a sign that it is selling a product rather than solving your trouble.
Before you go any further, set out your own list of expectations. Define which process you want to improve, what savings in time or money you assume and how you will know the rollout succeeded. Without that foundation every conversation with a vendor turns into a discussion about technology rather than about business outcome. The more concretely you describe the goal, the easier it is to filter out the companies that can talk nicely about AI from the ones that can actually implement it.
Tip: prepare a one-page description of the problem before you start talking to vendors. Include the business goal, the available data, the systems to integrate and the success criterion. That document will immediately show you which people are genuinely listening to your needs and which ones recite their own offer regardless of what you say.
Technical skills you have to verify before signing a contract
The most common mistake when assessing a vendor? Looking only at impressive AI demos. Whereas the success of the project is decided by experience in areas that are not obviously associated with a language model: API integrations, B2B systems, e-commerce, data flows between applications. An AI application almost never exists in a vacuum. It has to pull data from your ERP, save results in a CRM, react to events in an online store or expose answers through a secure API. A team that has never done that will learn on your budget.
The second pillar is knowledge of application architecture. A solid vendor thinks in layers: separating the interface from the business logic, and the logic from the data layer. It applies the single source of truth principle, designating one place that owns a given piece of information and is the only one allowed to change it. It maintains a unidirectional data flow, which makes the system predictable and easier to debug. These are not academic ornaments. They are the foundation that decides whether, after a year of development, the application can still be maintained at all.
Application architecture documentation puts it concisely, and the principle carries over directly into the world of AI:
The single source of truth principle is often used with the unidirectional data flow (UDF) pattern. In UDF, state flows in only one direction, typically from parent component to child component. The events that modify the data flow in the opposite direction. This pattern better maintains data consistency, is less prone to errors, is easier to debug.
The third skill, simply not to be skipped in knowledge-based applications, is the ability to build RAG systems, that is retrieval-augmented generation. In that approach the model does not answer from its own general knowledge. It answers on the basis of information found in your curated knowledge base. The quality of the answer is then decided not by the model itself, but by the quality of the semantic search engine. Because if the retrieval mechanism hands the model irrelevant fragments, the answer will be grammatically coherent yet factually a miss. A vendor who understands RAG will devote at least as much attention to that part as to the model itself.
The way a candidate vendor talks about these matters says more than an entire portfolio. Ask concrete questions and listen to whether the answers are precise or hidden behind generalities. Below is a set of questions that genuinely test a team:
- Which technology stack do you propose and why this one? A good team will justify the choice with specific features of the project rather than with fashion.
- What does your code review process look like? No code review means quality depends on the mood of a single developer.
- How do you make the solution testable? An application that cannot be tested in isolation will produce regressions with every change.
- How do you approach technical debt? Awareness of debt and a plan to pay it off show engineering maturity.
- How do you measure the quality of answers generated by the model? If the answer is “by eye”, the rollout will be unpredictable.
- How do you isolate business logic from the model and the interface? Good separation of layers lets you swap the model without rewriting the whole application.
Pay attention to how the team talks about testing. AI applications are non-deterministic by nature. The same prompt can yield different answers. A mature vendor has ways of dealing with that: testing the retrieval layer separately, using control question sets with expected answers, monitoring quality in production. Architecture documentation stresses that designing for testability has to be planned from the start, not glued on at the end.
Consider how to make each part of your app testable in isolation. A well-defined API for fetching data from the network facilitates testing the module that persists that data in a local database. If instead, you mix the logic from these two functions in one place, or distribute your networking code across your entire codebase, testing becomes much more difficult, if not impossible.
Finally, check whether the team can say “we do not know that” or “that needs to be verified”. Certainty about every answer, especially about costs and deadlines, is in AI projects a sign of inexperience rather than competence. A real vendor knows the areas of uncertainty and can propose how to reduce them. For example through a short discovery stage before quoting the whole thing.
Tip: ask to speak not only with a salesperson, but with a technical person who will actually work on your project. Fifteen minutes with a developer will tell you more about the team’s skills than ten pages of a proposal. And if a company guards access to its engineers, think about who is really going to build your application.
Data, security and quality: where AI projects most often fall apart
If we had to name one place where AI projects most often come undone, it would be the data. The quality of an artificial intelligence rollout depends not on the chosen model, but on the quality and organization of the source data. You can plug in the best model on the market and still get weak answers if the data is inconsistent, incomplete or badly processed. The data holds both the greatest risk and the greatest leverage on value.
In RAG systems, three stages of data processing are crucial. The first is parsing: correctly extracting content from PDF files, documents, web pages or databases while preserving structure. The second is chunking, that is splitting the content into fragments of the right size so that the search engine can match them accurately to a question. The third is building a semantic search engine that will pick the genuinely relevant fragments out of hundreds. An error at any of these stages ruins the whole thing, because the model will receive either too little context or context that is off topic.
The importance of the retrieval mechanism is hard to overstate. Industry material on RAG architecture says it outright:
The retrieval mechanism in RAG is critically important. You need the best semantic search on top of a curated knowledge base to ensure that the retrieved information is relevant to the input query or context. If your retrieved information is irrelevant, your generation could be grounded but off-topic or incorrect.
That sentence should hang above the desk of everyone who implements knowledge-based applications. An answer that is “grounded but off topic” is more dangerous than an obvious error, because it sounds credible and users trust it easily. So ask the vendor not which model they will use, but how they will build and test the retrieval layer on your data.
Another area that separates professionals from amateurs is measuring the quality of generated content. The mature approach, sometimes called RAG Ops, treats quality as a metric you observe and optimize rather than as an impression. What gets measured includes groundedness, that is the degree to which an answer rests on the supplied data, plus coherence and relevance. Industry studies describe this way of working unambiguously:
A RAG Ops, metrics driven approach like this will help you hill climb to high quality RAG and grounded generation. Implementing these evaluations gives you a baseline measurement and you can optimize for RAG quality by configuring your search engine, curating your source data, improving source layout parsing or chunking strategies.
Without that approach you do not know whether a change to the system improves quality or ruins it. Every modification becomes a shot in the dark. A vendor who can show how they will measure quality and how they will raise it through better parsing, chunking or data curation gives you something priceless: predictable development.
The third dimension is security and the risks specific to AI. The most important of them are:
- Model hallucinations – generating information that is not in the sources. It is limited through good retrieval, clear instructions for the model and enforcing answers based exclusively on the supplied context.
- Leakage of sensitive data – the risk that data reaches an external model provider or is revealed in an answer to an unauthorized user. It requires access control, anonymization and a conscious choice of where the data is processed.
- No control over the source of truth – a situation in which nobody knows where a specific piece of information in an answer comes from and who is responsible for it. Without a designated data owner, the system eventually becomes impossible to audit.
Each of these risks can be limited by design, but only if you name it at the start. Ask the vendor directly how they intend to protect your data, where it will be processed and how they will ensure the model does not invent an answer when data is missing. The answer will show whether they are thinking about production or only about a demo.
Remember too that data is not a one-off task. The knowledge base ages, documents get updated, new products and procedures appear. A good vendor will plan how the data is refreshed and who will be responsible for it on your side of the organization. Without that, even a beautifully built system will start giving outdated answers after six months, and users will lose trust in it.
Tip: before you start the rollout, run a short audit of your data. Check what format it is in, how it is described and whether an up-to-date, reliable source of knowledge exists at all. It often turns out that the first stage of an AI project is not the model, but tidying up the documentation. And that is a good sign that the vendor understands the subject.
Architecture, scalability and maintenance: what happens after go-live
Launching an AI application is not the finish line. It is the start. From that moment the life of the system begins: users ask questions, data grows, provider models change and business requirements evolve. How smoothly will that period go? To a huge extent it depends on the architectural decisions made right at the beginning. Good architecture is the foundation of a scalable and maintainable application. One that is easier to test and easier to onboard new people into.
Engineering documentation leaves no doubt about the role of architecture:
App architecture is the foundation of a high-quality application. A well-defined architecture lets you create a scalable, maintainable app that can adapt. Having a good architecture implemented in your app improves the maintainability, quality, and robustness of the overall app and lets the app scale.
One of the most valuable principles says not to store state in ephemeral components. Meaning ones the system can destroy and recreate at any moment. In the world of applications that means data is not kept in interface elements. Translated into AI it sounds like this: business logic, data and the choice of model have to be separated from the presentation layer and from the model itself. That way you can swap the model for a newer, cheaper or better one without rewriting the whole application. You can also change the interface without touching the logic.
That separation has measurable financial consequences. The language model market changes every few months: new versions appear, prices fall, capabilities grow. If your application is rigidly entangled with one specific model, every such change means a costly refactor. And if the logic is separated, swapping the model is a minor operation. Asking how the vendor plans to isolate the model from the rest of the system is one of the most important questions in the entire conversation.
Maintaining an AI application also involves costs that are easy to forget in the excitement of the rollout. They include, among others:
- Model call costs – every request to the model costs money, and at high traffic the bill can be a surprise. Good architecture limits unnecessary calls and applies caching wherever possible.
- Quality and operations monitoring – you need visibility into how the system answers, where it gets things wrong and how fast it responds. Without monitoring, problems only surface through user complaints.
- Model and library updates – providers retire older model versions, libraries need security patches. This is a steady, predictable cost that has to be planned.
- Growing the knowledge base – adding new documents, correcting wrong answers, tuning the retrieval. This is continuous work, not a one-off.
Scalability, in turn, means the system will handle both ten and ten thousand queries a day. A layered architecture, clear boundaries of responsibility between modules and limited dependencies between them make it possible to spread the load and optimize selectively. An application in which everything is entangled with everything else starts falling apart unpredictably as traffic grows. And diagnosing the problem eats up entire days.
Team onboarding is an often underestimated aspect of maintenance. If the system is well designed and consistent, a new person will grasp it quickly and start working effectively. If the architecture is chaotic, every change requires archaeology in the code, and the knowledge stays locked inside a few people’s heads. From the client’s perspective that is a business risk: you become dependent on specific individuals rather than on the vendor as an organization. So ask whether the project will be documented and whether somebody other than the author will be able to develop it.
Predictable system development is exactly what separates a mature partnership from constant firefighting. When the architecture is solid, adding a new feature is a planned task with a reasonable estimate. When the foundation is weak, every change carries the risk of breaking something else, and estimates keep growing, because the vendor has to price in uncertainty. A cheap application built without architecture eventually turns out to be the most expensive investment.
Tip: when talking to a vendor, ask not only about the price of building the MVP, but about the plan for maintenance and development over the next twelve months. Ask for an estimate of model call costs, monitoring and updates. A company that has a ready answer is thinking about your system for the long term. A company that looks only at the rollout will leave you with the problem right after launch.
Common mistakes when choosing a company and how to avoid them
The costliest mistake when choosing an AI application vendor is a decision made on price alone. We understand the temptation: proposals can differ several times over, and the lower figure looks like a saving. In practice, choosing the cheapest vendor usually means paying off technical debt many times over. A cheap team cuts stages short, skips tests, ignores architecture and delivers something that works in a demo. And when it comes to production, development and maintenance, it turns out it would have been cheaper to do it properly straight away.
Technical debt is not a metaphor. It is real money and real time. Engineering documentation warns against it directly:
Don’t create shortcuts that expose internal implementation details. You might gain a bit of time in the short term, but you are then likely to incur technical debt many times over as your codebase evolves.
That sentence describes perfectly the mechanism we observe in clients who come to us with inherited, failed rollouts. An apparent saving at the start turns into a chain of increasingly expensive fixes. At some point it is cheaper to rewrite the system from scratch than to keep patching it. Which means the first investment was lost in full.
The second common mistake is the absence of clear boundaries of responsibility and scope. A project without an explicit data owner and without a precisely defined scope simply blurs: nobody knows what belongs to the vendor, what belongs to the client and what was left out. An AI application suffers particularly badly from that kind of chaos, because it requires close cooperation around data. If you do not establish who supplies and updates the knowledge, who defines acceptable quality and who is responsible for security, the project will get stuck in mutual expectations.
There is a set of red flags that should put you on alert immediately in a conversation with a vendor. Here are the most important ones:
- Invented or unverifiable case studies – descriptions of spectacular rollouts that cannot be verified, with no client name, no measurable outcome and no contact person.
- No questions about integrations – if the vendor does not ask which systems the application has to connect to, it means they are not thinking about production.
- Promises without architecture – declarations of a finished solution in a week, with no conversation about data, layers and maintenance.
- AI as the answer to everything – proposing a generative model even where a simple rule or automation would do.
- No mention of maintenance costs – focus solely on the rollout price, ignoring what happens after launch.
- Reluctance to show the technical team – contact only with a salesperson and avoidance of conversations with engineers.
Each of these flags on its own does not disqualify a vendor, but a cluster of them is a clear signal to keep looking. Pay particular attention to case studies. A real project can be described in concrete terms: which problem was solved, by what methods, what result was achieved and what the difficulties were. A story made up entirely of superlatives, without a single hard moment, is suspicious. Because no real AI project runs without trouble.
The third mistake is ambition out of proportion to the risk. Clients often want to build a huge, complete system covering every process at once. That is expensive, slow and loaded with enormous risk, because many assumptions turn out to be wrong only on contact with reality. It is far more sensible to start with a small, measurable MVP that solves one specific problem. A project like that quickly shows whether AI genuinely helps, what it really costs and how users react.
An MVP also gives you something hard to overvalue: knowledge. After the first stage you know far more about your data, about the model’s behavior and about your own needs than you did at the outset. You then make decisions about further development on the basis of facts rather than assumptions. Risk is spread across stages and you spend the budget deliberately. A vendor who proposes such an approach themselves, instead of pushing you into a big contract straight away, is thinking about your interest. Not only about the size of the invoice.
Tip: ask every vendor under consideration for references you can call. A short conversation with a previous client about how the team handled problems, kept deadlines and communicated difficulties will tell you more than an entire portfolio. And the complete absence of any verifiable references is in itself a serious warning sign.
FAQ: the most common questions about choosing an AI application vendor
Below we answer the questions we hear most often from clients considering an artificial intelligence rollout. We phrase the answers from the vendor’s perspective, concretely and without marketing gloss. Because that is exactly the kind of information we would be looking for in your position.
How much does an AI application rollout cost and what drives the budget?
The honest answer is: it depends. But it is possible to say on what. The budget of an AI application is shaped above all by three factors: the state and volume of the data, the number and complexity of integrations with existing systems, and the required level of quality and security. A simple tool using an off-the-shelf model, with no deep integrations, is a completely different order of magnitude than a RAG system on a large knowledge base integrated with an ERP and a CRM, with access control and quality monitoring.
The second, often overlooked component of the cost is maintenance. On top of the rollout price you have to add the costs of model calls, monitoring, updates and growing the knowledge base. That is why we warn against proposals that quote a single figure for building the system alone. A reliable vendor will split the budget into rollout and maintenance and point out which costs are one-off and which are recurring. The cheapest offer at the start is often the most expensive over a year, because the lack of architecture and tests comes back many times over.
Is it worth starting with an MVP, or building the full system right away?
In the vast majority of cases we advise starting with an MVP, that is a small, measurable rollout solving one specific problem. The reason is practical: AI projects carry plenty of unknowns, and data and model behavior are hard to predict before they meet reality. An MVP lets you verify whether artificial intelligence genuinely helps, learn the real costs and collect user reactions before you invest in a big system.
Building a full, extensive system straight away only makes sense when the domain is well understood, the data is organized and the requirements are stable. Which in practice happens rarely. Even then it is worth splitting the project into stages with measurable milestones. A staged approach spreads the risk, gives you control over the budget and lets you correct course on the basis of facts. A well-designed MVP is not a dead end: with the right architecture it is the foundation on which you can calmly grow the full system.
Summary: how to choose an AI application company sensibly
Choosing a company that builds AI applications comes down to a few criteria worth pulling together into one picture. First, technical skills that go beyond impressive demos: experience in integrations, knowledge of layered architecture, the ability to build semantic search and to test non-deterministic systems. Second, a mature approach to data, because data decides quality, and parsing, chunking and curating knowledge weigh more than the choice of the model itself.
Third, architecture designed for scalability and maintenance, with business logic separated from the model and the interface, so that the system can be developed and models swapped without costly rewrites. Fourth, a clear maintenance plan covering call costs, monitoring and updates, not just the price of building an MVP. And fifth, transparency in the cooperation: concrete answers instead of promises, verifiable references, clear boundaries of responsibility and a data owner on your side.
If you line up proposals against these criteria rather than against price alone, you will quickly see who understands engineering and who is selling hype. The cheapest vendor is rarely the cheapest over a year, and the loudest in marketing is rarely the best at maintenance. Look for a team that asks hard questions about your data and systems before promising anything, and that proposes starting with a small, measurable step instead of a big, risky system.
At Web Systems we have been designing and delivering web and mobile applications, B2B systems, API integrations, automations, e-commerce and solutions based on artificial intelligence since 2006. We approach AI like any other serious engineering project: we start with the data and the business goal, we take care of the architecture, we measure quality and we plan for maintenance. We do not sell a revolution. We deliver working systems that can be developed and maintained.
If you are considering an AI rollout, building an application, integrating systems, automating processes or modernizing an existing solution, let us talk about your MVP. Tell us about your problem and your data, and we will propose a sensible, technical and measurable first step. Get in touch with the Web Systems team and we will help you assess what can realistically be built and where it is best to start.
