Dennis van Remortel
Dennis van Remortel Portfolio Architect Public Cloud
21 August 2026

Setting up AI without regret: six architectural choices for day one

A successful AI pilot is not yet a manageable service. These architectural principles help organisations combine speed with security, control, and scalability from day one.

The temptation of speed

Almost every organisation is currently exploring how AI can be deployed. Pressure comes from above, from the side, and sometimes from outside. And so action follows: a pilot here, a tool there. Before long, a Large Language Model (LLM) is running in an environment that was originally built for something entirely different.

That urgency is understandable, but from the role of Portfolio Architect Public Cloud, the focus extends beyond the model or the first application. The foundation around it matters most: identity, data, network, logging, compliance, costs, and management. That is precisely where it is determined whether an AI solution grows from an interesting experiment into a reliable cloud service.

That is why, for every new AI use case, a few straightforward questions arise from day one. Who owns this service? Are the usage, risks, and costs manageable two years from now? And is it possible to replace a model or vendor without rebuilding the entire solution?

The answers determine whether an AI environment delivers lasting value or gradually becomes an operational, financial, or compliance risk.

AI without ownership quickly becomes unmanageable

An AI implementation is not a project with a fixed delivery date, but a living service. Models are updated or phased out, changes to prompts and data sources influence behaviour, and growing usage increases dependency.

Yet organisations appear that set up a generative AI solution without clear answers to questions such as: who is responsible when the model suddenly responds differently? Who monitors output quality? Who decides which data may be used? And who determines when a model must be replaced?

These are governance questions that must be anchored in the architecture, not resolved only when something goes wrong.

Record not only who “owns AI”, but also who is responsible for the use case, the data, the platform, and the risks. Establish a management framework for monitoring, changes, incidents, and lifecycle before the first production user goes live.

Security is not added afterwards

The second pitfall is more familiar, yet still underestimated too often: security as an afterthought. With AI, this is particularly risky, because the attack surface differs from that of traditional software.

The infrastructure is not the only source of risk. User input, data presented to the model, connections with other systems, and model output are all potential risk vectors. Consider prompt injection, unauthorised access to data sources, exposure of confidential information, and misuse of generated results.

Security-by-design therefore means more than network segmentation and access control in an AI context, important as those fundamentals are. The design must address concrete questions about data, access, processing, model training, logging, and human oversight from the outset.

Which data may the model use? Which users and systems gain access? Where is the data processed? May a vendor use the data for model training? Which model interactions must be recorded? How is sensitive information protected within logging? When is human oversight mandatory? For organisations in government, financial services, and healthcare, these are not nice-to-haves. They are architectural requirements that determine the choice of model, platform, and vendor.

Prevent every pilot from becoming its own island

A significant risk is that every AI use case ends up with its own infrastructure, security, and management process. This may appear fast at first, but ultimately leads to fragmentation, duplication of effort, and dependencies that are difficult to manage.

A reusable cloud foundation prevents teams from having to redesign identity and access management, network access, secrets, logging, policies, and cost management for every new pilot. Teams then experiment faster within pre-established boundaries.

Governance is therefore not a brake on innovation. Sound guardrails enable controlled speed.

The choice of an AI solution extends beyond public or private cloud. Depending on the use case, an organisation may opt for AI-SaaS, a managed model service in the public cloud, a self-managed model, or a hybrid solution. The right choice follows from data classification, regulations, desired control, costs, and the operational capability of the organisation, not from a principled preference for one technology model.

The six principles that always apply

Based on practical experience, these are the six principles that make the greatest difference.

1. Start with the use case, the risk, and the data

Determine the purpose of the AI solution, which decisions it supports, and what risk the organisation accepts before any technical configuration takes place.

Then classify the data in use and record which sources the model may consult. Enforce those boundaries technically with appropriate access rights, policies, and logical separation. Full isolation per use case is not always necessary or desirable, but controlled separation is.

The classification of the use case and the data must drive the architecture, not the other way around.

2. Record ownership and lifecycle

Explicitly record who owns the use case, the data, and the platform, and who monitors the quality of results and accepts risks.

Also record how models and prompts are evaluated, modified, replaced, and ultimately phased out. Models change, prices rise, and services may disappear. Without lifecycle management, that dependency only becomes visible when change is unavoidable.

3. Build for production and management

A pilot is not yet an architecture. Design with production and management in mind from the start.

Where possible, use a standardised cloud foundation with provisions for identity, network, secrets, logging, policies, and cost management. Ask whether the way the system works and the choices made are explainable to an auditor, regulator, or new administrator eighteen months from now.

Manageability also concerns consumption. Set up budgets, quotas, cost allocation, and monitoring of token and infrastructure usage. A technically successful AI service that is financially unpredictable is not yet a mature service.

4. Embed security and privacy in the design

Use security not as a checklist applied after the fact, but as a prerequisite that steers design decisions. This applies to the choice of model and vendor, but also to the location of data processing, network access, and connections to business data.

Make model interactions auditable through targeted logging. Protect sensitive content, apply appropriate retention periods, and restrict access to audit data. Storing all prompts and responses without restriction can itself create a new security and privacy risk.

Ensure that sensitive information is used only within pre-approved trust boundaries, regions, and processing agreements.

5. Keep humans in the loop and monitor behaviour

AI assistance is not the same as AI autonomy. Determine per use case where human validation is mandatory, and establish this within the system — not solely in a policy document.

Beyond availability and technical performance, monitoring covers output quality, deviating model behaviour, unintended use, security incidents, and costs. Only then is timely intervention possible when the system or its usage changes.

The level of human oversight must match the potential impact. An AI assistant that summarises internal texts calls for different measures than an application that supports employees in decisions affecting citizens, patients, or clients.

6. Design for change and invest in AI literacy

Prevent prompts, data integrations, and business logic from becoming unnecessarily locked to a single model or vendor. Full independence is not always achievable or advisable, but dependencies should be chosen deliberately.

Establish in advance what happens when a model is phased out, prices rise sharply, terms change, or a vendor no longer meets requirements. An exit strategy does not need to mean migration is possible tomorrow, it means knowing what that migration requires.

Technology and architecture alone are not sufficient. People must understand what an AI system does and does not do, which risks correspond to their role, and when human judgement is necessary.

Since 2 February 2025, Article 4 of the European AI Act requires providers and organisations deploying AI to ensure an appropriate level of AI literacy among the people working with AI systems on their behalf. The required level depends on, among other factors, their role, knowledge, and the context in which the system is used.

AI literacy is therefore not a standalone training added as an afterthought, but a precondition for responsible use. Without knowledgeable users and administrators, even the most carefully designed environment remains vulnerable.

Speed and soundness are not mutually exclusive

This is not an argument for delay. Organisations need not wait until everything is perfect. That moment never arrives.

Pressing ahead without a foundation is equally unwise: organisations that start without one build problems that are more costly and harder to resolve later, such as fragmented environments, unclear ownership, unpredictable costs, and dependency on a single model or vendor.

The organisations that succeed treat AI not as a collection of isolated pilots, but as a new category of cloud services within their portfolio. They combine ambition with discipline and make deliberate choices about ownership, data, models, vendors, and lifecycle.

This keeps the environment scalable, manageable, and demonstrably secure, even as usage grows, a model changes, or a regulator asks questions.

That foundation starts on day one.

Want to know more?

Curious what such a foundation looks like in your organisation? Solvinity supports organisations in designing and managing secure AI environments in the cloud, from use-case and risk analysis to a standardised cloud foundation, governance, observability, and operational management.

My colleagues and I would be glad to discuss this further. Get in touch via our contact form.

Want to know what Solvinity can do for you?

Contact us and find out today.

Also read

More articles

Meer