Software Architecture
Software architecture is the set of structural decisions that make a system easier or harder to change, operate, secure, and scale. It is not the diagram itself; it is the set of boundaries and trade-offs the diagram records. In ML and AI systems, architecture must include data contracts, model versions, evaluation gates, production integration, observability, privacy, and rollback.
Containers, contracts, and ownership
The C4-style view is a useful contract: system context, containers, components, and code. For a document-answering product, the container view might be:
flowchart TD Browser[Browser] --> Backend[Web backend: auth, validation, streaming response] Backend --> Retrieval[Retrieval service: permission-filtered candidate passages] Retrieval --> Model[Model service: prompt assembly, generation, validation] Model --> Review[Review queue: human correction and audit trail] Ingestion[Ingestion worker: OCR, chunking, index writes] --> Retrieval Backend --> Metadata[Metadata store: document, model, prompt, and trace versions] Ingestion --> Metadata Model --> Metadata Review --> Metadata
This artifact is not executable, but it is concrete: each arrow implies an API design contract, an owner, and a failure mode. A technical decision record should capture why retrieval is a separate service instead of a module inside the backend, especially if it creates a microservices boundary.
Intuition
Architecture works by making expensive decisions explicit while they can still be discussed. If latency is dominated by model calls, web backends need streaming and cancellation. If privacy is the primary constraint, retrieval and model-serving boundaries must enforce permission checks before context construction. If operational risk is high, the architecture must reserve a rollback path before launch.
Failure modes
Architecture fails when diagrams omit runtime behavior, data ownership, or deployment order. A box labeled “AI service” is not enough: name the schema, timeout, retry rule, observability fields, and degradation path. Do not introduce design patterns or service boundaries unless they reduce a real coupling problem.
References
Nav