A modern data architecture diagram can contain 40 boxes, 20 arrows and a dozen recognizable technology names and still fail to explain how the system actually works.
The same problem repeats in architecture conversations.
Someone says:
Oracle → Kafka → Spark → ADLS → Databricks → Power BI
Technically, that may be correct.
But an executive looking at the diagram still needs to understand something much simpler:
What is each component actually doing with the data?
And surprisingly, technical teams sometimes struggle to answer that cleanly too.
The most useful mental model is not a product architecture.
It is this:
Business Event → Record → Move → Process → Store → Query → Consume
Each component along the way does a different job: application, operational database, movement, processing, storage, tables, query and consumption.
Once those responsibilities are clear, most modern data architectures become considerably easier to understand.
Start With the Business Event
Consider something familiar.
A customer transfers $500 from checking to savings.
Before thinking about Kafka, Spark, cloud storage or dashboards, ask:
Where did the business event happen?
Usually:
Customer → Banking Application → Operational Database
The application executes the business action.
The operational database records the transaction.
That database is designed primarily to support the operational application not necessarily enterprise analytics.
But eventually other people and systems want the data.
Risk may need it.
Finance may need it.
Controls teams may need it.
A dashboard may need it.
An AI model may need it.
That is when the second journey starts.
1. Move the Data
The transaction now needs to leave the operational environment.
This might happen through:
- a file,
- an API,
- an ingestion pipeline,
- change data capture,
- or an event streaming platform.
The technology is less important than the architectural responsibility:
Movement gets data from one place to another.
One of the easiest mistakes is confusing movement with processing.
If I copy a file from an operational system into cloud storage, I have moved the data.
I have not necessarily changed it.
That distinction sounds basic. It matters enormously once teams start discussing ownership, failures, reconciliation and controls.
AWS similarly separates ingestion, processing, storage and consumption as distinct logical responsibilities in data lake architectures.
2. Process the Data
Now imagine the transaction data contains:
- duplicate records,
- inconsistent date formats,
- missing customer identifiers,
- invalid transaction codes.
Something has to work on that data.
That is processing.
For example:
Read → Validate → Clean → Standardize → Join → Calculate → Write
Technologies such as Apache Spark can perform distributed data processing across large datasets.
The executive question is not initially:
Are we using Spark?
It is:
What is being done to the data, why is it necessary, and who owns the result?
That question exposes far more about the architecture.
3. Store the Data
Here is where terminology frequently creates confusion.
A data lake does not automatically “process” data simply because data is sitting inside it.
Storage holds data. Compute works on data.
In practice, a file can remain in storage without any processing occurring.
Think about a warehouse.
The warehouse holds inventory.
The workers and machinery inside the operation perform work on that inventory.
Cloud data architecture follows a similar principle.
Azure Data Lake Storage, for example, provides scalable storage capabilities designed for analytics workloads. Processing frameworks can access that stored data, but the storage service and the processing engine remain different architectural responsibilities.
This distinction becomes important when discussing:
cost, performance, scaling, security and failure ownership.
If processing is slow, adding storage may not solve the problem.
If storage is poorly organized, adding compute may not solve the problem either.
4. Do Not Confuse the File With the Engine
Another surprisingly common statement is:
“Parquet processes the data.”
It does not.
Parquet is a file format.
Apache describes Parquet as an open, column oriented format designed for efficient storage and retrieval.
A simple distinction is:
Spark → processing
Parquet → format
ADLS/S3 → storage
Three related concepts.
Three different responsibilities.
Executives do not need to know the internal implementation of Parquet to participate intelligently in an architecture discussion.
But they should be able to ask:
What performs the processing?
In what format are we storing the output?
Where is that output physically retained?
Those three questions can expose architectural confusion very quickly.
5. Query the Data
Eventually someone wants an answer.
Perhaps:
How many transactions failed yesterday?
The files sitting in storage cannot answer the question by themselves.
A query engine must read the relevant data and execute the request.
Another small terminology issue matters here:
SQL is a language.
Something must execute that SQL.
Depending on the architecture, that might be a database, warehouse, lakehouse SQL engine or distributed query engine.
So when an architecture diagram simply contains a box labeled SQL, a useful question is:
What actually executes the SQL?
That is a small question with surprisingly large architectural implications.
6. Consume the Result
Finally, the information reaches something useful:
Dashboard
Analyst
Business Application
Control
Regulatory Report
AI/ML Model
This is consumption.
At this point the architecture has converted an operational event into something another business or technology process can use.
The full mental model becomes:
Business Event
↓
Operational System
↓
Data Movement
↓
Processing / Compute
↓
Analytical Storage
↓
Query Engine
↓
Business Consumption
That is the architecture I want executives to understand before discussing products.
Where Bronze, Silver and Gold Fit
This is also where the popular Bronze → Silver → Gold model becomes easier to understand.
It is useful but it should not replace the more fundamental architecture model.
Databricks describes the medallion pattern as logical layers representing progressively improved data quality: Bronze for raw data, Silver for validated/refined data and Gold for enriched or business oriented data.
The important word is logical.
Bronze, Silver and Gold do not mean that the data lake itself suddenly performs processing.
A processing engine reads data, performs transformations and writes the resulting data into the appropriate logical layer.
Think:
Raw Storage → Compute → Refined Storage → Compute → Business Ready Storage
The distinction matters.
Why Executives Should Care
This may sound like terminology.
It is really about accountability.
Imagine a regulatory report contains incorrect information.
Where did the problem occur?
Was the source wrong?
Was data lost during ingestion?
Did transformation logic change?
Was the wrong dataset queried?
Did a dashboard apply additional business logic?
Without clear architectural responsibilities, incidents quickly become organizational debates.
The same issue appears with controls.
Who validates completeness during movement?
Who owns transformation quality?
Who controls access to stored information?
Who monitors processing failures?
Who certifies the data consumed by the business?
Architecture becomes operational when those questions have named answers.
The Question I Would Ask in an Architecture Review
If someone presents a complicated data architecture, I would resist starting with:
What technology is this box?
I would start with:
What business problem are we solving?
Then:
Where does the data originate?
How does it move?
What changes it?
Where is it stored?
What queries it?
Who consumes it?
Then I would move into the questions that matter at enterprise scale:
What happens when something fails?
How do we know the data is complete and accurate?
Who has access?
Who owns each stage?
That progression moves the conversation from reading an architecture diagram to evaluating an architecture.
And that is where technology leadership begins.
Executive Takeaway
Modern data architecture becomes unnecessarily difficult when we start with product names.
Start with responsibilities.
Move. Process. Store. Query. Consume.
Then place Kafka, Spark, ADLS, S3, Databricks, Trino, Snowflake, Power BI or whatever technology the organization uses into those responsibilities.
The technology will continue to change.
The architectural questions change much more slowly.
For executives, that is the useful abstraction.
For engineers beginning their careers, it is also one of the simplest ways to stop memorizing technology diagrams and start understanding how data systems actually work.
Related reading on PRODCOB
- Begin with Databricks: Lakehouse Fundamentals Explained
- I Tried Data Loading Patterns in Microsoft Fabric Using Kaggle Data. Here’s What Broke First.
- Unity Catalog: Overrated or Misunderstood?
- Apache Parquet Explained: Columnar Storage, Compression & Analytics
- Databricks Pipelines: Beyond the Notebook
References
- Microsoft Learn: Azure Data Lake Storage overview
- AWS Whitepaper: Logical architecture of modern data lake centric analytics platforms
- Apache Software Foundation: Apache Spark
- Apache Software Foundation: Apache Parquet overview
- Databricks: What is the medallion lakehouse architecture?