Get your essentials straight.
A modern data platform is essential when it comes to building data-driven solutions in an effective manner. A modern data platform can act as a mesh that ingests or bring together data for analytical purposes.
The building blocks for these cloud platforms generally start at Infrastructure-As-Code (IaC). IaC helps us to provide a highly transparent and maintainable cloud infrastructure to our clients.
The ingestion layer of a data platform needs to be flexible in order to ingest current on-prem data sources, third-party data sources and data from future sources the company will use. Import here is not only the manner in which data is ingested (e.g. streaming or batch), but also the maintenance and skills level needed to support it.
Data storage is a topic that has strongly evolved over the last years and will keep evolving for a long time to come. Discussions around ETL vs. ELT and structured vs. unstructured are important to have. Future-proof thinking is also here highly important, needs will evolve going forward.
Your business intelligence, data science or machine learning team needs a development environment, close to relevant data, that allows them to experiment with and build data driven solutions.
Data driven solutions, be it a report, a machine learning model or a visual UI need a pathway to deployment for it to be useful to business users or systems.
These platform design projects typically involve a lot of stakeholders. Each of them with different requirements for this platform. Given this context, we prefer to start lightweight. Let's have an initial design discussion in the form of a brainstorm to see what you need, explain you how we can support the building of a modern data plaform and develop a roadmap for your context from there.