New approaches for getting value from data

Title image
14th Jan 2024

With more and more data within organizations, the need on exploit the value of data has become more and more challenging. Organizations have found that there is a gap between data engineers and data analysts.

The reason is that data engineers usually focus on moving data, but usually, they don't know what is the value in there, whilst data analysts focus on getting insights, and they usually are not involved in knowing the whole set of data points within an organization. But knowing how to make sense of the data within an organization is something that matters, so recently a new approach has come up into the scene.

A new role has come to the scene to fill the gap. A data translator. who can understand business needs and make sense of the data to suggest changes backward and push them ahead.

If you don't know, the new role is the Analytics Engineer.

Traditional and modern data teams involves different approaches. Let's explore the differences. 

Traditional data teams

Traditional data teams typically operate in silos with distinct roles such as data analysts, data scientists, and data engineers. Data analysts focus on analysis and reporting, data scientists work on predictive models, and data engineers handle data pipelines. Collaboration between these teams is limited, and data often moves through handovers, making the workflow less integrated. Specialized tools are commonly used by each team, contributing to a fragmented tool landscape. This structure can result in slower responses to changing business needs and a less efficient overall workflow.

Modern data teams

On the other hand, modern data teams with analytics engineers have a more integrated and collaborative approach. Analytics engineers play a crucial role in bridging the gap between data engineering and analytics, promoting cross-functional collaboration. They often take on end-to-end ownership of projects, reducing handovers and streamlining the workflow. The focus is on unified tooling, selecting platforms that serve multiple purposes and increasing adaptability to changes. Automation and a commitment to efficiency make these teams more responsive to evolving business requirements, fostering a more productive and flexible data analytics environment.

CapabilityData EngineersAnalytics EngineersData Analysts
Data IngestionDesign and build robust data pipelinesContribute to data pipeline developmentLeverage existing pipelines for analysis
Data ModelingDevelop and maintain data modelsCollaborate on data model designUtilize data models for analysis
ETL ProcessesImplement Extract, Transform, Load processesContribute to ETL process optimizationUse ETL outputs for analysis
Data ArchitectureDesign and maintain data architectureContribute to data architecture decisionsUnderstand and work within existing architecture
Tool SelectionSelect and implement data engineering toolsContribute to tool selection for analyticsUse tools for querying, visualization, and reporting
Database ManagementManage and optimize databasesContribute to database managementUtilize databases for querying and reporting
End-to-End OwnershipHandle the full data lifecycleTake ownership of analytics projectsFocus on specific analysis tasks
AutomationImplement automation for data processesAutomate analytics workflowsRely on automated reporting tools
Business UnderstandingUnderstand business requirementsTranslate business needs into analytics solutionsAnalyze data to provide business insights

The previous table provides a summary of the main capabilities for data engineers, analytic engineers and data analysts, but it's important to note that these roles can vary across organizations, and individuals may possess a mix of these skills. Additionally, the boundaries between roles may blur in modern, collaborative data teams.

Do you see a gap between data engineers and data analysts?

About me
Adrian Rodriguez

Spaniard working in the data field since 2013 in different industries as BI developer, Data Analyst as well as Data Scientist. I consider myself a data analytics passionate.