{"id":2248,"date":"2020-01-20T21:44:20","date_gmt":"2020-01-20T08:44:20","guid":{"rendered":"http:\/\/www.helenanderson.co.nz\/?p=2248"},"modified":"2020-05-15T12:44:21","modified_gmt":"2020-05-15T00:44:21","slug":"designing-data-team","status":"publish","type":"post","link":"https:\/\/helenanderson.co.nz\/designing-data-team\/","title":{"rendered":"Designing the data team"},"content":{"rendered":"\n

So you want to start a Data Team. <\/p>\n\n\n\n

Maybe you need reporting for Sales and Marketing. Perhaps you want to see if there are insights into the application data you\u2019ve been collecting. Maybe you want to learn more about Data Science<\/a> and if that will help keep investors happy.<\/p>\n\n\n\n

Whatever your reasoning there is some planning and some tough questions to ask yourself before you engage a recruiter<\/a>.<\/p>\n\n\n\n


\n\n\n\n

Understanding<\/a>
Collection<\/a>
Cleansing<\/a>
Structure<\/a>
Exploration<\/a>
Empower<\/a>
Prediction<\/a><\/strong><\/p><\/blockquote>\n\n\n\n


\n\n\n\n

Understanding<\/h2>\n\n\n\n

The first hire you should make is not a Data Scientist or Machine Learning Engineer. It’s time to take a long hard look at your business. The purpose of this step is to understand where the gaps are, how decisions are made and what it is you really need <\/strong>from a Data Team.<\/p>\n\n\n\n

While the idea of having a PhD level Data Scientist onboard has been romanticised<\/a>, there must be a reason for them to start digging through Data for those elusive insights. And when they have worked through a dataset, what happens next. <\/p>\n\n\n\n

Appoint a Project Manager<\/a> or Team Lead to sit down with the leaders of your Sales, Marketing, Finance and Product functions to identify where the gaps in knowledge or reporting are<\/strong>, and how the business fits together. <\/p>\n\n\n\n


\n\n\n\n

Collection<\/h2>\n\n\n\n

The next step may be a hire, or in a pinch may be something that can be done with your current team. Collecting the Data identified from the sources identified in step one. You need something for your Data Team to analyse and if there is nothing to look at, they will be stuck.<\/p>\n\n\n\n

The Data I\u2019m referring to here is from systems which already exist. Log files, Production databases if you are building software, CRM systems, Billing platforms, Financial systems, Marketing Automation tools. All of these systems tell a story about your organisation and customers. <\/p>\n\n\n\n

“If you are not aware of where and how the Data is stored, no Analysis can take place.”<\/p><\/blockquote>\n\n\n\n

A Data Engineer, BI Developer, or BI Engineer can help get things moving. This role is focused on the \u2018plumbing\u2019 of the Data world and sets up pipelines to collect Data in a constant stream or in batches throughout the day. However, their first job will be to work with your Project Manager or Team Lead to determine what is available and how easy it is to get at.<\/p>\n\n\n\n

Data Teams can struggle to produce reports, insights and models because they simply don\u2019t have the Data they need. If you have the luxury of starting a team from scratch make this your priority. <\/p>\n\n\n\n


\n\n\n\n
Source: 
Monica Rogati<\/a><\/em> <\/figcaption><\/figure><\/div>\n\n\n\n
\n\n\n\n

Cleansing<\/h2>\n\n\n\n

Data may not be structured in a way that\u2019s right for modelling<\/a> or so granular it needs work to process first. At this stage, a decision needs to be made on where the Data is going to go and how it will be processed.<\/p>\n\n\n\n

“Just because you can get at the Data, doesn\u2019t mean you can get to work on Analysis.”<\/p><\/blockquote>\n\n\n\n

Will the Data flow into a Data Warehouse, a Data Lake, or a Database for the next step? Or does it make more sense to leave it where it is and query it there using another tool? A Data Engineer will be able to understand the complexity around the landing and processing large amounts of data and the best way to get it where it needs to go. This repository becomes your \u2018single source of truth\u2019<\/p>\n\n\n\n

This isn\u2019t the end of the story for the collection and cleansing of Data. Changes at source and changes in what is required for Analysis means there is always a need for a Data Engineer. <\/p>\n\n\n\n


\n\n\n\n

Structure<\/h2>\n\n\n\n

Traditionally, Data or Business Intelligence teams have reported into the IT function as they are heavy users of Databases and have specialised infrastructure needs. These teams generally work closely with, but not as a part of Product or Sales. But have the advantage of being able to lean on their IT teammates for support. <\/p>\n\n\n\n

“Data is becoming its own centralised team and is seen as a service function.”<\/p><\/blockquote>\n\n\n\n

If the team are reporting into a Product or Sales function this changes their focus. They have much more of an inside view of the team they are producing reports and Analysis for. However, they may not be able to share knowledge as easily. This also brings up the topic of Career Progression. If you are the only Analyst in the Sales team there is no obvious place to move up.<\/p>\n\n\n\n

The third possible structure is a hybrid. Teams share knowledge with their peers but remain reporting in to their Functional teams. <\/p>\n\n\n\n

Each has a tradeoff so it\u2019s important to decide which focus your team will take once the team is big enough to need to decide.<\/p>\n\n\n\n