7 “Not Easy” Steps for Securely Using Data for Real-Time Decisions

A step-by-step roadmap for taking control of your data, securing it and making it meaningful to everyone at the same time, in the same way.

Originally published on LinkedIn on Oct. 22, 2019.

Companies have data in many places. And many companies do not know what data they have, where it is stored, who and what has access to it, the trustworthiness of the data or how to organize it in a timely manner into decision criteria for leadership teams.

The easiest way to know if what I’m saying is truth is to ask someone on your technical staff to provide you an asset and access inventory. Ask them the following:

Tell me:

- All software applications used in the company
- All places data is stored in the company
- All hardware used in the company to host, edit and manage both
- Who/what has access to these things and with what levels of power

And

- How the data is secured in transit and at rest

Give them one business day. Their reaction will reveal your truth.

Running a company minimally requires two things: knowing where you want to go and having access to timely, trustworthy data that will guide your journey. This article discusses the data aspect only.

And as you may already hope, suspect or know, addressing unsecured, unmanaged, disparate applications, data and permissions is a solvable problem. Accessing one view into your company is also solvable. Let’s look at the plan.

1. Find Your Data

Inventory all software applications and data repositories inside and outside your company, as well as, anything interacting with or exchanging data with your applications and repositories.

2. Determine The State of Your Data

What is the technology collecting, managing, editing your data? Where is it hosted? By whom? Is it good, questionable or corrupt data? Who and what has access to it? What are they doing to the data? Who is managing the security and sanctity of the data? How do you know you can trust the data? Is the data current and with what frequency?

3. Secure your data

Is the data managed via role-based permissions or is it wide-open for too many people and systems to manipulate, extract and exploit? Is it direct-connect? Copy-paste? Batch-uploads? API-accessible? Is it secured while at rest? Is it secured while in transit?

Think your company not likely to be attacked, corrupted, ransomed or otherwise exploited? Consider your brand value, consumers, privacy laws and bad company press. Do people trust your brand today? Will they after a breach?

4. Establish a Common Data Format

When data originates from multiple data sources, the structure of the data is usually non-uniform. The first step is to understand the current structure and state of all data at the origination point.

The second step is to determine to what Common Data Format (CDF) all data will be funneled and/or otherwise re-organized. In other words, if your company’s growth strategy has been through Mergers and Acquisitions, you likely have many data stores with similar types of data, but with different states of sanity. If you want one view across all of these data stores, words must have the same meaning for all instances of all data. Establishing the same meaning for all similar instances is “normalization” or “establishing a Common Data Format”.

Many to one.

Only after there exists a common data format are you able to see, understand and make decisions that confidently and consistently take into consideration all parts of the company.

No alt text provided for this image

5. Extract, Normalize and Put

When you understand all places from which data originates and have a CDF, your teams are then able to write predictable, repeatable and auditable methods of extracting, normalizing and putting data into your new, single source of truth.

To be clear, the methods of extracting data, normalizing data and putting data must be predictable, repeatable and auditable. And the structure into which all data is put is itself the CDF. Anything less and you will simply be creating a new mess that must be managed on top of your existing ecosystem — whatever the state.

6. Pull Data Predictably

Now that you’ve made the effort to ensure all data, from all locations, is secured and normalized, protect it. This means there must exist a predictable, repeatable and auditable manner by which applications, systems and companies access your data. Notice I didn’t say people.

To access data from the single source of truth, there must exist predictable, repeatable and auditable set of actors, permissions and activities. If there is variability in actors, permissions and activities, it will no longer be a single source of truth.

Require anyone or thing that wants access to your data to follow your rules. Non-negotiable. This includes people in Mensa, people with twenty years of tenure who have been there since the company started, the CEO’s nephew and your mom.

Your single source of truth is special. No one who wants access to the data is special. Despite what their mom told them when they were young.

7. Use Your Data to Inform Your Decisions Dynamically

Attach reporting solutions. Attach streaming solutions. Attach elastic search. Attach dashboards. Follow the rules. Enjoy peace.

Now you can trust that your data has integrity. You can trust it is secure. You can trust your data is predictable, repeatable and auditable. You can trust your company has one message.

And you can trust that you know all applications, repositories, data management and security behaviors, actors, hosting solutions and reports are something upon which you can bank your company’s reputation.

____________________

If you would like to take control of your data, secure it and make it dynamically meaningful to everyone in your company, the teams at Trility help companies solve these challenges with a focus on predictable, repeatable and auditable behaviors. Email us at forthejourney@trility.io.

Did You Mean DevOps or DevOps?

DevOps is another word starting to lose its meaning in today’s marketplace. Ask an Engineer and DevOps is a set of tools or programming languages to take code from the Developer, package it up, and promote it through environments to production. Ask a Product Owner and DevOps is a separate team of individuals maintaining the Continuous Integration/Continuous Deployment pipeline. Ask a Senior Manager and DevOps is a culture inside the organization to promote interdepartmental communication and breaking down barriers. Three opinions: a suite of tools, a team of individuals, or a new corporate culture.

More often than not when an organization is looking to solve problems with throughput from the product teams, a DevOps group is assembled to bridge the silos between various parts of the organization. Unfortunately, instead of bridging the gaps between the silos in the organization, a new silo is created around a set of new tools aimed at Continuous Integration and Continuous Deployment. Instead of connecting groups, the organization now has another team competing for precious budget, is bringing new tools and processes into an already complicated technical stack, and in the end the core problem still remains.

In the F5 State of Application Delivery report for 2017, DevOps was only selected by 20 percent of the 2,197 respondents as having a strategic impact on application delivery. Software as a Service (SaaS), Big Data, and Infrastructure as a Service (IaaS) in the public cloud were far ahead of DevOps in strategic impact for the organization.

Interestingly, overall, DevOps was selected as having strategic impact by just 20 percent of respondents. Among those in executive roles, DevOps was identified by only 17 percent, well below front-running SaaS (42 percent), big data (41 percent), and public cloud (IaaS) (39 percent). Among those identifying as having DevOps and cloud-related roles, DevOps took third place with 39 percent behind SaaS (44 percent) and big data (42 percent). This is in spite of evidence that organizations are engaging in at least the automation and orchestration aspects of DevOps.

However, according to the 2018 Open Source Jobs Report published by The Linux Foundation and Dice, DevOps skills (59%) are second only to Developers (72%) as the most sought after position. Organizations have a real need to solve operational throughput problems to get products from the napkin to in front of the customer efficiently. Simply having a DevOps group inside the organization is not getting the job done. Clearly there seems to be a disconnect between the strategic value of DevOps and the drive to continue to build a DevOps team.

Given all of the confusion regarding how DevOps is defined and the strategic impact to an organization, it is important to have a single definition communicated throughout the organization. 

At Trility Consulting, we believe the delivery of testable, secure, software to production on a regular, repeatable cadence is the responsibility of everyone on the team. There are no walls, there are no silos. It all starts with amazing Product Owners, Developers, and Delivery Engineers who are focused on delivering an exciting application development experience. Everyone on our project teams are dedicated to diving in and understanding how we solve business problems together. 

When you hear a member of the Trility Consulting team talk about DevOps we are talking about our commitment to the business problem at hand and delivering value to the clients we have the privilege to serve. As a team we are focused on delivery.