Ten Things Every Data Developer Should Know About BI and Analytics
Do you remember the days when automated reports were called Business Intelligence, or BI? Shortly afterward, discussions about workflows, processes, and user experiences among technical and business users was a true definition of Business Analysis, or BA. Recent advances in data analysis, predictive modeling, machine learning on data mining, plus social, text, and risk analysis, have created a focus on Predictive Analysis, or PA. BI, BA, and PA are concerned with the discovery, interpretation, and communication of meaningful patterns in data and that is the definition of analytics, according to Wikipedia.
BI Business Analysts (BI BAs) align with non-technical subject matter experts (SMEs) to understand their data needs for making informed decisions. Predictive Analysts (PAs) are often data scientists with a strong background in data mining and statistics.
Proper BI implementations start with user stories written by or elicited from subject matter experts. As a BI BA, you will ensure that these user stories are written in the voice of the business and at a granularity that can be discussed by both business users and data technologists. For example, "As a product manager, I need to see the profitability of components by supplier for each unit so that I can assess our supply chain." As a PA in the insurance industry, you might mine demographic, psychographic, econometric, and statistical data while assessing pricing models.
As a developer, you create code that requests, validates, copies, moves, stores, transforms, derives, and displays data in a widening array of formats. Here are ten things to know as you work with BI and analytics.
1. Self-service BI Is a Requirement
We are nearly two decades into the 21st century. Modern BI environments are expected to have data analysis features for business users that are as evident to use, as an airport flight kiosk or bank ATM. For extended analysis, self-service BI should be as rich as music discovery systems. Amazon Music and iTunes allow stored selections, related selections, and discovery by genre, year, artist, and other metadata. You are familiar with those interfaces and are seeking information on unknown MUSIC. Similarly, when you use CarMax or Autotrader, you have literally dozens of options available for narrowing your data mining activity for a VEHICLE. Music and vehicles have self-service options. It is no wonder that today's savvy business user expects and understands self-service analysis.
2. Alerts Valued over Static Reports
Your phone is used for far more activities than voice conversations. Messaging is a significant category of code on its own. Far more than simple SMS, rich messaging is leading users to expect short, meaningful alerts as more than text. A picture, emoji, animation, or video is enhancing the information previously sent as a text message. Even a text message to your phone is quite an advance when compared to the vintage practice of preparing a full report for recounting a change in a key performance indicator. Deep dives into the why and how of an event are useful, though not when the knowledge of an event is the primary message.
3. Descriptive Analytics Are but a Starting Point
"What happened?" Rearview mirrors are valued. They have not been considered accessories for decades. Reports share a similar valuation. Your multi-column reports with rollups and drilldowns of data for activities that happened in the past are needed within the organization. The pace of data acquisition and the need for current or forward-looking analysis will guide your analytics maturity.
4. Data Integration Has Moved Beyond Operational Systems
An inventory of your internal data systems can be eye-opening. It is not at all unusual for a regional bank, for instance, to have 50 or more data producing software systems. Core banking, teller management, ATM, lease management, and duplicative systems gained through mergers have underlying database systems. They range from homegrown to file managers to SQL Server, Oracle, MySQL, and a slew of other commercial and open source engines. If you work for a mid-size or larger organization in any industry, you are familiar with the range of DBMS systems, versions, and data silos that exist with internal operational systems and related subsets of data found in Excel documents on the shared file systems.
For most industries, this data previously existed only in character-based formats. In recent years, the data has begun to include social media threads, audio, images, and video. Your data integration processes are expected to include such data for analysis. Your mining will need to include qualified datasets from government and industry sources to provide a proper analytics base.
5. Diagnostic Analytics Look Back in Time
"Why did it happen?" is your question for diagnostic analytics. Why did the commercial builder fail to repay the loan? Why did the patient need to wait for two hours past the appointment time? Why did the software stop working unexpectedly? Your analysis into each example involves acquiring and mining historical data and metadata.
6. Data Visualization Is not Shiny
Are you a Tableau developer? It is the leading commercial data visualization software. When users seek trends in data, visualization is a common approach, rather than looking at page after page of a report or multiple column sorting in Excel. Visualization was the shiny object until just recently. Now, the use of augmented reality to analyze data is possible. Your choices among platforms include phones to full computers. The application programming interfaces and developer programs are available. You can build a data analysis solution that is "in the room." Your imagination is valuable in augmenting reality with data in, for instance, a heads-up display.
7. Predictive Analytics Are Probabilistic
"What is likely to happen?" Predictive analytics are focused on prediction of future probabilities and trends. You do not have access to data from the future, so you can only predict outcomes to a given probability. That surety rises with the quality and volume of data for a given issue. Determining the next item that an online shopper might order is a recognized example. Predicting the value of a customer over a period of time is often cited. You will likely use your background in statistics, beginning with regression analysis of a set of variables as you seek a statistical correlation. Coefficients lead to analysis of variables within larger datasets and probabilities that are tuned over time. You will find that valuable predictive analytics can have probabilities that are barely above 50%.
8. IoT Stream Analysis Will be Pervasive
The Internet of Things (IoT) provides data at a pace and volume not usually seen in typical transaction systems. Sensors on vehicles spew data that, unlike the Popular Science articles of the 1960s, are not dependent on embedded devices in roadways. The sensors are on-board, communicate with vehicle-based processors, and interact with other vehicle systems. Driverless vehicles are the result. Your approach to analyzing streams of data in real-time narrow the scope. Further analysis of streamed data at a broad scope is conducted in near real-time or historically. You will need to filter "noisy data" that is not relevant to a study.
9. Prescriptive Analytics Recommend Courses of Action
"What course of action should be taken?" That is the question that prescriptive analytics attempt to answer. Your approach to answer the question is based on optimization or simulation models that are often associated with forecasting and risk avoidance.
10. Analytics as a Service Provides Deep, High Quality Data for Enrichment
The maturing of the cloud is giving rise to many levels of services. Software as a service is offered in thousands of systems, some as small as a to-do list, and others as large as the multi-tenant Salesforce CRM. Now, everything as a service is taking a place at the table. With that, data and analytics are to be key services. Enriching your internal data with high-quality broad data is offered by boutique and internationally-known firms. IBM's Watson is available as a service. You can quickly hook up a RESTful API and get massive compute power against your data. Expect narrowly focused data analytics clouds to be widely available in the next few years. Your data will become a currency with which to enrich your internal analytics operations.
Change is continual in business. BI and analytics are your means of providing the organization with indicators and measures on the trends and health of the company.
About the Author
With decades of experience around the world in data-focused IT consulting, David Leininger is Fusion’s Solution Director for Data whose mission is to help managers and executives understand the importance of crafting an operations strategy that realizes the business value of their data. David leads a team of data technologists who carry out this mission by integrating and exposing data for the value that is usually hidden from everyday operational reporting systems.