Architecture & DesignUnstructured Content: An Untapped Fuel Source for AI and Machine Learning

Unstructured Content: An Untapped Fuel Source for AI and Machine Learning content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

By Alex Welsh, Vice President, Analytics Practice, Ephesoft

Would you choose where to go on vacation if you could only access 10 to 20 percent of the reviews and information on a travel Web site? If you do, you will probably have an unforgettable trip, but for reasons you might not like. Yet, government organizations and businesses—from manufacturing to insurance companies, and healthcare to banking—are making decisions along this very same line. And, they’ve been doing so for years. They look at the easy information they can get from structured data while ignoring their unstructured data, which Deloitte believes may account for 80 to 90 percent of content generated globally, making unstructured data a tremendous source of untapped value.

Fortunately, advancements in AI (Artificial Intelligence) and machine learning now make it possible and affordable to sift through and find meaning in vast amounts of unstructured data obtained from video and audio files, e-mails, logs, social media posts, and even notifications from Internet of Things (IoT) devices. All of this data can bring about enormous benefits, such as when used to automate tasks that are manually intensive and often highly repetitive. One task, for example, is to watch out for red flags: specific criteria or behaviors that may indicate something is amiss and corrective action must be quickly taken. Let’s look at a few cases from different industries.

How about an insurance claim that appears fine on the surface, but deserves to be investigated, or a job applicant who may be hiding information? What about a shipment of highly perishable pharmaceutical products that may not have been refrigerated for a portion of their journey, or a contract that may be in violation of a country’s laws or breaks an existing agreement with another company? The important thing is that a red flag indicates issues that, if left unchecked, could cause great damage.

Artificial Intelligence Is Massively Data Hungry

How do AI and machine learning enable more efficient and effective data analysis? Through feeding it data. By giving a machine learning model examples of good and bad transactions, it teaches itself to distinguish between the two types. And, the more data the machine learning model processes, the greater it reinforces those lessons, enhancing accuracy.

So, although AI and machine learning are making great strides, businesses and other organizations need to catch up. Think of it this way: data is like fuel. We need it to power our thinking to make wise decisions. But, we’ve mined all the easy stuff, the structured data that arrives in nice and neat packages. But, here’s where the fuel analogy breaks down: Even though another gallon of gas lets us drive another 20 to 30-odd miles, the more data we put in enables us to make significantly better and more accurate decisions—not just another 20 to 30-odd miles worth—and to make them even faster.

Yet, for so long an enormous portion of our data, our unstructured data, has remained unexploited because it had been too expensive and too difficult to access and process. And, although that’s no longer the case as new technology to gather and analyze unstructured data becomes available, many people in business and other organizations have overlooked these advances.

Where the Smart Money Is

International Data Corporation (IDC) predicts that by 2020 organizations that analyze both structured and unstructured data—that is, all relevant data—and deliver actionable information will achieve an extra $430 billion in productivity gains over their competitors that do not perform such data analysis. And, businesses that understand this are not waiting until 2020. An executive at a multinational insurance company based in Germany refers to unstructured data as their greatest risk. They understand the numbers involved, and are working to ensure they’re not caught off guard by writing insurance policies that expose them to liabilities they could have avoided.

The combined power of big data, AI, and machine learning can make it easier to process information related to even more complex challenges. For example, banks and other organizations can more accurately and more rapidly detect fraud, tax evasion, money laundering, and other schemes by mining what had previously been unprocessed, unstructured data. This enables them to catch and shut down cases of fraud and abuse, as well as avoid the many false positives that can occur when relying only on structured data. Trade finance agreements, including contracts and multiple data sources, between countries or companies also can be scoured to determine if fraud or inequities exists, whether they’re intentional or not.

Furthermore, AI and machine learning can help banks and other kinds of businesses better identify and verify the identity of their clients through automated Know Your Customer (KYC) procedures. Such procedures can help prevent them from being used, deliberately or inadvertently, for money laundering activities as well as help avert bribery and other forms of corruption from occurring. KYC procedures also can enable businesses to better understand their customers’ financial dealings and needs, as well as help them more prudently manage risk. Other advantages include speeding up time to revenue when onboarding new customers, making KYC not another cost to incur but, instead, a source of profit.

AI and Machine Learning Can Increase Your Competitiveness

With all of the benefits gained through AI and machine learning—and the advances in technology used to process structured and unstructured data—it’s time for more businesses and organizations to take advantage of the greatest source of information available: their own unstructured data.

About the Author

Alex Welsh leads Ephesoft‘s global Analytics Practice. He is an experienced sales director, project manager, and entrepreneur with a passion for solving customers’ mission-critical problems with innovative cost-effective technology solutions.

Get the Free Newsletter!

Subscribe to Developer Insider for top news, trends & analysis

Latest Posts

Related Stories