Implementing Visual and Data Mashups Online
What Are Online Web 2.0 Mashups?
In my previous article, "Web and Enterprise Mashups for Web Services and Data," I covered different tools for the enterprise mashup creation, including web services mashups and data aggregation mashups. I also have shown how tools such as Aqua Logic can facilitate web services mashups. These tools are geared to provide a new breed of enterprise services and to ease business application development. But, these tools do not work with some popular communication protocols, such as RSS. In this article, I will discuss online mashups, which are not yet designed for the enterprise but could lead to new and exciting online applications. This, in turn, can lead to new businesses based on the ideas or implementations that arise from these application and services.
When culinary master chefs compete on the Iron Chef TV show, they are given a lot of ingredients and one secret ingredient to be used in all the dishes. They also are given an entire professional kitchen to prepare their sumptuous creations. The final dishes then are judged by a small panel of experts who get to comment on taste, texture, presentation, and overall quality. The viewers get to see the whole process but don't get to participate in it, nor do they get to try the food.
Imagine for a moment that all of the millions of TV viewers would get to participate, and would get the same access to the same ingredients and same professional kitchen and its equipment. On top of that, whatever one viewer creates can be taken and improved by another viewer and then the entire community of viewers would get to taste and judge the creations.
Even though the master chefs have years of experience in the culinary arts, if they compete against the "community" they would not be able to come up with what the majority of people exactly want. The same concept is behind online web mashups, but instead of food, the ingredients are much less caloric, and much more data and solution centric. The idea of the millions of developers creating something together online practically guarantees that whatever will survive the judgment of the masses will be the most useful and popular application.
Many large software companies jumped on this idea and developed environments where developers can collaborate on combining content, functionality, and APIs to create new applications and services. These environments also let people share and publish whatever they create. Anything from simple apps for personal use to complex applications and services can be created in these environments. These apps then undergo process of elimination, improvement, and evolution by the whole community. The concept of online mashups is in line with the web 2.0 ideology as its very social in nature.
Mashup Development Environments
The online mashups would not be possible without advances in the modern web technology standards and technologies. Communication protocols and data exchange standards, in particular, act as a corner stone for the modern Mashup development. All of the mashups environments are ether downloadable browser extensions, downloadable software, or web based solutions. The technology behind these tools and sites is different, but they all rely on HTTP, RSS, SOAP and other standards to exchange data, call APIs and communicate with other mashups. They all allow some form of editor to create the combination of data, services or APIs and then some form of test interface and publishing interface.
Tools and Browser Extensions
Intel MashMaker is an example of a browser plug-in for Mashup development. This browser extension installs in Firefox or IE and runs the machine. It's a visual tool, for data and presentation mashups, where all of the mashups are shareable and appear as composite widgets inside of the browser window. Essentially, the pages are modified to include an xtra layer of information on top of the actual page rendered in the browser. After installing the plug-in, Mashmaker will analyze sites being viewed for any possible technologies it can understand (web content, videos, maps, RSS feeds, photos) and allows users to manage the underlying data and combine it with other data sources. Afterwards, it can store the compilations, share them, and let others vote on them. It also allows users to discover new mashups. For example, the tool can find geolocation information on a popular social networking site (like Facebook), combine it with the Google Map widget, and show the location of your friends visually on the map, with their pictures. To do so, it uses a collaboratively edited shared database of extractors to obtain meaning from pages on the web.
Among all other downloadable tools, MashMaker is the most refined. It has a good collection of pre-build widgets, good documentation, and examples. Conceptually, it's also very different then the other tools; it can discover data sources and change the way a page is presented to the user based on them.
Another example of a downloadable package is Serena Business Mashups (which together with Serena Mashup Composer), provides developers with a point-and-click environment for business-related mashups creation. The package essentially is a web application running on an app server. The mashups developed with Serena will run in an application container deployed to an app server and combine content and processes from many different systems, using industry-standard SOA and Web services interfaces.
Online only mashup creators
Yahoo Pipes is designed primarily for visual creation of data mashups. It can take any data source, such as a CVS file, RSS feed, search engine output, url output, and so forth, and apply certain manipulation on the data to get the final output "through the pipe" in the way that the user wants. An example would be an RSS feed in French, translated into English dynamically, or a combination of two news RSS feeds into one and containing only terms that the user wants. Yahoo Pipes is the most polished and stable from the current online mashup offerings. However, it does have some limitations. For instance, the majority of mashups done with Yahoo are done around some sort of data feed because it is the easiest way to get data into the pipe.
Yahoo Pipes also has a collection of predefined operations that can be applied to the data source, such as extract of geo information, sorting, splitting, regex, and so on. But, it does not have a robust collection of predefined widgets like MS Popfly or MashMaker does. I could not find a way to extend the default widget set or to do a simple operation, such as an intersect of two feeds—a scenario where the pipe would output only items that are common in both feeds. This means that users can use only what Yahoo is providing and this somewhat limits the benefit of the application.
Here are screen shots of Yahoo Pipes:
Figure 1: Yahoo pipes mashup editor