Microsoft & .NET.NETBuilding Smart Apps that Understand Language, Speech, Images, and Face Recognition

Building Smart Apps that Understand Language, Speech, Images, and Face Recognition

Developer.com content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

Introduction

The amount of information that users access today is unprecedented and the growth of Internet and mobile devices has just added to the demand.

With the increased amount of data available to users, it is becoming increasingly important to have a way to smartly process the data. Microsoft’s Project Oxford offers APIs that can be called from an application to interact with audio, text, image, and video data.

Here are the APIs that are offered as part of Project Oxford.

  1. Face APIs: These APIs allow to see users with Face detection and recognition. The APIs support the following:
    1. Face detection
    2. Face grouping
    3. Face identification
  2. Vision APIs: These APIs support:
    1. Analyzing image
    2. OCR
    3. Generating thumbnails
  3. Speech APIs: These APIs support:
    1. Speech recognition
    2. Text to speech
    3. Speech intent recognition
  4. Language Understanding Intelligent Services: These APIs help with understanding natural language commands tailored to an application. Examples include:
    1. Detect intent
    2. Determine entities
    3. Improve models

To use Oxford services, visit http://www.projectoxford.ai/ and sign up to use the subscription keys.

In this article, we will show how to use Face APIs. Sign up for the Face APIs at https://go.microsoft.com/fwlink/?linkid=271033&clcid=0x409. Once you have signed up, you can download the SDK from http://www.projectoxford.ai/SDK.

We will now create a simple Windows 8 application. Create a new Visual Studio 2013 project that uses the Windows 8 “Blank App” template.

SmartApps1
Figure 1: The new, blank project

Next, add a reference to the ClientLibrary project you downloaded from the SDK link above.

SmartApps2
Figure 2: Adding the ClientLibrary reference

Then, we add a few controls to the Default XAML page created for the app. We will add a button and a couple of textBox and TextBlock controls.

The XAM codebehind for MainPage.xaml will look as under:

<Page
   x_Class="ProjectOxfordDemo.MainPage"
   
   xmlns_x="http://schemas.microsoft.com/winfx/
      2006/xaml"
   xmlns_local="using:ProjectOxfordDemo"
   xmlns_d="http://schemas.microsoft.com/expression/
      blend/2008"
   xmlns_mc="http://schemas.openxmlformats.org/
      markup-compatibility/2006"
   mc_Ignorable="d">

   <Grid Background="{ThemeResource
         ApplicationPageBackgroundThemeBrush}">
      <Button x_Name="buttonIdentify"
         Content="Identify"
         HorizontalAlignment="Left"
         Margin="442,177,0,0"
         VerticalAlignment="Top" />
      <TextBox x_Name="textBoxGender"
         HorizontalAlignment="Left"
         Margin="884,200,0,0" TextWrapping="Wrap"
         VerticalAlignment="Top"/>
      <TextBlock x_Name="textBlockGender"
         HorizontalAlignment="Left"
         Margin="809,200,0,0" TextWrapping="Wrap"
         Text="Gender" VerticalAlignment="Top"/>
      <TextBlock x_Name="textBlockAge"
         HorizontalAlignment="Left"
         Margin="809,273,0,0"
         TextWrapping="Wrap" Text="Age"
         VerticalAlignment="Top"/>
      <TextBox x_Name="textBoxAge"
         HorizontalAlignment="Left"
         Margin="884,254,0,0" TextWrapping="Wrap"
         VerticalAlignment="Top"/>

   </Grid>
</Page>

Next, we make changes to MainPage.xaml.cs as follows:

Include the following namespace.

using Microsoft.ProjectOxford.Face;

Create an instance of FaceServiceClient.

   public sealed partial class MainPage : Page
   {
      private static FaceServiceClient _instance;
      public MainPage()

Finally, we implement the Click Event handler.

      private async void buttonIdentify_Click
         (object sender, RoutedEventArgs e)
      {
         // Note that you need to insert your subscription key here.
         _instance = new FaceServiceClient("YOURSUBSCRIPTIONKEY");

         string imageURL = "http://upload.wikimedia.org/wikipedia/
            commons/thumb/8/8d/President_Barack_Obama.jpg/
            220px-President_Barack_Obama.jpg";
         var detectFace = await _instance.DetectAsync(imageURL,
            true, true, true, true);
         foreach (var face in detectFace)
         {
            var gender = face.Attributes.Gender;
            var age = face.Attributes.Age;
            textBoxAge.Text = age.ToString();
            textBoxGender.Text = gender;
         }


      }

Note that in the function above, you will need to use your own subscription key. I have removed mine for security purposes.

In the preceding method, I have referenced the URL for the President of United States. The previous implementation will detect the person and detect the age and gender of the person.

Now, compile your code and execute the application.

Summary

In this article, we learned about the Oxford service and how we can use it to make applications smarter. You can download the code sample at the bottom of this article.

About the Author

Vipul Patel is a technology geek based in Seattle. He can be reached at vipul.patel@hotmail.com. You can visit his LinkedIn profile at https://www.linkedin.com/pub/vipul-patel/6/675/508.

Get the Free Newsletter!

Subscribe to Developer Insider for top news, trends & analysis

Latest Posts

Related Stories