Encrypt DataSets for Offline Storage

Data stored on a client machine represents a security hole in all of your carefully considered plans. This article borrows from the IssueVision smart client sample application to demonstrate how to protect offline data. IssueVision is a sample help desk management application that generates some choice aspects of Windows Forms programming in .NET. (click here for the IssueVision source code.) Included in IssueVision are examples of smart client updating, encryption techniques, Web Services use, a clever use of the Observer behavior pattern, and some cool GDI+ code.

This article contains original code except for a derivative of the Data Protection API (DPAPI) wrapper code presented in IssueVision. The motivation for using the DPAPI is that IssueVision is an application that works in a disconnected mode. When the application is disconnected, IssueVision serializes, encrypts, and stores changes to the DataSet on the local client. This article focuses on reading, serializing, encrypting, storing, retrieving, and deserializing a dataset. (You can use the DPAPI wrapper to encrypt and decrypt data—whatever your motivation demands.)

Reading Data into a DataSet

First, you need some data to encrypt. You can use any data, but to demonstrate an example in context, pretend that you want to support a client application for the Northwind database and that this data needs to be stored when the client is disconnected from the server.

Reading data to and from a dataset has been covered thoroughly during the couple of years .NET has been available, so I will summarize this part of the process and follow it with example code. You can use components to manage data (as demonstrated in IssueVision) or simply write the code manually as summarized here and shown in Listing 1:

1. Create a text file with a .udl extension.
2. Double-click on the .udl file to open the Data Link Properties editor. Use the editor to create the connection string. Close the editor, open the .udl file with Notepad, and copy the connection string.
3. Add an imports state for system.data and system.data.sqlclient. This tutorial uses the SQL Server version of the Northwind database, but you are welcome to use the MS Access version.
4. Create a SqlConnection object and open the connection.
5. Start a Try block, and create a DataSet and a SqlDataAdapter.
6. Use a select * from customers query and the Fill method to populate the DataSet.
7. In the Try block, return the DataSet.
8. In the Finally block, close the connection.

Listing 1: Filling a DataSet.

Public Function GetCustomers() As DataSet
  Const connectionString As String = _
    "Integrated Security=SSPI;Persist Security Info=False;" + _
    "Initial Catalog=Northwind;Data Source=localhost;" 
  Dim connection As SqlConnection = New SqlConnection(connectionString)
  connection.Open()
  Try
    Dim adapter As SqlDataAdapter = New _
      SqlDataAdapter("SELECT * FROM CUSTOMERS", connection)
    Dim data As DataSet = New DataSet
    adapter.Fill(data)
    Dump(data)
    Return data

  Finally
    connection.Close()
  End Try
End Function

Serializing the DataSet to XML

ADO.NET stores datasets internally as XML, and the ability to serialize a dataset to an XML string is an inherent capability of datasets. The DataSet class implements a WriteXML method that writes the serialized XML to a stream. To utilize a string for encryption purposes, you need to convert the contiguous block of bytes that represents a stream to a string using the UTF8.GetString shared method. The three lines of code in Listing 2 implement all this.

Listing 2: Serializing a DataSet to a string.

Public Shared Function Serialize(ByVal data As DataSet) As String
  Dim stream As MemoryStream = New MemoryStream
  data.WriteXml(stream)
  Return Text.UTF8Encoding.UTF8.GetString(stream.ToArray())
End Function

Encrypting a Serialized DataSet Using DPAPI

The copy of IssueVision I happen to have as I write this is in C#, so I ported the code in IssueVision’s DataProtection.cs file to Visual Basic .NET. All this code really does is wrap Data Protection API methods to make them easier to use in .NET, and manage dynamically allocated memory. You don’t have to create a wrapper for DPAPI, but it is a consumer-friendly thing to do.

You will use three API methods: two from the DPAPI and one for managing dynamically allocated, unmanaged memory. The methods are CryptProtectData, CryptUnprotectData, and LocalFree. To import them, you can use the VB style of API declaration or the .NET style, using the DllImportAttribute. Because you probably know the VB style of declaration—which is similar between VB6 and VB.NET—I will demonstrate how to import API methods using the DllImportAttribute. The rest of the code converts between .NET data types and API data types.

DPAPI encryption is based on Triple-DES, which uses a strong key. DPAPI is designed for user protection and relies on user credentials implicitly for its key. For added protection, the DPAPI permits you to provide an additional entropy value, which is designed to make it more difficult for a second application running under the same user logon to read data encrypted by another application. The code in Listing 3 demonstrates the DPAPI wrapper code. (If you want to learn about DPAPI from a technical aspect, check out the MSDN help from VS.NET.)

Listing 3: A VB.NET implementation of a DPAPI wrapper, based on the DPAPI wrapper in IssueVision

Imports System.Text
Imports System.Runtime.InteropServices

Public Module DataProtection

    Public Enum Store
        Machine
        User
    End Enum

    Private Class Consts
        Public Shared ReadOnly entropyData As Byte() = _
            ASCIIEncoding.ASCII.GetBytes("1295C82E-6D6E-4a01-96DD-
                                          1BF76B7F4CB4")
    End Class

    Private Class Win32
        Public Const CRYPTPROTECT_UI_FORBIDDEN As Integer  = &H1
        Public Const CRYPTPROTECT_LOCAL_MACHINE As Integer = &H4

        <StructLayout(LayoutKind.Sequential)> _
        Public Structure DATA_BLOB
            Public cbData As Integer
            Public pbData As IntPtr
        End Structure

        <DllImport("crypt32", CharSet:=CharSet.Auto)> _
        Public Shared Function CryptProtectData(ByRef _
            pDataIn As DATA_BLOB, _
            ByVal szDataDescr As String, _
            ByRef pOptionalEntropy As DATA_BLOB, _
            ByVal pvReserved As IntPtr, _
            ByVal pPromptStruct As IntPtr, _
            ByVal dwFlags As Integer, _
            ByRef pDataOut As DATA_BLOB) As Boolean
        End Function


        <DllImport("crypt32", CharSet:=CharSet.Auto)> _
         Public Shared Function CryptUnprotectData(ByRef _
           pDataIn As DATA_BLOB, _
           ByVal szDataDescr As StringBuilder, _
           ByRef pOptionalEntropy As DATA_BLOB, _
           ByVal pvReserved As IntPtr, _
           ByVal pPromptStruct As IntPtr, _
           ByVal dwFlags As Integer, _
           ByRef pDataOut As DATA_BLOB) As Boolean
        End Function

        <DllImport("kernel32")> _
        Public Shared Function LocalFree(ByVal hMem As IntPtr) As IntPtr
        End Function
    End Class

    Public Function Encrypt(ByVal data As String, _
        ByVal store As Store) As String
        Dim inBlob As Win32.DATA_BLOB      = New Win32.DATA_BLOB
        Dim entropyBlob As Win32.DATA_BLOB = New Win32.DATA_BLOB
        Dim outBlob As Win32.DATA_BLOB     = New Win32.DATA_BLOB

        Dim result As String = ""

        Try
            Dim flags As Integer = Win32.CRYPTPROTECT_UI_FORBIDDEN
            If (store = store.Machine) Then
                flags = flags Or Win32.CRYPTPROTECT_LOCAL_MACHINE
            End If

            SetBlobData(inBlob, UTF8Encoding.UTF8.GetBytes(data))
            SetBlobData(entropyBlob, Consts.entropyData)

            If (Win32.CryptProtectData(inBlob, "", _
                entropyBlob, IntPtr.Zero, _
                IntPtr.Zero, flags, outBlob)) Then

                Dim resultBits() As Byte = GetBlobData(outBlob)
                If (resultBits.Length <> 0) Then
                    result = Convert.ToBase64String(resultBits)
                End If
            End If


        Catch ex As Exception
            Return String.Empty
        Finally
            If (inBlob.pbData.ToInt32() <> 0) Then
                Marshal.FreeHGlobal(inBlob.pbData)
            End If

            If (entropyBlob.pbData.ToInt32() <> 0) Then
                Marshal.FreeHGlobal(entropyBlob.pbData)
            End If
        End Try

        Return result
    End Function

    Public Function Decrypt(ByVal data As String, _
                            ByVal store As Store) As String

        Dim result As String = ""

        Dim inBlob As Win32.DATA_BLOB      = New Win32.DATA_BLOB
        Dim entropyBlob As Win32.DATA_BLOB = New Win32.DATA_BLOB
        Dim outBlob As Win32.DATA_BLOB     = New Win32.DATA_BLOB

        Try
            Dim flags As Integer = Win32.CRYPTPROTECT_UI_FORBIDDEN
            If (store = store.Machine) Then
                flags = flags Or Win32.CRYPTPROTECT_LOCAL_MACHINE
            End If

            Dim bits() As Byte = Convert.FromBase64String(data)

            SetBlobData(inBlob, bits)
            SetBlobData(entropyBlob, Consts.entropyData)

            If (Win32.CryptUnprotectData(inBlob, Nothing, entropyBlob, _
                IntPtr.Zero, IntPtr.Zero, flags, outBlob)) Then

                Dim resultBits() As Byte = GetBlobData(outBlob)
                If (resultBits.Length <> 0) Then
                    result = UTF8Encoding.UTF8.GetString(resultBits)
                End If
            End If
        Catch ex As Exception
            Return String.Empty
        Finally
            If (inBlob.pbData.ToInt32() <> 0) Then
                Marshal.FreeHGlobal(inBlob.pbData)
            End If

            If (entropyBlob.pbData.ToInt32() <> 0) Then
                Marshal.FreeHGlobal(entropyBlob.pbData)
            End If

        End Try

        Return result
    End Function

    Private Sub SetBlobData(ByRef blob As Win32.DATA_BLOB, _
                            ByVal bits() As Byte)
        blob.cbData = bits.Length
        blob.pbData = Marshal.AllocHGlobal(bits.Length)
        Marshal.Copy(bits, 0, blob.pbData, bits.Length)
    End Sub

    Private Function GetBlobData(ByRef blob As Win32.DATA_BLOB) As Byte()
        If (blob.pbData.ToInt32() = 0) Then Return Nothing
        Dim data(blob.cbData) As Byte
        Marshal.Copy(blob.pbData, data, 0, blob.cbData)
        Win32.LocalFree(blob.pbData)
        Return data
    End Function
End Module

Start the overview of the wrapper module from the top and work your way to the bottom:

• The Store enumeration is used to determine whether the key is associated with the computer or the user. If the key is associated with the computer, any user can decrypt the data.
• The private class Consts uses a GUID for the entropy value. Using a GUID ensures that the entropy value is unique. You can create a unique GUID in VS.NET from the Tools|Create GUID menu.
• The private Win32 class is used for convenience to import the DPAPI methods and LocalFree. You used the DllImportAttribute to introduce the API methods, but VB.NET supports the Declare syntax for importing API methods too. The two constants are CRYPTPROTECT_UI_FORBIDDEN and CRYPTPROTECT_LOCAL_MACHINE. The former constant prevents a DPAPI prompt from being displayed, and the latter constant is Or’d into the flags argument passed to the DPAPI methods, if the Store.Machine enumeration value is passed to the wrapper methods.
• The DATA_BLOB structure represents the data argument that is passed to the DPAPI methods. The StructLayoutAttribute permits you to control the physical layout of the DATA_BLOB structure. LayoutKind.Sequential means that the data is laid out in the order it appears.
• The most interesting methods are the Encrypt and Decrypt wrapper methods. Encrypt and Decrypt are very similar in implementation except that Decrypt ultimately calls CryptProtectData and Decrypt calls CryptUnprotectData. The following is a basic summarization of the steps each method employs:
1. The flags argument is initialized.
2. SetBlobData is called to convert the input string into a DATA_BLOB; memory is dynamically allocated in SetBlobData.
3. SetBlobData is called to fill in the DATA_BLOB for the entropy value.
4. After the data and entropy value are copied into a DATA_BLOB, the DPAPI encryption or decryption methods are called.
5. If the API method succeeds, the encrypted or decrypted bytes are converted and returned as a string.
6. Because dynamically allocated memory must be released, SetBlobData is called in a try block and it allocates memory using Marshal.AllocHGlobal. This memory is freed in a finally block using Marshal.FreeHGlobal.
7. (LocalFree is used by GetBlobData.)

Writing an Encrypted FileStream

Naturally, the next step in storing your encrypted offline data is writing the encrypted data to a file. The .NET framework supports this. Listing 4 demonstrates the code necessary to both write and read a file using a StreamWriter and StreamReader.

Listing 4: Writing to and reading from a file stream

Public Shared Function WriteToFile(ByVal encrypted As String) As String
  Dim fileName As String = Path.GetTempFileName()
  WriteToFile(fileName, encrypted)
  Return fileName
End Function

Public Shared Sub WriteToFile(ByVal fileName As String, _
                              ByVal encrypted As String)
  Dim writer As StreamWriter = New StreamWriter(fileName, False, _
                                                Text.UTF8Encoding.UTF8)
  writer.Write(encrypted)
  writer.Close()
End Sub

Public Shared Function ReadFromFile(ByVal fileName _
                                    As String) As String
  Dim reader As StreamReader = New StreamReader(fileName, _
                                                Text.UTF8Encoding.UTF8)
  Dim result As String = reader.ReadToEnd()
  reader.Close()
  Return result
End Function

The example in Listing 4 overloads WriteToFile to permit using a temporary file name or providing an explicit file name.

Decrypting and Restoring the DataSet

When you read the encrypted file, you need to call the Decrypt method to return the XML string to its unencrypted value. To restore the dataset, you need to call DataSet.ReadXML to restore the unencrypted XML string to a DataSet and then perform an update operation against the database. Listing 5 shows how easy it is to reconstitute the dataset from the decrypted XML string.

Listing 5: Restoring a DataSet from an XML string

Public Shared Function Deserialize(ByVal XmlData As String) As DataSet
  Dim stream As MemoryStream = _
    New MemoryStream(Text.UTF8Encoding.UTF8.GetBytes(XmlData))
  Dim data As DataSet = New DataSet
  data.ReadXml(stream, XmlReadMode.InferSchema)
  Return data
End Function

Encrypting Offline Data, .NET Style

The .NET framework helps you work at a higher level of abstraction by encapsulating a lot of complex subjects into higher-level constructs. However, a lot of legwork still has to be done for useful tasks—such as encrypting offline data.

This article offered you an opportunity to experiment with ADO.NET, XML serialization, streams, and the DPAPI. You can reuse the DPAPI any time you need to encrypt user data.

About the Author

Paul Kimmel is the VB Today columnist for www.codeguru.com and has written several books on object-oriented programming and .NET. Look for his upcoming book, UML DeMystified, from McGraw-Hill/Osborne (Spring 2005). Paul is also the founder and chief architect for Software Conceptions, Inc. founded 1990. He is available to help design and build software worldwide. You may contact him for consulting opportunities or technology questions at pkimmel@softconcepts.com.

If you are interested in joining or sponsoring a .NET Users Group, check out www.glugnet.org.

Copyright © 2004 by Paul Kimmel. All Rights Reserved.