March 3, 2021
Hot Topics:

Visual Basic 6 Win32 API Tutorial

  • By James Limm
  • Send Email »
  • More Articles »

If you've ever programmed in languages other than Basic, you've probably noticed that most of the data types are very similar overall, but not exactly the same. For example, if you added the following variable declaration in Visual Basic:

Dim intCount as Integer

You would have a variable that could hold integer values from -32,768 to 32,767. However, if you make the following declaration in Visual C++:

int intCount;

intCount would have a range of -2,147,483,648 to 2,147,483,647.

I'm already envisioning a response like, "Well, what does this have to do with us? We're VB programmers, not Pascal or C programmers." True, you don't have to worry about other languages when you're in VB. But when you make an API call, you're starting to walk out of the world of Visual Basic and into foreign territory. There are a lot of little surprises in store for the VB developer who doesn't take the time to figure out the exact data type that should be passed into an API call. For example, say a C programmer made a DLL for you on April 1st named HaHa.DLL, and said that the Gotcha function takes one integer parameter called WatchOut. It also returns a string. Ignoring the subtle hints, you add the following declaration to your code:

Declare Function Gotcha Lib "HaHa" _
(WatchOut as Integer) as String

This looks like it should work OK, but if you try and use this function like a VB function, you will soon run into a few problems. After this chapter is done, you'll be able to see the warning signs immediately. For now, let's go over the data types that can do cause trouble if you're not careful (I'll leave the method of revenge that should be incurred upon that C developer as an exercise to the reader).

By the way, even though I'm going to show you code in C, don't think that you've got another language to learn. I'm only doing this to demonstrate the differences in data types between languages, especially with a language that is used to create DLLs. Appendix A lists the data types you should use in VB for the C data types found in most DLL calls.

Visual Basic Strings and API Calls

If you stay within the Visual Basic world, strings are pretty straightforward. Variable-length strings by definition grow and shrink with no special coding tricks. Here's a code snippet to demonstrate this:

Dim strTest as String

strTest = "First value."
strTest = ""
strTest = "Make it relatively bigger than before." 

There's no magic going on here. VB is handling whatever memory allocation is needed to add and remove space as needed. C programmers don't have that luxury. They have to declare their strings using character arrays, like the following C code line:

char strTest[13] = "First value.";

C also requires that you end, or terminate, any string with the null character ' '. Most VB programmers use the returned value from Chr$(0) to create this character. When you declare strTest in C this way, you don't have to explicitly add the null character to the end of the initialization value - it's done for you. Therefore, even though the string is only twelve characters long, we have to make sure there's room for that terminating character. You could allow the compiler to figure out how much space strTest needs by changing the code like this:

char strTest[] = "First value.";

but you could never have more than 13 characters in strTest after this declaration.

Of course, C has pointers and addresses that makes string manipulation much faster than VB, so if you have a lot of string processing going on in your application, you may want to have that friendly C programmer down the hall create a DLL for you to speed that up. (You may also want to tone down that revenge thing I mentioned before a little bit). But that's exactly where one of our problems lie. In C, strings are usually declared of the type LPSTR, or long pointer to a string. Here's what that looks like in memory:

Technically, the acronym LPSTR is a bit deceiving. In C, LPSTR is typed as:

typedef CHAR* LPSTR

This means that LPSTR is actually a pointer to the first character of a string.

As we have seen, this string is just an array of characters. In VB, however, strings are actually of a type called BSTR. Here's what this looks like:

VB used to use the HLSTR data type to define its strings. I won't go into the memory structure here, but if your fellow VB programmer who's stuck on maintaining a 16-bit application in version 3.0 is having problems with API calls and strings, you may want to start looking at this type. They used a pointer to a pointer to the first character in the string, which really made things confusing.

You may wonder why you don't see the termination character whenever you use strings in VB. It's one of those magical features of the tool - it simply hides it from you whenever you read from or write to strings. Technically, it is there and as we'll find out, it can be quite useful.

Differences Between Fixed and Variable Length Strings

Fixed-length strings are similar to variable-length strings in that they both hold character information. Also, you pass a fixed-length string ByVal just like you would for a variable-length string. The only way they differ is that with fixed-length strings…well, they're fixed in size. As we saw with variable-length strings, we can grow and shrink the string with ease. However, as this following line of code shows:

Dim strFixed as String * 5

we can only have 5 characters in the string at one time. Personally, I use variable-length strings when I make API calls that require strings as parameters, but there are always exceptions.

As you can see, there's one big difference between BSTRs and LPSTRs, and it's the length parameter that's tacked onto the beginning of the BSTR. There's a nice feature to this - VB doesn't have to calculate the length of a string every time the Len is called. It simply looks up the value stored within the string, and passes that back to the caller. (Of course, VB must change this value when the string changes length.) However, one major drawback of this is that a DLL function expects a different type of string. The DLL doesn't want a BSTR - it wants a LPSTR.

So how do you get around this? Is there some convoluted method to convert a BSTR into a LPSTR? The answer is no - once again, VB handles all of this for you. Let's look at a simple API call to illustrate this - it's called FindWindow:

Declare Function FindWindow Lib "user32" Alias "FindWindowA" _
(ByVal lpClassName As String, ByVal lpWindowName As String) _
As Long

This function will return a window handle for a window that matches the information given. What we're more concerned about here is how we're going to get that information to FindWindow correctly.

You may wonder why the function is really called FindWindowA in the DLL. This is the ANSI declaration; a Unicode version called FindWindowW exists as well. We'll cover Unicode later on in the chapter.

As it turns out, it's the ByVal keyword that saves us. When you pass a variable of a String type using this keyword, VB will actually pass a reference, or pointer to be exact, to your string. Furthermore, since your string is actually null-terminated internally, you don't have to append a termination character to the end of your string every time you call an API that requires a string.

There are a couple of catches here. The function will stop at the first null terminating character in the string. Remember that when VB passes the string to the API, it's actually passing the address of the first character to the call. Therefore, the call really has no idea how long the string is, or, more accurately, how much memory you've allocated to hold the string's contents. It's depending upon that end character to notify the DLL when the end of the string has been reached. So make sure that there are no null characters within the string when you pass it! Although you probably won't throw a memory exception if you do this, any of the data that exists past the first null character will not be read by the DLL.

Another catch is the keyword overloading. Although you have explicitly declared that the string should be passed ByVal, the string could get modified within the DLL. This is a head-twister when you first read it. You're passing it ByVal, but it's acting as though you passed it ByRef! Why is this the case? I really, really wish I knew! Personally, it would make sense to me that if the DLL function can change the contents of a variable, it should be passed ByRef. But that's not the case. This is one of those little quirks of the VB environment that, whether or not it makes logical sense, you must follow to ensure your strings are handled correctly by DLL functions.

Also, be sure that you read the documentation on the call you're making - since the DLL is actually referring to your string variable, they can alter the contents of the string itself. This is usually what you want to have happen in most calls, just don't be surprised if the data is different after an API call.

Usually when the DLL will change your string, it also will ask you to tell it how long the string is. For example, let's take a look at the following API call to illustrate this scenario:

Declare Function GetWindowsDirectory Lib "kernel32" _
Alias "GetWindowsDirectoryA" (ByVal lpBuffer As String, _
ByVal nSize As Long) As Long

This function returns the full path of the Windows directory. The lpBuffer is the variable that GetWindowsDirectory will fill with the path, and nSize is the length of the string. This function will return the length of the string copied into lpBuffer on success (minus the termination character), and zero on failure. Note that if the string you passed in wasn't big enough, this function will return the length of the buffer required. Here's how you'd get the path in VB:

Dim lngRet as Long
Dim lngSize as Long
Dim strWindowsPath as String
lngSize = 255
strWindowsPath = String$(lngSize, " ")
lngRet = GetWindowsDirectory(strWindowsPath, lngSize - 1)
If lngRet <> 0 then
If lngRet > lngSize Then
' We have to make the call again
' with a bigger string!
strWindowsPath = Left$(strWindowsPath, lngRet)
End If
End If

What we're really doing here is creating a memory buffer. We use the String$ function to fill the string with 255 spaces. After calling GetWindowsDirectory, we check the return value (note that we didn't have to use the ByVal keyword for the two arguments, since the function was declared with the arguments ByVal). If everything's OK, we get rid of the spaces in strWindowsPath using the Left$ function. Note that we passed in the length of the string minus 1. This is kind of a "safety net". Since we know that we've allocated for 255 characters, and we also know that the string is null-terminated underneath the covers, we should expect that the DLL will only use those 255 characters. But to be safe, we'll tell the function that we've only allocated enough space for 254 characters.

The problem lies with initializing the string and passing in the right length value. If we're not careful in our code, we may pass in a value like 2555 for nSize. The DLL thinks that you've allocated 2555 bytes in your string, and it might try to use them all. If it actually tries to go past that 255th byte, you're in big trouble! Whatever memory exists past that 255th byte, we didn't allocate it. Therefore, you're almost guaranteed to cause a memory exception in this case. This is a classic problem that many VB programmers have run into, so make sure you pass in the size correctly!

Another problem with allocating the string is not doing it at all. In this case, the value of nSize always corresponds to the size of the string. However, some functions do not have arguments that allow you to tell the function how much space is available in a buffer, and require that you allocate enough space in a string. If you don't do this, it's possible that the DLL might try to write to a location in memory that your string hasn't allocated, and a memory exception will occur. Again, make sure the string is allocated correctly for the function used.

One nice trick that you can use to make sure you always pass in a nice, "safety net" value for string lengths is by using the Len function on the string, and then subtracting one from the value. This will ensure that the DLL function has enough valid space to write data to.

There's one more issue we need to address with strings. We now know how to pass them in, but what happens if the call returns a string? You may think that it's not that big of a deal, but if you're guessing that it gets really ugly, you're right. Fortunately, these calls are the exception and not the rule, but we still need to understand what we need to do to make the call correctly. Let's look at another API call:

Declare Function GetEnvironmentStrings Lib "kernel32" _
Alias "GetEnvironmentStringsA" () As String

All this API call does is return the current Windows environment settings. Note, though, that the Declaration Loader says the call returns a String data type. From what we just went through with string lengths, do you think this code will work?

Dim strEnv as String
strEnv = GetEnvironmentStrings

When I tried to do this, I got a bunch of garbage. DLL functions don't return a String data type the way you would in VB. They're actually returning a LPSTR - a pointer to the first character in a string. In fact, if you did some more research on this call, you'd see how the function was declared for a C point of view:

LPVOID GetEnvironmentStrings(VOID)

That LPVOID is telling us that we're getting a pointer back. As you know, VB doesn't have any kind of pointer operations to help us out, so it looks like we're stuck. Fortunately, there's an API call that can help us out here, and by using this call, we'll move into another area where memory exceptions can really nail you: typeless variable declaration.

To find out what the function declarations look like in C (like I did above for GetEnvironmentStrings), go to Microsoft's search site at http://search.microsoft.com/default.asp. All of the Win32 documentation is at their Premier level, but registering for this level of service is free

Page 2 of 13

This article was originally published on November 20, 2002

Enterprise Development Update

Don't miss an article. Subscribe to our newsletter below.

Thanks for your registration, follow us on our social networks to keep up-to-date