The API Field of Dreams: Build It [Correctly] and They Will Come
This article is aimed at architects and technology leaders, as a guide to basic design concepts for application programming interfaces (API). It is not intended to be an introduction to APIs themselves, a discussion about why you might want to use an API, or a presentation of the different types of APIs and technologies available.
Although the title of the article is a bit tongue in cheek, it might need a little qualification. In my opinion, you don't want to build an API that will make people come to it…you want to build one that will make people come BACK to it again and again…and make them WANT to use it. If your company's brand is big enough, your founder or CEO or sales staff has carved out some market share and most likely ensured that they will come—as soon as the word hits the street that there is an API for developers to interface with Megacorp, they will want to check it out—your job is to build an API that is so awesome that their excitement level remains high as they use your API the first time, and every subsequent time. Remember, your API is ideally going to facilitate those developers to help you extend the reach of your brand.
Know Your Customer
First things first, you should know your customer. The customers of an API are developers. You should always keep that in mind as you design and build your APIs. Developing something that will be used by developers, such as an API, is fundamentally different than developing something that is designed for the masses, like a publicly facing web site. The distinction is important and can make all the difference.
We will discuss this more in the "Think Like a Developer" section later in the article.
Simplicity should be the order of the day. If your API is not intuitive to use, and you make developers spend a lot of time reading about your API instead of just getting to work using it, you are asking for problems. The "KISS Me" section below will address some strategies for keeping the API simple and easy to use.
The Duracell Approach
"It Just Has to Work." When a developer first wants to use your API, they should be able to, almost instantly, send a simple request and see that kind of information comes back. Based on the object and attributes requested, they should get a response that is reasonable. The nicest thing someone could say about your API is: it just works.
Don't Reinvent the Wheel
Sure, you have wonderful, smart, and highly creative developers. Your API is not the place to let them color outside the lines. If there is a standardized way of doing something, take full advantage of it. Even though some of the largest brands have provided APIs that are filled with one proprietary feature after another, it would be a mistake to do so with any "new" API development.
Remember, your target audience is the community of developers. It does not have time to learn a new technology just to engage your brand with their code.
Think Like a Developer
So, for the purposes of this article, let's say that thinking like a developer is, first of all, something desirable, and that it also means that you should design your APIs with the following several things in mind.
Speak To Me
Your APIs should speak to developers. When they use your API, it should feel like they are using something they would have been completely proud to have written themselves. Anything less than this is not only unacceptable, it may cause developers to seek out your competitor.
Thinking like a developer—imagine the typical use case for your API. Developer X wants to write an application/web site that helps consumers engage Megacorp, and either charge money for the application/web site or generate enough usage that advertisements can provide a reliable revenue stream. If they do not feel your API has been designed with them in mind, or is difficult to understand and use, do you think they will try to get you to change your API or find another company in your industry and see what their API looks like?
I understand that this logic doesn't exactly require excellence…you just have to provide an API that is better than others in your industry. Hopefully, I do not need to explain why providing an API as close to perfect as possible the first time, and every subsequent time, is high encouraged. Yes—I did just reference subsequent times. Your API is a living thing and should evolve over time.
So, can we all just agree that making something the best developers in the world would be proud to call their own is a sizable undertaking? Good. Now, how can you accomplish that daunting feat, every single time? Well, when it comes to APIs, all you need to know are three key design concepts.
Make your API simple and beautiful. No one wants to use an API with 20 different layers of complexity. We will provide more information below, in terms of specific design guidance, but for the purposes of this concept…just make it look pretty…or not ugly, if you prefer.
Using your API should be intuitive. Take a look at these two URLs, designed to return a listing of all products in the current Megacorp catalog:
Which one do you think developers will want to use? The one they can probably guess the URLs or the one that is more convoluted?
Also, please take note of the usage of the subdomain in the first URL. Consider it a best practice.
This should be self-explanatory, but you would be amazed how many times per week I have say this. If there is an industry standard, comply with it. Sure, some of the really big boys can do their own thing, but why add to the risk that the best and brightest developers will not want to engage your brand through the API, especially if you are fairly new to the concept? Better to start out being compliant and predictable with V1 than start off half-cocked and later, when you want to become compliant, it requires a fundamental change in the API design.
The military principle of "Keep It Simple, Stupid" should apply at all times during API design (although depending on your audience, replacing the last "S" with something other than "Stupid" might be a savvy career move). Remember, you want maximum usage of your APIs by the people who matter. That is not to say that you cannot enable highly complex usage of your APIs, allowing developers to get as granular as they want…but that should be in the "Advanced Concepts" section of your API's documentation.
Strive to make the "Hello World" example developers use to get up and running as simple as possible. One way I have heard this described is using the concept of a URL, with a query string. Move the complexity past the "?". That is, keep the root level of engaging your API simple, but allow for complexity, at the developer's discretion, not as something mandatory.
Let's look at the following URL:
As we used the simple URL referenced earlier to return ALL products, we can assume by looking at the query string that our API allows the developer to determine which fields they want returned, which field to sort on, the sort order and how many records to return.
All of these items are fairly useful in a "real world" usage of the API, where performance matters…but to just return a list of all data about all products, the developer can be up and running in just a minute or two using the base URL.
Many times in life, being shallow is not a good thing. As you could guess, API Design is not one of them. As you may have also guessed, I am talking about the "Base URL" portion of how folks will engage your brand through the API, not about the "Query String-Based" complexity mentioned earlier.
The guidance below is just one man's opinion, but I have been consulting Fortune 100 companies about their APIs for many years now, and at some point along that journey, my opinions have become part of the "ignore at your peril" club.
It is a good idea to keep the levels of your API to a maximum of five. Let's look again at our example from earlier, with all but one query string parameter removed and an additional level added in for "get one" functionality:
Level 1: Base URL for the API - https://api.megacorp.com
Level 2: Version - v2
Level 3: Object - products. If you end the URL here (with or without the query string), assuming a "GET" request, you will be executing a "Get All Products" query.
Level 4: ID - 1342077. Adding in this level (assuming a "GET" request, once again) allows you to execute a "Get the Product with ID = 1342077" query.
Level 5: Query String - fields=id,name,shortdesc,msrp,color,size. Depending on how much granular control you want to allow the developers, this is where almost all of the complexity can, and should, live.
We will be referencing a "Level 3" and "Level 4" URL moving forward to denote a "Get All" and "Get One" request.
HTTP Request Verbs
Using our general guidance of leaving most of the complexity to the right of the "?", we can not only use query string parameters to enhance our general Level 3 and Level 4 URLs but also include the HTTP Verbs in our request. If you think about all the ways an application might want to interact with one or more Products from Megacorp through the API, reference the following table.
Level 3 URL - Get All
Level 4 URL - Get One
|GET||List all products||Get one product|
|PUT||Bulk update products||Update one product, if exists; otherwise, error|
|POST||Create new product||Error|
|DELETE||Delete all products||Delete one product|
Admittedly, the table looks incomplete. For example, it might be nice to be able to manage products within the same category through the API. You are more than welcome to add URL layer(s) (/v2/products/categories/gloves-winter/), a Query String parameter (category=gloves-winter), or other piece of metadata that you require be transmitted with the HTTP request.
In fact, this might not be a great example, because if you wish to allow developers to access and work with products through their category, you probably do not need a reference to the parent "products" object—you can simply use "categories" as your Level 3 URL entity (/v2/categories/gloves-winter/). That kind of query will return details about all products that are in that category, including the ID. If you want to get details about a specific product, you can always use the ID and the Level 4 URL for a Product (/v2/products/1342077/).
So, you see, the sky really is the limit, in terms of the levers you want to allow developers to pull. But always remember, the simpler the better…especially in the Base URL.
You may notice that I have used PLURAL NOUNS as my Level 3 URL Entities. This is by design, but also by personal preference. Some folks will take a much more hard-line approach here. My only requirement for my clients is a draconian policy on consistency.
Use nouns or use verbs. Use singular or plural. But be consistent. The last thing a developer needs is to try to guess which entities have an "s" on the end of their Level 3 URL component and which ones do not.
As a plug for my personal preference, I just think it makes sense to use plural nouns. In our example:
- For Level 3 ("Get All") requests, you are returning a list of all Products, so the plural form makes more sense.
- For Level 4 ("Get One") requests, you are returning one of a list of all Products. I guess you could say that you are returning one Product, so the singular form is applicable, but I think it's always better to force developers to remember as little as possible…so the Level 4 requests should build on the Level 3, keeping the same form.
- As for the nouns vs. verbs debate, it just seems that almost all objects/entities in the application and database world are nouns…so why not use nouns to enable developers to work with your business entities?
This is a short discussion that builds on some earlier concepts. Don't reinvent the wheel and be compliant to widely accepted standards.
OAuth is the widely accepted standard right now. Use it. If something better or more widely used comes along, use that.
The following advanced concepts are presented at a high level to spur independent thought as part of your API Design journey.
There is almost nothing a developer hates more than a cryptic error message. Robust, descriptive error messages should be mandatory here. Developers will love you for it.
This is the official name of enabling any and all kinds of "Result Filters," either through Query String Parameters or through other metadata you may require as part of the HTTP Request. Once again, the sky is the limit here…but please do not make it part of your Base Functionality or your Base URLs.
Once again, you want to walk the line between making it as easy as possible to get started while also offering the flexibility for developers to get as granular or complex as they want.
Data and Metadata
You own your data. Determining what to allow developers to access is a similar conundrum to the filtering/partial response question.
Only you have the correct answer, and your data governance policies should rule the day here.
My only guidance is that no client of mine has ever come up to me after implementing an API and told me that they wish they had been LESS deliberate in determining how much data to expose to developers through the API.
Allowing "generic" search requests through your API is another topic worth entertaining as part of your design exercise. Depending on your technology maturity, the search scopes you have generally available and the expected volume of API Search requests, this can be a "Performance" discussion more than anything else.
Once again, be deliberate and intentional here. Remember, anything you put into a version of your API has to be darn near perfect. Better to "next release" something than run the risk of a self-inflicted black eye with the developers that are hopefully going to help expand your brand through the API.
In conclusion, I hope you have learned something that will help you as you enter the world of APIs or seek to make your APIs better. Remember, if you build it, they will come…but if you build it CORRECTLY, they will come BACK!
About the Author
Joe Mack is a Software Development Practice Manager for Sogeti USA and founder of ThisCloud.Rocks, as well as a recognized technology evangelist, thought leader and public speaker. He has been consulting, architecting solutions and writing code for companies big and small for almost 20 years, generally focusing on the Microsoft development stack.
If you would like Joe to speak to your company, event or group about Azure, Agile or APIs, contact him through his LinkedIn profile (http://linkedin.com/in/joemack) or through the public speaking page at ThisCloud.Rocks (http://www.thiscloud.rocks/speaking).