Portal Performance Planning, Page 2
Pointing Fingers Before Pinpointing Problems
This is an entire section that I consider as SOIDNTBM, yet the only projects I have not seen this occur on were projects where the entire team had done at least three previous projects together. A development manager I once worked with described the phenomenon as "developer's ego." Many developers believe that everything they write is so well done that the problem must be somewhere else. Anyone with a great deal of experience in debugging other people's code knows that it is most often the minority of developers who don't subscribe to this belief that generally produce the fewest bugs.
The finger-pointing problem is even worse between project sub-teams. Large projects generally have one team working on the web application and other teams developing service-layer APIs. All too often, whichever team discovers the performance issue will immediately contact the other team and demand they fix the problem without specifying even an approximation of the cause because they didn't even try to find out. Nothing could be more counter-productive. If a developer finds a bug (I will save the importance and general lack of thorough unit testing for another article), they should trace it down as far as they can. If it is a simple fix, it takes far less time to fix it than to pass it off to someone else to fix. It is productive, however, to gently let the creator of the bug know that it was found and how it was fixed as a learning experience.
Plan a Proper Portal Performance Platform
One technical team lead created a list of the Ten Most Common Developer Mistakes. Even though I don't recall them all, one that has always stuck with went something like "The environment you deploy to will never be the same you develop on." In the context of performance, this means that production is going to contain much larger data sets and a great many more concurrent sessions than you will have when building the code to support production. A good rule of thumb is to figure what the edge performance parameters will be in production and double it for your performance testing. If you don't have a performance testing environment (or one that is smaller than production), it is very important that you make it clear to all stake holders that you do not know how the application will perform in production. Yes, this is definitely SOIDNTBM, and yet I guarantee that there will be at least four major releases somewhere this year that fail in production due to performance issues that weren't tested for.
Avoid Aggregation Aggravation
Portals often bring disparate applications together to save time, money, or both. If these applications have never been used together before, it shouldn't be a huge surprise if they don't perform well together. Portlets that combine calls to multiple source systems that run beautifully individually can become a huge performance hit when called sequentially and then the results are evaluated for presentation. These portlets should be built early and tested under load to determine whether alternative designs may be necessary.
One alternative is to store data that doesn't change often in a local database. You may need to change synchronous calls to asynchronous calls running in individual threads. In some cases, using asynchronous rendering will be the best you can do. Especially when you have multiple portlets making individual calls that add up to a long page load time. When possible, work with the UI designers and Information Architects to spread these portlets across separate pages.
WebLogic Portal-Specific Approaches
As promised (or warned about), here are some approaches specific to WLP, though the first one has a corollary in any J2EE web application.
JSP compilation is an overhead. Sure, it only occurs once, but it is once per page, and if there are half a dozen portlets on a page, the first user there is going to be very disappointed.
I've seen some interesting solutions to the JSP compilation issue, including setting the ANT task to pre-compile (which the WebLogic Server sometimes ignores and rebuilds the pages anyway) and a servelet that runs once after deployment to load every JSP (it was named touchUrl).
The most effective soluition is to set pre-compilation in weblogic.xml as follows:
<wls:jsp-descriptor> <wls:keepgenerated>true</wls:keepgenerated> <wls:page-check-seconds>1</wls:page-check-seconds> <wls:precompile>true</wls:precompile> <wls:working-dir>/tmp</wls:working-dir> </wls:jsp-descriptor>
The working-dir node is optional and useful for debugging as the line numbers reported in the logs will refer to the compiled class rather than the raw JSP (note that /tmp is an arbitrary path and you can pick your own).
Design Re-Usable Controls
One little-known fact about recent versions of WLP is that controls go in session. This was one of the driving factors behind the solution that lead to the Developer.com article "Reusable Syndicated Media Portlets: An Example of Simplified Content Presentation." Large user sessions lead to poor performance (or increased profits for hardware vendors). The solution is to make your controls more re-usable. This doesn't mean to build your portal so that a single control runs everything (well, it might not be a bad idea for a really small portal used by lots of people, but I've never seen one in the wild). However, it is a good idea to write abstract controls and to combine obviously related forward methods into a single control. Remember, it is the portlet configuration that determines the first action called.
Another thing to remember about the control lifecycle being in the user session is that any objects you declare at a class level in the controller will live for the entire session. While this may be desirable occasionally, it would be more efficient and manageable to create a session object with a POJO to manage it and then reference the object through the POJO on a method level. The reasoning behind this is that if you really need it in the session, you should need it more than one control.
Use a Single Level Menu Where Possible
Another unexpected side effect in WLP is that the multi-level menu causes all of the controls reference by pages it contains to be loaded into the session the first time the menu is called. Multi-level menu loads all controls, so use single level where possible. Obviously, there are many situations where a multi-level menu is the best choice for navigation. The caveat here is to make sure it is the best choice to avoid the overhead involved in loading that first page.
SOIDNTBM. Read The [Fancy] Manuals. Capacity planning and performance tuning guides are available with each installation, and can (currently) be found at http://edocs.bea.com/.
As always, this is not an exhaustive discussion of all of the performance issues and preventive measures that can be taken for portal applications. One theme that ran through this article is that no matter how hard you try, you won't be able to cover every possible problem before QA, and sometimes not even before production. The best you can do is to plan for the ability to deal with performance issues and to avoid the approaches that you know from either experience or this article will most likely result in poor performance. Although much of this article is SOIDNTBM, I have spend an average of 10% of the debugging stage dealing with performance issues, and the idea for this article came from my editor because, well, it was SOIDNTBM.
About the Author
Scott Nelson is a Senior Principal Consultant with well over 10 years of experience designing, developing, and maintaining web-based applications for manufacturing, pharmaceutical, financial services, non-profit organizations, and real estate agencies for use by employees, customers, vendors, franchisees, executive management, and others who use a browser. He also blogs all of the funny emails forwarded to him at Frequently Unasked Questions.