Dither and Bicker

I've developed my share of network control panels and large scale CMSes, and along the way made plenty of mistakes. What follows is a few truisms I learned the hard way. I'm writing this down so hopefully I don't end up making these mistakes again.

Its pretty easy to find documentation on best practices for handling typical Content Management Systems (CMSs). They usually center on the content-creation portion of the process as that generally serves the needs of the numerous small level organizations and hobbyists. In fact most of the off-the-shelf CMSs tend to support content driven websites: blogs, galleries, simple storefronts. Drupal, Wordpress, Serendipity and their ilk are fairly sophisticated and mature platforms that can manage basic content delivery fairly well.

For large scale Network Control Panels, you tend to have components rather than frameworks. So you might encounter "extjs" or "dhtmlx", the custom libraries for jQuery or Prototype, charting software like Anychart or Fusioncharts, or even specialized languages like CakePHP, Adobe Flex or Google's Web Toolkit. I wouldn't be surprised to find out that many Control Panels included any of these components. While these are certainly a help, the core development is still an open-ended affair.

So what is a network control panel? For the purpose of this post I'll peg it as any web site or interface used to manage a company's operations. It may be used entirely by internal staff or also for customers. It may be a simple layer to browse a database, or a means to connect and manage multiple systems in various locations and data access methods.

Developing for network panels is a bit different from general web development. When building a public website you generally won't meet your end users and wind up targeting a generic and ill defined "audience". You need to worry about load (what if I get a million hits a day, can my site survive the Digg effect), and the additional problems of exposing much of the site functionality to everyone on the internet. By contrast, a Network Control panel will only serve a small group of people: employees at the company, maybe your customers.

Just because you're building for a smaller audience doesn't mean the quality of work will suffer. Unlike general web development you'll often have access to the end users; they may well be sitting twenty feet away from you. In my experience the more access you have to the actual users during the development process, the better chance your product will be successful.

From a security standpoint the smaller user set may give you additional tools: you might be able to restrict access based on IPs, you might require users to first be on the company's network before they can access the site. Also, all of those fancy user interface tools--drag and drop lists, dynamically sortable forms, animated drill-down charting components--tend to serve a useful purpose managing the complex data structures and processes exposed in a network control panel. By contrast, those same fancy interface components bog down general web sites, which seem best suited for simple image and text layouts (compare the effectiveness of Yahoo's current mail platform versus Gmail)--but that's a argument for another post.

Don't Fear Constraints

In this heady time of smart interfaces where there are such persons that could be considered "user interface experts", many project managers have fallen under the misconception that to add any constraint to the prospective users of their interface is entirely unacceptable. Unfortunately this thinking has crippled more projects than I could name.

All of the fancy persistence layers, specialty databases, behind-the-scenes user authentication systems are rarely as effective as instituting a few key constraints.

For example, say business wants to fancy up some of the pages with reports for your company's sales department. They want a salesman to be able to see the aggregate activity of all their clients up to the current minute. You look at the requirements, and realize that to show all activity would be an expensive operation--the client activity stretches back years, and the reports would join multiple tables in your database, etc...

In this case, it'd be perfectly reasonable to go back to the business owner and propose alternatives: "instead of showing the total history of all clients, why don't we constrain the results to the past thirty days? If the sales user wants to see past data let them request it explicitly through the interface" or "instead of up to the minute reports, why don't we provide dailies? That way we can cache and pre compile some of the common reports and dramatically improve run-time". In either alternative, the page will load faster and the data will be in a much more manageable state.

I've seen too many features fail because a developer (oftentimes myself) tries to give a business owner exactly what they ask for when the correct design choice would be to simply ask the user to sacrifice a bit.

Business Logic

Your operation will grow in complexity regardless of performance in the marketplace. Eventually you'll hear people say some form of "option x only really applies in situation y". At that point there will be pressure to add business logic to the Control Panel to streamline functionality.

Business logic is a little different than error conditions. Business logic is meant to ease the typical operations, to strip away redundant information from the interface.

Rules to remember:

• The control panel's goal is to allow employees to manage the system.
• Do not try to outsmart the users.
• Always allow users the ability to bypass the business logic.

Here's an example: say your company's expanding its global presence, and you're offering payments to your customers in a variety of currencies. Its determined that to save your staff time, when creating a new client the interface will set their currency to match their home country. By this example, French users will automatically get paid in euros, American users will be paid in dollars, pounds for the British, and so on.

But say you have a very good client in Canada that wants to be paid in American dollars. Even though this breaks the business logic, your interface should still allow to make such a setting. If clients aren't permitted to make such modifications then you should at least provide an admin-only interface to set the currency. If an engineer needs to make manual modifications in the database, your interface isn't functioning properly.

For the sake of discussion an error condition is data that the interface must prevent from saving. For instance a different spin on the above example, suppose there's a legal reason why you can only pay users in the currencies associated with their home countries. In that case the interface should be steadfast in preventing the Canadian user from receiving American funds.

These examples are pretty straightforward, but there may be a tendancy to try to outsmart your users. Phrases like 'the ghost in the machine' and 'the system has a mind of its own' are red flags that your process is too opaque to be trusted.

Don't Hide Functionality From Your Users

We live in the age of Apple and its button-phobic design strategy. I'm not going to bash the design principles of America's largest tech company, but you shouldn't assume that intuitive controls explain themselves. For years there's been a large market of "missing manuals" for Apple products, don't think anyone will lavish as much attention on your product.

In my personal experience business users tend to be cautious with network control panels, where flubbing a form submission can cost the company revenue. They also tend to follow the path of least resistance, which is to say if they can solve their problem by spending three minutes opening a support ticket and letting another department handle the process rather than spending fifteen minutes struggling with a cryptic interface, they'll pick the ticket every time.

To that end don't be afraid on including some manner of on screen instructions:

• If setting up a form try to include brief descriptions of what each field means.
• If a the field transforms data in some way let the user see the rationale (for example, if a field converts dollars to euros, show the exchange rate used in the computation).
• If there's error validation on a field, give an example of an acceptable entry (this is especially helpful for fields like social security or phone numbers which may or may not handle dashes).
• If the operation of the current form will produce other data objects, let the user know what they are and where to find them. For instance, if when creating a "company" a default user is also created.

In modern browsers there's many means to accomplish this including whether a hover event produces pop under windows, or through the regular old html title attribute.

The more users know about the overall process, the more likely they'll use the product to its full potential, and the more information the interface provides the less likely you'll be sought out to answer questions.

Let's take a variation on the currency example from the business logic section above and make it entirely cryptic. Say a fellow employee comes to you saying the "machine has a mind of its own" and for no apparent reason the currency for some users are being paid out in Canadian dollars and for others in American dollars. You dig through the code and find the system was behaving correctly: when new users are created a "default currency object" is created in a separate table. This currency object will look to whether the user was associated with a company, and if so it'll use the address of the company's headquarters to determine currency, if there's no associated company then the user's address will be used, and if the address is invalid then the default is American dollars. And on and on. In this way well intentioned systems become untrustworthy and unused.

Success is Worse than Failure

There are few data structures that can keep up with sustained growth. The dataset will grow, and you should be prepared.

Look at the inline menus on the user pages. What happens when instead of 100 items, the list swells to 1,000? How will that affect navigation, load times or memory usage?

There are stock ways to handle this scenario:

• Make inline menus multi-paged.
• Lazy load of additional items--only load the first n results, and make the user specifically request new information. This is similar to a "multipaged" solution, but this functionality can be invisible to the user such that it doesn't require a page refresh or even a "loading bar pop-up". DHMTLX charts have an interesting implementation of this: their "grid" will only render items when the user scrolls to them in a list. The result is a grid that can load some 50,000 elements with great speed
• Only show a finite number of "top results" on a heavy traffic page, but provide users a link to another page with the full list. For example "Top ten entry points" in Google Analytics.

There's still another problem, if you're parsing tons of data, your company might decide to partition the dataset. Say at one point client transaction information spanning from the dawn of the company to present was stored in a single database accessible to your interface, but dataset started to grow exponentially and in order to maintain the system, only the past x days of data are now kept in the database. After partitioning, it may not be possible to display all data over a long rang in the same formation, but this should not be seen as a fatal flaw. Most banking sites I've used only keep the past three months of specific transaction data. If you want anything later, they provide a pdf copy of your statement. To me this is an elegant solution: it's easier to store and index pdfs than to manage a public interface that would allow millions of users the ability to search trillions of transactions.

But somewhat contrary to my arguments in this section, you shouldn't fall into the trap of optimizing everything with the expectation that it will all expand exponentially in short order. Its a safe bet that certain things--your db logs, certain reports--will grow rapidly and consistently, but other items may not be grow at the same rate, if ever. At the same time, its hard to anticipate how some of the internal data structures will expand, whether they'll gain fields and properties, sub classes, or be broken into separate entities.

With this in mind, you shouldn't be afraid of practicing a bit of simplicity. Some forms and data management components can be built quick and dirty, if for no better reason than you're unable to predict whether or how it might grow. There have been occaisions where I've invested too much time building smart interfaces designed to handle tons of complex data structures, only to find in the natural course of the system's lifespan these data structures changed so much that my fancy interface became terrible to use. Likewise, I've seen developers become enamored with boutique databases, clusters, cloud computing solutions, and all manner of web services to support datasets that didn't grow fast enough to warrant such things.

I guess in other words: its okay to be very simple in the earlier versions of your Network Control Panel and sometimes its okay to let problems arise before you implement solutions.

Set Up Logs

Mystery has its place, but not in any product tied to your company's well being. People are more likely to believe a system failed than accept the possibility of human error, and you'll need to be able to make the case for your system at any given point.

For the most part, I like to set up at least two levels of logging. One level writing to flat files on the server to act as access logs, or to file verbose custom errors. The other level would ideally be kept in a database, easily sortable with a unique key field that can be distributed to users and customers.

As a rule of thumb, all financial transactions should be put in a database. Imagine the scenario where a client swears your interface lost their money. You want to give your support staff every possible tool to follow the money.

Going hand and hand with this is the necessity to enforce unique user log ins. Your logging apparatus may need to act as evidence for some serious trangressions whether against external clients or fellow employees.

Use it to Prove it

When implementing a network control panel, you're going to spend a fair amount of time proving the accuracy of the information it provides. The timeline will go like this:

1. Initial bugs associated with release
2. Questions when you introduce the software to the company
3. Adding new features or dealing with features added to other software that shares the backend dataset.

To be fair, if your own bugs weren't enough, you probably won't be even notified about all the changes made to the codebase within the corporation.

Final point here, plan for your control panel to "speak on its own behalf" and provide the functionality that will allow its users to answer every question that may arise. The best way to do so is to become the biggest booster of your work. You should try to use your network control panel like any other user would for every case applicable. If there are questions that can't be answered through your interface, you should work with the business owners to expand functionality to handle those cases.