Are you Using Caching As a Bandaid?

When your web application is responding slowly, there are several levels at which the problem could be occurring. Until you instrument the application and begin the diagnostic journey, you really don’t know whether the problem is.

It is often difficult to convince business stakeholders to allocate budget to uncover and fix an undefined problem, rather than investing in customer-facing features and enhancements. And so the front line attack on the problem is often to simply apply caching; a particularly easy solution with most modern application frameworks providing either integrated caching or easily applied solutions via 3rd party plugins or modules.

So let’s just assume that the first line of defense is going to applying a cache bandaid. Fortunately many of the caching frameworks provide multiple layers of caching and by applying caching at the lowest levels and gradually stepping up through the application, this can give a relatively easy opportunity to understand at least what level the issue is occurring at.  And this information can be useful in convincing stakeholders of the need to dig deeper.

i. DB Objects
One of the most common problems with slow applications, are slow database queries.  Either because of complex queries or poor database architecture, there is the potential to exhaust the database connection pool because each connection needs to hold open and isn’t returned back to the pool quickly enough to manage the current traffic load.  That doesn’t mean you need more hardware necessarily. But it does mean you need to look more closely at the database queries, the structure and the indexing of the database.

ii. Code Objects
The next level is to cache programmatic objects. This might be helpful if you’re holding objects open while either waiting to populate them or structure them.  Structuring complex objects can sometimes be an issue when dealing with large complex XML objects for example. More often though, it is about delays in fetching data from external services buses or web services, where the routine is waiting for this data so that it can popular the object.  By caching these objects, you avoid ever having to instantiate, structure, or populate these objects.

iii. Page Caching
Moving another level up, you can cache the entire page requested by the user.  In this case, a copy of the output stream data (HTML) is kept either on the local file server, or in RAM with more aggressive caching.  Inbound page requests are intercepted, existence of a copy of the output stream in cache is first evaluated and the user is provided this completed object if its available.  if not, it proceeds to construct the response to the query, save a copy into cache and return that to the user. By doing this, you’ve cached and avoided all of the underlying processing for that page request.

If this works to resolve latency whereas DB and object caching did not, this likely indicates  inefficient code somewhere in the logic of that processing. For example, I once worked on a project in which a junior developer had written a database query that generated over 100 results, and subsequent database queries were being run for each of those results. So rather than running a single complex database query, the code ran 101 queries, and possibly more, for a single page. Clearly this was a problem.

iv. Content Delivery Network
If you’re working on a larger application and have access to a CDN to offload serving your assets, that’s a huge advantage.  If your site is media intensive, you could be serving a lot of heaving images and/or video.  If could be that perceived slowness of your application actually has nothing to do with your application, but rather has to do with network latency in returning these assets, if your server is far away from the user. CDNs solve this problem by mirroring these heavier assets on a network of servers throughout your targeted geography, and this can significantly improve the speed of these assets, not to mention freeing up HTTP server threads that may be hanging open too long and causing a bottleneck and queuing user responses.

If you’ve walked through each of these steps sequentially, you should now have a pretty good sense of whether you’re dealing with issues related to the database, inefficient database queries, latency in web services or other external system integrations, inefficient code in your business logic, or network latency related to delivery of your media assets.  Since you likely know the application fairly well, this should hopefully be enough to trigger a few thoughts and theories of what might be the problem and what to look at next.

Custom Development Costs Too High?

Small business has entirely different needs and expectations from technology than larger enterprise.  Okay, that is an obvious statement, right?  So then why are small businesses always so surprised to learn about the cost associated with custom projects?



First, a Little Background

There was a time 10 years ago, when the company I worked for would build websites for mid-size companies, and we could never quote less than $15-20k for a simple website.  In today’s world of WordPress and 1000s of templates to choose from at $30 a piece, that seems almost unbelievable but remember, frameworks like WordPress didn’t exist and neither did the templates, or even best practices for design in the early days.  As a result, you had to do pretty much everything by hand. You would write every function, connect it to the database, hand-code the HTML, and equally the designs were mostly original work that took time and research.  And so yeah, all-in, creating a WordPress equivalent website from scratch would have easily required 3-4 weeks of developer time, 40 hours from a designer, and another 40 hours for project management and requirements gathering.  All said, you’re talking about 6 weeks of work!

Fast forward 10 years and there are numerous companies offering to build a website for $500, or less!  By using an open source framework, free plugins, and by customizing an existing template for your design rather than starting from scratch, the website developer really only needs to snap peices together now – the pieces are built already, making it a relatively easy project to put together in days or even hours, rather than weeks.  No worries really about bugs either, since a lot of the logic is already written and revised from other projects.   Effectively, those problems have been solved now and we have commodity solutions to those problems.

Expectations Mismatch

The availability of lower-cost solutions is a great thing, but it creates a mindset of cost expectation that often isn’t realistic when you begin to look at custom coding work.  For example, even just adding a functional interface with JavaScript and AJAX to optimize user experience or validate a form, might not already exist and will require custom development.  On the surface this statement seems reasonable to a developer, but what if it is 8 hours of work and costs as much as the initial website itself? Will the client/customer still find this reasonable?

As businesses go deeper into custom work, there are fewer templates and fewer frameworks and technology stacks upon which they can stand to accomplish their task and as a result costs and time requirements will begin to scale exponentially. There is afterall, a reason why employs 1000s of people and yet you can have your own ecommerce site built in a day.  This dichotomy seems to always be an issue for the non-technical person to accept, not just at the small business level, but any time a businesses tries to go a layer deeper.  I’ve seen the same issue among mid-size businesses as well, when looking at going into deeper systems integration work.  One company in particular  I observed, went from using PHP/LAMP technology for their website in,  to deciding it was time to go deeper. They embraced an enterprise Java stack to begin developing the business logic and work on other systems integration work, and were stunned to realize their time estimates were less than a third of what was now required to do the work.

Development Value Pyramid

Lessons to be Learned

A lesson here, is both for developers to realize the paradigm that businesses are coming from when their expectations are challenged, and also for business owners to understand the reality behind why costs scale exponentially as we go deeper into systems.   When a business owner is use to $500 price points, a $5-20k price tag for the completely custom project they asked for is going to be scary and will invoke trust issues.  But that doesn’t mean the developer should work for less to appease a client, or that they’re necessarily wrong for proposing a higher cost for what the client asked for either.  The problem to be solved, is one of expectation management.

Advice For Developers

For developers, this is going to be a larger challenge the higher up on the pyramid you go, where the most commoditization has occurred.  If you’re trying to operate one layer below templated solutions, you’re going to have the biggest challenge convincing clients that you should be charging so much more, if they cannot see tangible value as a result, compared to just using a template.  In this case, you’d better align yourself with clients who have a lot of money to spend on branding design, rather than technology.  And if you’re offering customized app or interface development solutions a layer down, you’re going to see more and more frameworks trivialize what you do over time, and push you further toward commodization.

Indeed, it would seem a developer is better off specializing as low as possible on the pyramid, if they’re going to offer custom solutions.  Businesses however will continue to find opportunity to push value further up the pyramid and further commoditize and productize the solutions for the small business consumer.  This is why the SaaS business model is so poignant, since it promises to make available solutions that have real value for smaller businesses, who could never afford them before. So if there is any lesson to be learned here at the bottom line for developers, perhaps it is to be realistic in understanding who your audience is, and who is willing to pay for custom work versus a customized commodity. If you’re going to address the commodity audience, find a away to productize and scale it as a business.  But if you’re going to focus on skills and custom work, go as deep into the value pyramid as possible…and stay there!

Do Things That Don’t Scale!

Paul Graham of Y Combinator fame once gave the advice to “do things that don’t scale”.  That advice was given to the founders in the first YC cohort.  I heard this while I was watching the interview with the founders on

I thought this was pretty interesting advice.  I think the logic here is that you want to go where the competition is not, and there is just so much money flowing through the venture capital funds right now, that just about any SaaS project run by a 20 year old out of Y Combinator or Tech Stars is going to end up with $500k to go build it.  And so if you’re not one of the funded, you need to be realistic and stay out of the way.

An analogy that occurred to me while driving the other day is that the “big money” drive large trucks on the Interstate, whereas the unfunded garage startup is like a moped.  Sure, the moped can get you where you are going, but not nearly as quickly and you’d better stay off the freeway or you’re going to get killed!

So for this reason, I guess it is pretty smart advice.  So then I guess the next question is, what doesn’t scale?  I always hear that VCs won’t fund consulting companies becuase they don’t scale.  Perhaps this is why so many solo-entrepreneurs end up starting consultancies and agencies (services businesses) instead of products.  Perhaps its simply adaptive in that way.


Systemetize Your Intelligence

One of the biggest criticisms I hear about services businesses is that they are not scalable.  Profit margins are low, clients are demanding, and you are highly dependent upon high-cost employees with specialized skills, who could leave at any time.  In fact, this is the way most small consulting companies run.  But it doesn’t have to be this way.

There is a quote that I have heard attributed to Ray Kroc in the past, though I cannot find reference to it now.  The quote is something like this: “Build a business in which the intelligence is in your systems, not your people”.   Let’s take the restaurant industry as an example.  If you subdivide the various tasks and automated as much as you can, you don’t need a team of Michelin chefs to run your restaurant.  In the case of McDonalds, you could have a moderately paid manager and a small army of teenagers doing the work.  Sure, you may have that high-end chef help you to define the “program” and oversee it, but you do not need n-number of them to scale the business or operate day by day.

The problem with a typical consulting company is that the engagements are too open-ended, meaning there is constant inefficiency and you try to cover special case scenarios, and the business cannot systematize their own operations sufficiently.  At first this is okay because your hourly rates are high and clients tolerate the inefficiencies since there is no other choice, but eventually those skills become commoditized, the client begins demanding more for less, and you’re left still paying for the high-end talent and seeing your margins compress.

Instead, imagine taking the opposite approach.  Taking something that is already nearly commoditized, and building value around it.  What value can you offer to augment the commodity?  Apple did this well by turning commodity technology into a work of art, that happened to be a computer for example. What they’ve essentially done is spin gold out of straw.  They’ve taken a commodity, added value all around it, “programitized” their efforts and now it is scalable and controllable.

Let’s take Starbucks as another case.  They basically are just selling coffee – a commodity.  But they created a hook in that it is a great place to hang out.  They added value by making it a gourmet product (lattes with infinite choices) and nice cushy chairs.  They invested heavily into their value chain to make sure they could reliably and efficiently reproduce the experience and provide the product to the consumer.  And because they made the investments into consistently replicating the experience and product, they were able to scale worldwide with relatively low-priced and easily replaced workers.

Of course Apple and Starbucks are essentially product companies, but Starbucks in particular is partly a services company.  There are other examples that lean toward services more heavily like H&R block for taxes, and GeekSquad for computer repair. In both these instances, they provide relatively high value to customers at relatively low costs. Because knowledge is systematized within the organization, training costs are low, and the value proposition of the company is otherwise contained within the value chain of the business, not in the skills of a few all-star team members.