Web 3.0 Has Already Begun

Web 3.0

This article was originally published in ACM Interactions Journal, Sept 2013.

A lot has changed since the release of Apple’s first iPhone in 2007. We have witnessed a profound shift in user behavior, away from desktop computers in favor of new form-factor devices. The enabling technology has also brought about an entirely new class of Web-enabled applications and architectural ideals. Meanwhile, monetization opportunities that capitalize on these changes are appearing. It is from a confluence of these changes that we assert Web 3.0 has begun.

We will support this assertion by looking at the types of changes that occurred during the Web’s previous era to support labeling that Web 2.0. We then apply those criteria to show how comparable changes are once again occurring, thus justifying the acknowledgement of an entirely new generation of advancement on the Web. Let’s start with a close look at Web 2.0.

In the opening remarks of O’Reilly’s first Web 2.0 conference in 2004, Tim O’Reilly and John Battelle defined Web 2.0 as “Web as a platform,” in which software would be built upon the Web rather than as a desktop application [1]. In response to this idea, Jesse James Garrett wrote an essay a year later in which he coined the term AJAX (Asynchronous JavaScript and XML), popularizing a client-side technique that became accepted best practice for implementing O’Reilly’s vision of the Web as a platform.

Concurrent to these technological revelations, use patterns online were also changing. With the rising popularity of social sites such as Friendster and MySpace, users were starting to contribute to the Web’s content, in contrast to previous, more passive use patterns in the Web 1.0 era. Blogging was also gaining in popularity, driven by the introduction of WordPress and Google’s popular AdSense network in 2003. Eventually this explosion of social sharing via profiles, articles, comments, and conversations would culminate in what Facebook would later, in 2007, call the social graph.

Interestingly, there was no single innovation that completely represents Web 2.0, yet most agree it was a pivotal time. Reflecting upon the above statements, we identify three significant forces that converged around 2004. Specifically, there was a convergence of new technologies, new user behavior, and new monetization opportunities. Table 1 compares the three generations of the Web from this multifaceted convergence.

web3-table

To make the case that Web 3.0 has begun, let’s discuss the enabling technology along with new use patterns and monetization factors that have arisen in the past few years.

Emerging Technology

Whereas Web 2.0 could be technologically characterized by the proliferation of Web services, one could argue that Web 3.0 is characterized by a proliferation of new form-factor devices, leading to new use patterns of the Web. In 2007 Apple introduced the iPhone, which would become the transformative device of its generation. In 2010 Apple released the iPad, which at a technical level is merely a larger version of the iPhone, but which evolved its own use patterns when introduced to consumers.

Concurrent to these activities, Google had been developing Android, which provides an open source alternative to Apple’s higher-priced devices. Hardware developers such as Samsung and HTC have developed popular lower-cost devices that leverage Android and provide a significantly lower-cost commodity alternative, which is helping to drive quicker adoption of these new form-factor devices. In fact, the rate of adoption for these devices is occurring so quickly that sales of them are now higher than that of traditional desktop computers.

There is significant anticipation among technologists and investors alike that a fourth new form factor will soon take hold, the Internet television. Many in the media are anticipating the release of such a device later this year, pending the resolution of licensing issues with the pertinent media companies. In anticipation of this, venture capitalists such as Marc Suster, who is investing in Los Angeles-based entertainment companies like Maker Studios, believe that television is ripe for innovative disruption. If they are right, this could emerge as yet another significant new form-factor device, representing another use pattern.

Another hallmark of a new Web era is the type of applications running on these new form-factor devices. Unlike previous Web applications, which were typically thin-client applications, we are seeing the reemergence of thick-client architecture for specific-use apps. In this context, rather than loading an HTML interface from a remote server, a device launches a native application that runs locally, providing a significantly improved user experience. The native app still utilizes stateless remote Web services, but state persistence and intelligence about location and user interface behavior—the true application logic—occur within the local native app.

The mobile Web is also making strides toward thicker-client applications. HTML5 has standardized the implementation of geolocation awareness and local persistence storage, enabling an HTML-based document to now manage its own session state and geo awareness, rather than needing to defer to the server to manage application-level functionality. The inclusion of the vector canvas further enables HTML5-based applications to create rich user interfaces and data representations without needing to call back to the server.

Microsoft’s new Windows 8 operating system takes notice of the power of HTML5 and now allows developers to create first-class HTML5/JavaScript-based Metro panel apps. This reflects the rise of technologies such as PhoneGap (Apache Cordova), which enable the compiling of native apps written with Web-standard HTML5 and JavaScript technology. Regardless how it is built, the trend is still the same—a thick-client application that relies upon thin stateless Web services, a clear departure from the thin-client architectural idealisms of only a few years ago.

Another significant technology that has influenced this new generation of Web use is the geolocation ability of the devices. This technology has given way to an entirely new class of applications that has substantially influenced use patterns and emerging monetization opportunities.

When cellphones began to include GPS capabilities in favor of the older triangulation method, it dramatically improved the accuracy of location awareness and opened the potential for real-time location updates being used for commercial purposes. When the iPhone introduced a meaningful software platform along with this potential, an explosion of location-aware software applications could begin.

When HTML5 was introduced, it included a provision for a geolocation API that Web browsers needed to implement, giving even HTML5-based applications the ability to be location aware. Most commonly, services such as Skyhook Wireless implement this by looking up nearby wireless access points and cross-referencing them for location data. It is also suspected that Google collected such geo-referenced Wi-Fi data while driving its cars around to take photos for the Street View service, thus providing another source for geolocation lookup.

New Use Patterns

As a result of the massively disruptive innovations that have taken place with emerging technology, users are no longer bound to their desktops in order to use the Web; the online advertising industry has started referring to this phenomenon as the “four-screen user” [3]. They are looking for opportunities to stitch together the entire user experience across numerous devices, both synchronously and asynchronously.

We continue to see increasing splintering of activities across our four devices, commensurate with the nature of those activities. Productive work and research are still primarily performed at a desktop device, but our phone is now frequently used for looking up location-based information, such as maps to nearby stores or houses for sale. When the iPad was launched in 2010, it introduced a new form factor that immediately appealed to the household for casual Web surfing, sharing photos, and other activities characteristically performed from the couch. And of course there is the much-anticipated Internet television, which seems poised for imminent release.

The introduction of new form factors surely will not end with the completion of Apple’s i-series of products, either. There are many apparent extensions of this proliferation of Web activity further into applied devices. Products have already been introduced that enable control of your home’s heating and lighting from a Web application, and industry thought leaders have begun talking about Web-enabled machines that are self-aware of efficiency and the need for repair.

Another meaningful trend we have observed in the past five years is the growth of context-specific applications. What arguably started with API mashups in Web 2.0 has matured into very useful apps that leverage specific meaningful segments of the Web for a specific purpose. For example, one click on the weather app on your mobile phone or tablet and you can see information about today’s weather, without ever needing to open a Web browser or enter your location. Or you can get directions to a destination based on your current location.

Beyond location, applications are learning our preferences and adapting accordingly. Google, for example, has begun factoring our search preferences after observing search results we click and recommendations made by others we are socially connected to via our Google+ accounts [4]. While this is a natural evolution in its service at some level, the fact that we’re using devices that maintain state for us means we no longer need to log in to those applications when we navigate to them. Instead, we merely click the search button on our device and the personalization of our results happens automatically.

Apple’s introduction of Siri with the release of the iPhone 4S was an interesting increment as well toward Tim Berners-Lee’s vision of intelligent machines that understand the Web and perform actions on our behalf. It is a spin-out from the SRI International Artificial Intelligence Center, and is an offshoot of the DARPAfunded CALO project. After you issue a verbal command to the phone, Siri will process your request and respond by either answering a question or taking a requested action, such as initiating a phone call, setting an appointment on your calendar, or playing music.

Enabling Monetization

Perhaps what has driven the rapid innovations of the Web more than anything else is the potential of generating profit. In Web 2.0 this was manifested by well-funded ventures seeking to capture a market share in the social ecosystem, which would enable monetization on a large scale as a media portal, as well as bloggers figuring out how to monetize their own content through Google’s AdSense network.

The acceptance of “Web as platform” also unlocked the potential for subscription-based online software as a viable alternative to the up-front licensing of a traditional software product. And indeed, those three areas where new monetization opportunities were present are where the majority of the innovation occurred during Web 2.0. In Web 3.0, we see an entirely new crop of monetization opportunities emerging, as we will discuss here.

The app store is a new monetization vehicle that has arisen in the past five years and has fueled much of the development of mobile devices. The Apple App Store began a new era of monetizing software as a product. It allows third-party developers to create extensions (apps) that take advantage of Apple’s platform and sell them into a marketplace that is prominently positioned for the users of the platform. The concept has been profoundly successful; the store now boasts more than 900,000 apps.

The idea of an app store has caught on with competitors and in other verticals as well. Google Play Store is the analog for the Android ecosystem (and now offers as many apps as the Apple App Store), and Microsoft recently launched its own app store for Windows 8. For B2B platforms, Magento has the Connect marketplace to extend its ecommerce platform; SalesForce introduced Force.com, facilitating extensions to its core cloud services by third-party developers.

In an effort to provide “frictionless” checkout opportunities for customers, physical stores are beginning to leverage mobile-based solutions to enable easier checkout. Companies like Starbucks are now offering the ability to make a purchase by simply tapping “pay here” or by scanning a QR code displayed at the point of sale. The system works with Square’s eWallet program, which links directly with the customer’s credit or debit account.

It is an interim solution, however, that companies like Square aim to improve. In an interview with CNN, Jack Dorsey, CEO of Square, said that their plan is to eventually not even require the user to take their phone out of their pocket. The user would merely say, “I’m Laurie, and I’d like a cappuccino,” the proximity of the device would be detected, and her account would be charged in the background [5].

The one issue remaining for mobile commerce is how to simplify making purchases online, since the small screen and lack of keyboard make it prohibitively difficult to complete a long checkout form. Though this remains a problem at the time of this writing, a solution will inevitably come as a result of this foray into mobile commerce.

The increasing amount of time that Web users spend on new form-factor devices is creating a challenge for content creators, who are finding that their content does not monetize as well through these devices. According to Mary Meeker of Silicon Valley-based venture capital firm Kleiner Perkins, the effective CPM (cost per 1,000 impressions) for mobile devices is five times lower than that of the desktop [6]. Not all devices are created equal, however, with tablets generally delivering higher eCPM rates than phones. Nonetheless, the point remains that the ad-banner and paid-click advertising models may not best fit the use patterns of these new mobile devices. Where one window of opportunity closes, however, another opens.

Geo IP fencing is one such opportunity that takes advantage of hyper-local targeting to serve ads relevant to your physical context. Whereas Google had success with AdWords advertising by dynamically placing ads into contextually relevant articles, GEO IP fencing enables the advertiser to specify a physical boundary within which an advertisement will appear, thus enabling ads to appear only when they are geographically relevant. For example, imagine a cafe offering 10 percent off a morning coffee if you come in now. This ad could be shown within 100 yards of its store, enticing pedestrians to stop in and take advantage of the offer.

Conclusion

It has been only six years since the iPhone was first released, which set into motion many of the changes we have discussed. It is difficult to grasp the extent to which the Web has changed in such a short period of time. When reflecting upon all of this change, one has to wonder if it is sufficient to declare that an entirely new generation of Web innovation has occurred since Web 2.0, and so we sought to answer that question here.

Through reviewing the innovations most commonly associated with Web 2.0, it appears that what defines that generation really comes down to a convergence of three things: new enabling technology, enhanced monetization opportunities, and consequently new categories of products that resulted in new use patterns for end-users. By evaluating these criteria, we observed that indeed a significant transformation has begun that could be described as a generational advancement. And so we conclude, affirmatively, that Web 3.0 has already begun.

Authors

Neal Cabage (nealcabage.com) is a digital product strategist, technologist, and author who has spent years leading the development of online products. He has worked with top online brands, spoken at leading industry conferences, and founded and sold two online startups. He co-authored The Smarter Startup (Pearson New Riders, 2013) and is a contributing columnist for Inc.com.

Sonya Zhang (sonyazhang.com) is a professor at the College of Business Administration, Cal Poly Pomona. She holds a Ph.D. in information systems and technology, an M.S. in computer science, and an M.B.A. Her research focuses on Web development and optimization, eCommerce, and Internet entrepreneurship. She has published in top journals and conferences, as well as co-authored The Smarter Startup.

Web 3.0 Has Already Begun!

Web 3.0 has begun and its not what you think.  I don’t think many people realized what Web 2.0 was until we were at the height of it, and already seeing the capabilities that had been set out years before it. But imagine if we had all realized in 2001 what was possible and all of the major social and technical shifts that would rise the an explosion of innovation and opportunity by 2006.

First let’s start by defining Web 2.0.   Depending on who you ask, you would get different answers.  A marketer would tell you it is about the User Generated Content (UGC).  A technologist would tell you it is all about AJAX and the API mashups that AJAX enabled.  A designer would tell you it was the new simplified aesthetic that focused on conversions and pragmatism.  They’re all correct, but let me distill this into a concise list:

  • Use Case:  Socialized content.
  • Technology:  AJAX (JavaScript) and Mashups
  • User Experience: Fewer page refreshes. Simplified design.

To say web 3.0 has begun, we’d need parallel impact on all these fronts, so let’s start there:

From the use case perspective, consider the semantic and meta tagging that are now a part of the HTML5 spec.  Semantic web promises to transform the web into an ultimately connected experience in which a machine has as much awareness of the content as a human.  Imagine your calendar warning you there is a conflict prior to booking a ticket on Expedia.  This is equally if not more significant to the social revolution of web 2.0.

As far as technology goes, HTML5 solves two major problems that will lead to a major revolution of web application architecture, moving in the same direction of Web 2.0.  First, no more limitations preventing cross-domain AJAX calls. That has huge implications for mashups!  Second, a robust local storage facility that can be used for JSON (serialized JavaScript objects) document storage locally on a device and will substantially help overcome the stateless persistence issues that have plagued web applications since the beginning. This is so significant in fact, that there is already a movement toward smaller event-driven server-side technologies such as Node.js and NOSQL document databases such as Couch and Mango, which are perfect for JavaScript object storage, in acknowledgement of the shift of dominance toward the client.  Think fat client applications written 100% in JavaScript.  The potential is already there and buzzing quietly under the surface. Node.js already has as many followers on GitHub as Rails (for Ruby)!

Finally, user experience.  This one can be summed up in two words – interactive media.  Finally with HTML5 and the proliferation of cheap bandwidth, it seems the pieces are in place for the much-anticipated online media revolution.  There are technologies already available and built upon HTML5 that enable bi-directional triggering and interaction between the HTML Document and the embedded media. Imagine you are a media content producer and you can embed triggers into your video that will instigate social interaction and information widgets at correct times in the media.  Simply amazing when you think about what might be possible here.

Putting it all together, the potential is there for a much larger wave of technological and cultural innovation now, than at the beginning of Web 2.0.  Not only is this significant enough to be compared to Web 2.0; its bigger!  And if you consider Roger’s Innovation Adoption Curve, NOW is the time for entrepreneurs and technologists to begin creating opportunities around these possibilities.  Don’t wait until we’re already 1/2 way through the innovation cycle and the retrospective innovation has become obvious.  By then, it is too late.

Website Design for Tablets and Mobile

Recently I went through a redesign, and as part of the effort, I reviewed how best to support tablets and mobile visitors. If you haven’t been researching this topic recently, no one could blame you for dismissing mobile/tablets as a novelty that won’t have much impact on your web presence. But if that’s where you’re at, consider this:

It was recently estimated (as of early 2012), that 10% of all Internet traffic is now coming from mobile devices and mobile traffic will be as high as 36% by 2016. And compared to this time last year, mobile traffic is up 131% … in a single year. We are clearly in a secular trend toward personal mobility in computing. Proliferation of mobile devices now is accelerating and sales of internet enabled mobile and tablet devices now exceeds that of traditional desktop devices; tablets alone will exceed desktop sales as early as next year!

So there is a compelling need to support mobile and tablet users with any new web design moving forward. But how do you do that? First let’s talk about our objectives and we can look at a couple possible strategies from there:

Objectives:

Consider the use pattern of these devices. When using a desktop computer, they’re typically at the office and either researching work-related information, or procrastinating between projects (yes, ahem, at work). Tablet users meanwhile are typically sitting on the couch in front of the television during evenings and weekends. Mobile users meanwhile are trying to squeeze in activity while waiting in the doctors office or at a red light in their car (sad to say).

So what is going to be a meaningful and satisfying user experience for these three users? The desktop and tablet users are going to be more receptive to alternative suggestions for content (upsell eCommerce and related posts links, etc). Tablet users will enjoy media and rich images more. And all of this rich user experience is actually an impediment for mobile users who are fighting against time constraints, a small screen, and slow load times. They just want to get to the critical information such as contact information as quickly as possible. So sticking with the traditional web design only, which is geared for the desktop user, is not going to well represent the other two new classes of users (table & mobile) that you need to start thinking about.

To facilitate these new users, there are a few options:

Native Application:

The first thing that comes to most people’s minds with mobile content, are the native applications for iOS and Android that you download and install on your mobile and tablet devices. These provide very rich user experiences but require that you develop a separate application for each device that you want to support. That can get expensive. And let’s be honest, how many websites are users going to want to take the effort to download an app for? So does all that extra expense justify developing for a use case that users likely won’t even engage? There are exceptions to this rule of course. If you’re a gaming company or you are a cloud software application that provides a productivity tool that the user will need to access on a regular basis, then a native app is certainly a possibility and probably even a superior user experience. For the majority for corporate and marketing websites though, this just isn’t a reasonable option, since a user is merely looking for some quick information about your site.

Separate Mobile Design:

The next reasonable consideration is creating a separate website to support tablets and mobile phones. This is relatively easy to accomplish. It is relatively straightforward to detect the device type (e.g. HTTPUserAgent) and redirect users to alternative content accordingly. If you use your phone to browse web content frequently, you’ve likely come across a few sites that will redirect you to a separate subdomain of their website. xyz.com for example might redirect you to m.xyz.com. And there you serve the simplified version of the website that better applies to mobile devices. But this gets complicated when you begin to consider all the various permutations. What about tablets for example that actually benefit from richer user experience?

Responsive Web Design:

There is a quiet movement in the design community that began with an article by Ethan Marcotte, describing the possibility of Responsive Web Design (RWD). The idea provides a technical framework for implementing a “mobile first” web design, in which you start by designing the constrained mobile experience and layer on the richer user experience based on capability from there. By working backwards, you ensure you’ve accounted for all the various permutations such as all the different screen sizes (media ports), landscape versus portrait layout, etc.

Responsive Web design is possible primarily because of the introduction of media queries with the new stylesheets standard in CSS3. Media queries enable the website author to encapsulate CSS classes within conditional statements that evaluate the media type of the user such as window size (viewport) and orientation. RWD also addresses use of flexible design grids and context-aware images that adapt the size and quality of the image also based upon device type.

A Hybrid Approach:

After evaluating these different options, I ultimately concluded a hybrid approach was in order. The philosophy of responsive web design is elegant, I naturally gravitate to it, and it worked wonders for optimizing the desktop version of the website for tablet devices. At the end of the day though, I did not feel it was appropriate for mobile devices. As mentioned earlier in this post, mobile users have fundamentally different goals and needs, and are simply not looking for a more elegant representation of my website; they want to get the information and get it quickly, period. I think about my mobile experiences and when I look up a business online, its typically because I just want to find their address or contact information. Thus, there is a lot of information on the primary website that does not belong there for the mobile version, and it really should be treated separately.

An Example:

If you check out Enlogica.com on your smart phone, you’ll see that there are only 5 pages and the first is the contact link. If you click on this, it is a very simple page with the address and three buttons that allow the user to: click to call, click to email, and click to get directions on their phone. That’s it. There are a few other pages below that for the sake of completeness (the blog section is accessible at the bottom), but I’m really trying to create a simplified user experience, based upon the assumed use pattern of the user. And this is fundamentally a different prerogative than Responsive web design, which is focused on adaptive aesthetics, not user experience.

Because I’m using WordPress as the CMS framework for the site, I was able to easily setup a secondary mobile template that is only delivered to mobile devices, based upon the device type (HTTPuserAgent). This provided an elegant solution, since I was able to and optimize the content for mobile devices without having to create a separate mobile site to maintain.

In summary, I determined a hybrid approach to mobile accessibility that married the benefits of responsive web design with the practicality of a separately mobile theme was preferable. Responsive (RWD) techniques were used to adapt to a reasonable tablet experience, but mobile devices receive a separate theme, with limited navigation, and the content of those pages is also minimized. The mobile experience is greatly simplified and models the native device interface (using JQuery Mobile). This approach provides an optimal user experience on all three device classes (desktop, table, and mobile).

Article originally published at SitePoint:
Website Design for Tablet & Mobile

Goodbye Flash, Hello Edge!

It is no secret that the iPhone does not support Flash.  Steve Jobs went as far as to explicitly rule out support of Flash by name, in his famous 2011 speech.  And now, Adobe has responded by announcing they will no longer support or further develop the Flash platform. Instead, Adobe is quietly releasing a new product called Edge, which outputs HTML5, CSS, and JavaScript, as presumed replacement for their popular animation authoring environment.

HTML5 is very powerful and a significant milestone in the evolution of web application interface architecture.  Its not just about animation and native support of audio and video.  In truth, HTML5 in conjunction with JavaScript can do just about everything that Flash could do. The name HTML5 in fact is perhaps a misnomer in this regard as it should be noted that HTML5 embodies the standards for a collection of technologies that facilitate audio, video, real-time rendering and animations, local persistence, and so much more.  But because it is all inherently a part of the HTML5 document model (not compiled binary code), accessibility and SEO are no longer the issues they have been in the past.  Finally, the design and technology prerogatives need not be in contradiction to one another!

So what exactly is Adobe Edge? The basic concept for developers is quite similar to Flash.  The differences  are minor, for example the timeline is based on elapsed time now rather than key frames, similar to Adobe’s After Effects video editing software.  Also notable is the use of non-destructive editing. Rather than over-writing the original HTML and JavaScript files you start with, it will create its own parallel file set to augment what you started with. Available binding events are also a little bit different and better reflect their HTML5 underpinnings. Overall though, you’d be surprised just how similar the tools are.

Adobe Edge

With Adobe making this change to embrace HTML5, it seems that just about everyone is on board now with HTML5 and JavaScript as the way forward for development of rich internet applications; Microsoft even announced recently that they are depreciating Silverlight.  That’s on the desktop anyway. Mobile is a more complex issue with so much momentum still with development of native applications (iOS, Android, etc). The cost of maintaining separate applications for various devices certainly isn’t ideal however and truthfully, 80% of those apps could be replicated using HTML5 and JavaScript without much negative impact on user experience, but at a substantially lower cost.  Its primarily the sophisticated game and media applications that might not port as well.

Adobe Edge seems well positioned to take become the default product for interactive and animation authoring for the web 3.0 applications of the future. The product has done a good job of playing off of the strengths and knowledge of the existing Flash platform and hopefully will not alienate the developer base.  They’ve satisfied UI architecture prerogatives by keeping the output artifacts aligned with HTML5 and the document object model, and they’re going to be the first significant tool to provide an easy authoring solution for what will inevitably be a major new wave of web application innovation.  I’m sure it was a painful decision for Adobe to kill their golden goose, but this move should be a positive for everyone and may actually help them in the long run.

The Future Is the Semantic Web

I have a dream for the Web [in which computers] become capable of analyzing all the data on the Web – the content, links, and transactions between people and computers. A ‘Semantic Web’, which should make this possible, has yet to emerge, but when it does, the day-to-day mechanisms of trade, bureaucracy and our daily lives will be handled by machines talking to machines. The “Intelligent agent” people have touted for ages will finally materialize.
~Tim Berners-Lee

In the 1999, Tim Berners-Lee described his vision for the future of the Internet.  He described computers being able to parse and understand the same information that we can, and become personalized virtual assistants on our behalves.  Imagine the calendar application on your mobile device observing a schedule conflict when you begin to book a trip on Expedia.  Or what if your computer could do all of the research necessary to justify decision you task it with, and it compiles a detailed research report in support of a decision, ready for you in the morning.  To many, this is the true destination of the Internet, and all of the social sharing that caught everyone’s attention in web 2.0 is a mere preview of what is to come. One blog author I came across, described it by saying “Its a data orgy and your server is invited”.

For roughly a decade now, a movement called the Semantic Web (aka HyperMedia) has been seeking to enable this vision. It is thought that by properly annotating an HTML document, you will enable a computer to consume and understand the information, much like a human would, and thus be able to make informed decisions and act on our behalf.  To accomplish this, there must be structure and relational definitions, and several annotation protocols and descriptive vocabularies followed.  So far, adoption has been anemic however, for numerous reasons: the effort has so far been splintered by various factions creating competing technologies, and limited support by browsers and search engines, limiting incentives for early adopters to explore the technology more deeply.

But recently this has been changing.  As per the introduction of HTML5, there is now support for semantic tagging in all modern browsers, search engines have begun to reward websites with rich snippets in their search results when the semantic data is present, and reports have been written, detailing how BestBuy on others have seen lift in traffic as much as 30%, as a result of their recent RDFa implementations and resulting rich snippets in search results.

Others are also making use of semantic markup recently. Facebook has been using RDF in their social graph implementation since 2009 and recently began using the hCard and hCalendar markups of user profiles and events, respectively. Google has been making a push to ‘authenticate’ authors and authored works with the XFN rel=’me’ annotation, Yahoo Tech and LinkedIn are both using Microformats in their data as well. So now we have support, peer adoption, and recent evidence of positive ROI from the effort and we’re starting to see results; RDFa adoption increased 500% in the last 12 months alone! The only thing missing was significant corporate backing.

In response to this confusion and complexity, a consortium of the major search engines, including Google, Yahoo, Microsoft and Yandex, came together to create a standardized approach, called Schema.org. It assumes the use of Microdata rather than RDF/RDFa and provides a single resource for the major semantic vocabularies to be used.  The hope is that by simplifying the technology, and standardizing it in industry, adoption barriers will be reduced and the average development team will begin to embrace the technology.

So how does one implement semantic annotation?  There two primary annotation frameworks are the Resource Definition Language (RDF/RDFa) and Microformats.  RDF can be used markup with the HTML document with what are called Subject-Predicate-Object triplicates.  It is a robust language that is most commonly used for more robust datasets that require deep data linking.  It has been criticized for its complexity however and so a recent revision called RDFa focused on making it easier to use, with a focus on use of attributes. Here is an example of what it might look like if you were to markup a simple object in RDFa:

    <div xmlns:v=”http://rdf.semantic-vocabulary.org/#” typeof=”v:Person”>
        <p>Name: <span property=”v:name”>Neal Cabage</span></p>
        <p>Title: <span property=”v:title”>Technologist</span></p>
    </div>

Microformats meanwhile, was developed with simplicity in mind and a more natural extension of the HTML document.  MicroData is a subset of Microformats that Schema.org has settled on and even has its own DOM API in the HTML5 spec, so this is the presumed future standard for most websites:

    <div itemscope itemtype=”http://semantic-vocabulary.org/Person”>
         <p>Name: <span itemprop=”name”>Neal Cabage</span></p>
         <p>Title: <span itemprop=”title”>Technologist</span></p>
    </div>

At some level the concepts are quite simple, however there are numerous Ontologies and semantic vocabularies , which have been defined and are referenced in order to give meaning to your RDF or Microformat attributions.  Notice the semantic-vocabulary.org reference above?  It calls out to an externally defined schema for the Person hCard being defined here.  Schema.org has defined many of these directly for MicroData but there are others such as Dublic Core, DocBook, and Good Relations which is specifically very popular for eCommerce. There are plugins available for many of the major CMS and eCommerce platforms to assist with semantic markup as well.

The Semantic Vision
In the near term, the Semantic Web (or “Linked Data”) is merely an exercise of annotating your content more precisely with hope of getting the Google carrot at the end of the proverbial stick (e.g. rich snippets).  As a result, any short-term concrete gain main seem a bit hollow.  The real promise however, is in what is possible when semantic annotations in content reach critical mass.  This chart shows what some have been predicting in terms of the direction of Internet technology and proliferation.  The real intelligence and interoperations still very much lie ahead in what could be described as a Web 4.0 World.  This assumes that HTML5 marks the precipice of Web 3.0 and we all begin working on proper annotations now, to lay the foundation for those achievements in the future.

Web 4.0In the meantime, there are companies already beginning to do cool things, experimenting with the sort of predictive intelligence that one might expect from a “linked data” web 4.0 World.  Hunch.com in particular attempts to help you parse through excessive data on the Internet, by looking at your past interests and those of your friends, to determine what you might like and limit your choices accordingly.  Hunch.com’s CEO talks about how research exists to assert that fewer purchases are made when consumers are presented with too much information or too many choices.  It is thus worthwhile technology for retailer to pursue, in an effort to predictively get fewer precision choices in front of each consumer.

And with all of the data that currently exists in the World, we just continue to collect exponentially more, via social interactions, tags, various use profiles, online transactions and analytics data.  The buzz word in many organizations now is “big data” and they are looking to new tools such as Hadoop to help them address these problems.  In this growing cluster of data, we will eventually outgrow search as our most useful paradigm for how to access all of this data. And what will that look like?

I was asking myself this question the other day when I looked down at my iPhone and realized, the interface may already be here!  What would a computer interface look like that is largely Internet driven, but for which the user experience does not begin with an Internet browser or Google.com?  In fact, if the real vision of a semantic web, is intelligent consumption of data and that intelligence applied for specific applications as virtual agents, wouldn’t that manifest as a lightweight albeit more mature mashup style app, similar to what we already have on mobile phones and tablets today?  Its an interesting thought.  I can imagine a progression in that direction, with these applications continuing to grow in sophistication, intelligence, and awareness.

In closing, here is a video that I found while researching, that provides an excellent introduction to the topic.  If you’re looking to introduce these concepts to your team or stakeholders, this is a great place to start:

Web 3.0 from Kate Ray on Vimeo.

100% Javascript Web App Architecture

Imagine creating a Rich Internet Application (RIA) built entirely using JavaScript, from client to server.  What would it look like? I’ve already discussed the benefits of HTML5 and the coming Web 3.0 movement in recent posts so I will not into those details now.  Instead, I want to focus on what an application like that might look like.

First, to review the new resources at our disposal with HTML5: Because of the required implementation of the HTMl5 spec localstorage(), you will now have a much greater facility for maintaining persistence across the user experience and even over the lifetime of the user. The implementation specification calls for a robust key/pair database that can store up to 5MB of data and doesn’t necessarily ever expire (ideal for storing JSON objects). Second, security provisions now allow for cross-domain AJAX calls, enabling mashups of disparate data sources, directly in the client application.  Third, js workers are now a native concept, allowing background processes for intensive JavaScript processes, to now kill browser memory.  Fourth, native support for media, enabling bidirectional control of audio/video by the DOM and vice versa. Additionally, libraries for rendering sophisticated SVG graphic and even drawing real-time on the canvas. These resources provide tools for a much deeper user experience and will provide the opportunity to create a fatter client application, shifting much of your central logic to the client.

Meanwhile on the server side, a substantial amount of momentum is building behind Node.js.  In fact, check out Node.js on GitHub and you’ll find it now has more followers than Ruby on Rails. The promise of Node.js is an extremely thin event-driven framework for creating web services entirely in JavaScript.  The benefits of this approach are (a) an extremely fast framework that avoids thread locking and an all-JavaScript developer experience, which provides for both faster development and less team bifurcation.  Node.js is also ideal for managing long-session connections which might be used for media or chat sessions, etc.

Node.js Popularity

There has also been a lot of interest in NOSQL (“not only SQL”) databases, that are document  rather than table oriented.  So, instead of a structured pre-defined schema, you simply store your own data in a structured way that can be independently structured from any other document.  The documents are essentially serialized JavaScript (JSON or the binary equivalent BSON) and are ideal for persisting data objects from the software.  In fact, wasn’t this largely the goal of bridging ORM technologies such as Hibernate?

javascript ArchitectureSo where is all of this going?  If we look at the technology stack we now have available, it is natively JavaScript from top to bottom, with persistence resources on both the client and server, that are both ideal for storing serialized JavaScript objects.  Combine this with the deep integration of media and imaging, as well as support for AJAX web service integration across any domain, not to mention the proclivity of JavaScript developers toward  a fat client architecture – I think you have a recipe for a popular new fat client architectural paradigm emerging.

Considering mobile, it is ideal not to have page refreshes more than necessary.  With a web application architecture that is much less dependent upon server-side page refreshes, this could be a much simpler alternative to building complex native mobile apps, using Objective C and still another version in Java for Android.  And with all of the media resources implicit in HTML5, it appears we’re going to see much deeper still, RIA user experiences, which again benefit from a fatter client and thinner server application. In this paradigm, the server application is minimized to a collection of web services in support of the fat client application, which aren’t even an exclusive provider for the RIA application, since it can also not pull directly from other external resources providing JSON web services.

Node JSObserving other early indications of movement toward this model, Yahoo is set to release a new platform called Mojito, which seeks to “blend” the server/client paradigms into one cohesive development experience.  It is essentially built upon Node.js on the backend and YUI for a rich AJAX library on the client side. There is a whole philosophy they are building around the best practice for developing RIA JavaScript-centric web applications that service mobile and desktops alike.  Or you could assemble your own stack using JQuery and Backbone.js on the client; Backbone.js in particular is an MVC framework for JavaScript and may prove particularly useful as you begin building more complex “fat client” applications.

This movement toward deeper JavaScript’ing and richer user experiences online, began in 2005 but stopped short due to key limitations of AJAX, media, and persistence, though I think many saw the promise.  With those key issues now solved, I very much look forward to seeing a continuation of interface innovation that leads to further refinement in both architecture and user experience.

HTML5 Is Kind of a Big Deal

Have you looked at the new features that are part of HTML5?  I must admit that it took me some time to actually look at it because I dismissed it as just some new markup to have to deal with.  In that way, its really not correct for this to be labeled as HTML at all!   In fact, HTML5 is a collection of technologies that have been sorely missing from web browsers, and upgraded markup is a rather minor footnote.

The introduction of these technologies now means that we’re probably about to witness a major new architectural paradigm for web applications, toward that of a fat client, and away from the traditional static page model. Dare I say Web 3.0?  As someone who has always had an affinity for UI centric application, I must admit that I find this pretty exciting!

So let’s get into it. What is so exciting about HTML5? There are 5 major categories that I would us to describe these upgrades:

1. Dynamic Image Rendering

There are two major technologies to discuss here: SVG and Canvas. SVG is ‘retained mode’ and used mostly for data modeling or whereas the canvas ‘abstract mode’ and can be used for real-time painting and animations. In totality, these vector tools are what will allow the technology to replace what has otherwise been accomplished mostly with Flash up til now.

SVG – essentially a markup language for vector images that live in the document. That in itself is pretty cool which you consider the semantic web potential of the image data.   Mozilla has a really great example of SVG with their glow map that shows a glow dot on a map for each download of Firefox. The real time update of the graphic shows what can be done here.

Canvas – more for user interactivity and animations. So whereas SVG might be used in place of traditional Flex modeling, imagine the canvas replacing most uses of Flash.  A friend sent shared an example of an interactive animation in which the user draws a stick figure and then interacts with the stick figure as he begins walking across the screen and doing random activities.  The painting of the screen is all possible via the HTML5 canvas. The animation and interactivity, is the interplay of JavaScript with the canvas.

2. Native Media Support
Embedding an image or audio file is now as easy as embedding an image. This is probably the most famous feature of HTML5 as we all have heard that Steve Jobs pointed out that you no longer need Flash to embed videos, right?   But that is  just a tactical concern.  What really makes this cool are the implications of having natively supported video.  Imagine what you can do when you have a UI that can directly interact with the video, rather than simply play it.  For one thing, you can have multiple iterative hot spots throughout the document that trigger videos based upon user or application events. That’s true Flash or Director multimedia, directly in the DOM and SEO and Mobile friendly.

There are also interesting experimental technologies such as the Popcorn.js library that looks at triggering events within the UI or application via embedded footnotes in the video itself.  There are many people buzzing that Interactive/Internet TV is finally going to take off in 2012 because the technologies are finally going to be present to do so. Imagine the implication for TV content producers if they’re able to conduct the iterative events on an Internet TV video embedded directives in their video content?  They could instigate real time data and social media and coordinate it with their media experience.  That is very compelling.

3. Semantic Tagging
This is compelling from a Semantic Web perspective. So far in the evolution of the Internet, everyone has focused on creating websites and web applications that engage a user.  But what if a computer needed to interact wit your content?  The only significant example of this to date are the search engines and we should all be familiar with SEO and how pages are altered to make a document parseable by search engines.

But imagine taking it a level beyond that.  What if your computer acted as a virtual assistant for you and you were able to book a plan trip on Expedia, when your computer interrupts you to remind you that you have a scheduling conflict already.  These sort of aware systems would only be possible if they were themselves aware of the content we are interacting with.  So the idea with the semantic web, is that if we all properly tag our content with appropriate tags and meta data, we make it possible for such systems to be aware and to consume our content.

HTML5 takes a big step forward on this, both with semantically appropriate tagging, but also with formal adoption of meta tagging standards such as RDFa and the @rel attribute that can be used to map together authors and their contributions online, which I discussed a bit in another post.

4. Local Data Storage

Initially the HTML5 specification called for a location implementation of a SQL database.  Sadly, this was deprecated last year.  Many of the modern browsers have already implemented but it may not be there in future browsers.

What is there however, is an client-side key-value database solution for local storage. Using the localStorage API, you can store up to 5mb of data and it seems to persist indefinitely, or until the user manually purges their database.  So this can still be very useful. This is basically cookies on steroids.

This probably is a better solution than a local SQL database.  Consider the movement of NOSQL database systems toward non-structured document databases rather than tables and schemas; they essentially store JSON objects which are native and ideal for persisting state of a JavaScript application.  Given that an HTML5 JavaScript application would be the consumer of this database, it seems this might actually be the perfect solution for maintaining state, compared to a traditional SQL dB.

As for why this is a big deal, it has the potential to completely change the architectural paradigm of web applications!  Persistent state is one of the big issues that pushed traditional “fat client” application design toward a thin client/fat server model, since web applications relied on the server to remember everything.  If this issue of state is finally resolved, we could see a return to a fat client model, in which we’re doing way more development in JavaScript on the Client side, and much less on the server.  Many, many implications here!

5. Standardized Resources

Other resources have also been standardized here as well:

JS Web workers – this is a subtle yet bit one.  We’ve all probably experienced the occasional web application that seemed to load really slowly and kill usability because the JavaScript had a lot of work to do and ran away with the app.  With web workers, its possible to isolate certain JavaScript threads as background processes, similar to Unix Daemon processes.  That could he very helpful for large those pesky social 2.0 JavaScript includes that delay your document ready event or other data fetching or calculation intensives such as prime numbers, etc.

Cross Domain – AJAX can finally make calls cross domain, rather than being limited to the domain or origin.  This is again huge in terms of being able to build a fat client app, particularly in the days of  mashup APIs.

GEO Location – Also apparently each browser now provides a standardized Geo-location API. The implementation details are up to each browser; Firefox in particular apparently now implements Google’s Geo API.  So now we have standardized location approximation, for all computers not just mobile.  This has recently popped up in Google’s maps in fact.  Clearly this is an attempt to support creation of a single fat client app for mobile and the desktop.

Conclusion

So there you have it, HTML5. Each one of these respective upgrades is a big deal, but in my opinion the really big deal is that I think this will trigger a new architectural paradigm in web applications.

Imagine writing a single application that is equally engaging on your mobile app or desktop.  It retains its own state and doesn’t require page refreshes, so its an optimal experience with even a slow internet connection or lack of connectivity such as being on the rode or in airplane mode.  Imagine the client is where the majority of the application logic lives and only minimal server calls are required any more, and even those being written now in an event-model pattern using JavaScript via Node.js, rather than an entirely different server side technology.   Yep, this indeed has the potential to trigger a really exciting new era of web application development!