I am well rested and ready for the final day of Fast Forward. 

I was bright eyed and waiting for the gym doors to open this morning at 6:30.   Ran for an hour, lifted some puny weights, left feeling buff (all is relative my friends), and sauntered into the main conference room about 5 minutes ago.  

The familiar pounding beats of overplayed top 40 welcome me.  It is a Microsoft conference afterall.    Sing it Britney:  “Womanizer, womanizer you're a womanizer ...”   And here comes Jared Spataro!   

Despite the desperate effort to inject 'Apple Cool' into a Microsoft event - the facade was dispelled by a key note speaker who shared his love of Star Trek and went on to perpetuate the PC stereotype no television commercial has yet to shatter.

Although I have stifled a few giggles here and there, I am genuinely impressed by everything that has been presented so far.    Like it or not some aspects of enterprise technology will forever carry a 'tech geek' stigma.   

Embrace your geek.  Revel in it!   The fact is, regardless of the packaging, technology is awesome.  ‘Geek envy’:  I have it.  I am not ashamed to admit it either; in this business, I feel as though I am constantly playing catch up to the geeks I’m convinced left the womb with wireless.

I spent yesterday in the Internet Business Track break out sessions, and had the opportunity to see a number of search demonstrations that literally left me, and those around me awestruck. 

Our reaction was not due strictly to the individual user experiences we were shown by the Bank of America, Reuters, Lexis Nexis, Contoso (MS sponsored demo), EveryZing, Zap and others.   While each of the demos showcased excellent search based user experiences designed to support their organization’s unique customer needs, we were not consumed by a single search implementation.    

On the contrary, the sense of amazement lay in the idea that all of this is possible.  The aggregate potential, the sum of these varied search experiences represents an almost unfathomable master search capability.  

   Sitting here ready for today's agenda I, for one, am on the edge of my seat.   And oddly, I find myself curiously interested in Star Trek…

It is 5am here in Vegas.  I am in town for the  Microsoft Fast Forward search conference, on the cusp of day two.   (www.fastforward09-micro.com)

Last evening I returned to my room after dinner with the intention of writing this post.   Many of my peers were preparing for a big night in Sin City, which leads me to believe there may be slightly less energy in today's early morning sessions.   Having never been the life of the party afterhours, I become the proverbial pumpkin by midnight and as a consequence do my best to make a subtle exit from festivities well before.  

As I got into bed with my laptop around 10:30 last night, I was in deep reflection on some of the ideas raised in the opening speeches of the day.     The key note speaker was Don Tapscott, author of Wikinomics and Grown Up Digital.   The focus of his talk was Generation Net - or as we have often heard them called, the Millenials and / or Generation Y.   His perspective on their culture, comfort with technology and demand for instantaneous, multi-sourced, authenticated information was fascinating.  Don was followed by two more speakers who elaborated on the same theme - engaging the user and understanding what the upcoming generation of consumers / workers require in order to engage.

In my previous post I said that on the web, we can influence user behaviour, but we cannot control it.   Yesterday that statement was taken even further.  User empowerment is the key to engagement.   The suggestion being that our only hope for influencing user behavior, is to provide users with tools that allow them to control it.

Today and tomorrow the sessions will dive into the mechanics of search and explore a variety of scenarios that demonstrate user expectations and the consequences of not meeting those expectations.   

Amidst all of the economic doom and gloom saturating the media, it is an incredible breath of fresh air to be listening to speakers who carry an optimism and curiosity about the future.   Rather than cling to a traditional means of business that is unsustainable, it seems Fast Forward is looking at this stage in history as an opportunity to embrace change and acknowledge that the next generation has insight and technical wherewithal worthy of our attention.   By recognizing their value and building our solutions to accommodate their technology needs, servicing this generation brings with it the potential to alter information access and global communications forever.

"Ahhhh Grasshopper....the student becomes the teacher."   

For any of you who did not smile at my lame David Carradine quote - circa 1975 with reruns through the 1980's, you are probably a Gen Net ... and I have sadly, just dated myself.    But of course I had done that already:  In bed by 10:30 in Vegas?     God help us all. 

This series of articles will examine FAST Navigators and how they can be utilized in your application.  Navigators are very much like filters.  Users are able to drill down into the result set by clicking one or more navigators.  In my opinion,  the future of search relies on being able to serve relevant results as well as allow the user to drill down into the result set with the help of navigators/filters.  Today, users tend to type in a keyword expecting that the results will be relevant to what they’re searching for.  Being able to return the relevant results is great but what if you have a very large index?  Returning relevant results may not be enough if the result the user wants is on page 5 or 10.  What if you could also allow the user to drill further into the result set to get even more relevant results?  Navigators could be the way to make that vision a reality.  Simple example when searching for products would be an ability to select a price range, color or even city that the item is being sold at.  More complex scenarios could utilize a feature called Result Clustering.  I will discuss Result Clustering as well as other navigation features in future posts.

Navigators can become costly when it comes to performance and with that in mind FAST has two main groups of navigators:  Deep Navigators and Shallow Navigators.  Deep Navigators are generated from the entire results set, whereas Shallow Navigators are generated from the top N (default 100) documents returned by the query.

There are three basic navigator types in FAST:

Numeric Navigator – applies to numerical data types (int, double, datetime).  This navigator can be easily configured based to break the results into groups (aka buckets) or it can be set to a fixed value.

String Navigator – is a navigator that is based on the values in a specified field

Word Navigator – is similar to String navigator except that the result set is generated from all fields (including composite fields) in the result.

In part 2 of this article I’ll drill into the navigator types in more detail as well as examine scope navigators and results clustering…   stay tuned!

Welcome to the FAST SharePoint blog. As this is my first posting to the blog, let me started by introducing myself. My name is Augustine Ng, and I am an architect at imason Inc. I have always been pushing for the creation of solutions that help companies in various fields to better serve their customers, and more recently focusing on companies in the financial and telecommunication sectors.

These solutions invariably involved the creation, management, analysis and retrieval of the right information and knowledge to the right people at the right time to help their staff make better decisions and better serve their customers. Business Intelligence (BI), Reporting and Search had always been recurring technologies we used in these solutions.

In this posting I would like to share some thoughts on how the Enterprise Search capabilities of the FAST search engine can be leveraged by enterprises to dramatically impact the ways that business intelligence were gathered, reports generated, knowledge pinpointed and insights derived through search.

Traditional RDBMS / OLAP Way

With the current state of the economy, there are consolidations in many industries across many regions. There will be increasing needs for companies to derive business intelligence form different data sources. Traditional BI solutions involve expensive and time consuming ETL (Extraction, Transformation, and Load) processing, creation of data warehouse, data mart, and multi-dimensional BI cubes for mining structured data. Inclusion of additional fields may require changes to the schema of the data warehouse that require engineering and database administrator supports which could be expensive if not untimely. More over not all relevant information exist only in a structured form. For example, emails, proposal, contracts cannot be readily mined in the traditional way.

The Search Approach

Enterprise Search Engine like FAST supports the indexing of both structured and unstructured information and provides the opportunity to include 100% of all available information. Content Processing like linguistic data cleansing and fuzzy matching, extraction of concepts and entities by modern search technology can be leveraged in the merging of information from different data sources.

The “relative inclusion” of relevancy ranking of search results is superior to the binary “inclusion or exclusion” behaviour of SQL query which is a programming language. While most RDBMS vendors support some forms of natural language query, they are far less capable than the linguistic processing and Query Processing capabilities of a modern enterprise search engine.

With a simple search box, and powerful navigators in which lists of terms reflecting concepts or entities in the corpus are dynamically displayed in real time, helps provide interactive user experience as user clicks on them to “navigate” their way through the contents. This eliminates expensive training compare with traditional BI system, and helps to make the search based system more accessible to a much boarder audience. The Results Processing capabilities of the FAST search engine also opens up the opportunities to render the search results in powerful charting and visualization formats.

Enterprise search engines are also design for high query volume in thousands of queries per second, which the traditional BI system is not designed for. The absence of a data schema and the total de-normalization of the data in the search index provide flexibility in ad-hoc querying against the index. This also lowers the change management and maintenance costs of the system.

I am excited and fascinated by the evolution of search.   

Since my first introduction to interactive consulting back in 2000, I have spent most of my time working with clients on their public facing websites.   I must admit, I had come to think about search almost exclusively from the perspective of SEO and SEM.  To me, and many of my industry peers, site search was really an afterthought.  Sure, the site needed a search box, just as it needed other website staples:   Main nav, sub nav, links to Contact Us, Home, French and on and on..

But really - who were we kidding?  Beyond dropping in the Google Mini or something similar, there wasn't a whole lot of time invested in the site search.   Why would there be?  With strong IA, compelling creative and useful features and functionality, users would have a great experience and would interact with the site according to our carefully conceived plan!

Times have changed.

We used to think about Web Strategy as a means to answer two core challenges:  

1. How do we get users to our site?
2. Once we have them, how do we keep them?

There was the Search Strategy to answer number 1:   Based on the concept of "If you build it, they will not necessarily come" - the search strategy focused on tactics like SEO, SEM and paid placements (banner ads, editorial content, online PR etc.) designed to drive traffic to the site.  

Website Strategy addressed question number 2:  Website strategy assumed that the search strategy was essentially completed once the user arrived at the site.  Thus the responsibility of the website strategy was to keep users engaged and give them compelling reasons to return.  This is where the importance of strong usability, inuitive navigation, quality content and designing for the target audience becomes critical.  

Of course all of this is still important, but now there is a third consideration:   Search As A Strategy.

The pervasiveness of "Just Google it" has created a reflex tendency amongst most of us to initate our web session at the search engine; punching key words in a frantic and hurried effort to retrieve the most accurate information in the shortest possible timeframe.  Blame a shortened attention span, hectic work day, limited patience - whatever.  That fact is, we have come to expect information, served up and to our liking, without delay.   The novelty of hunt, explore and navigate is long gone, and with that a revelation:

Like it or not, we can influence user behaviour, but we cannot control it.

What does this mean?   Despite our grandest efforts to position information and site features in the manner most conducive to an optimal user experience, if the site search cannot answer a user's query in a split second, in a split second the user is gone.  There he goes, back to the search engine to try again.  Why?  Because your site did not have the information he was looking for.   Even if it did, perception is everything, and in this user's mind, what could not be found does not exist.

Something to think about.  

In this blog, I will be rambling on about the changing landscape of enterprise search and the rise of information access technology on the internet.  Search as a strategy.    Getting the user to your site is not enough.  Not anymore.