By Joe Snevely, Data Architect
Here at Knowland, we are continuously thinking about ways to make our customers outperform their competition.
A little over a year ago, we believed our customers were too restricted with their search options and we really wanted to remove those restrictions. We believe the customer should be able to do things like find the best fitting leads for SMERF groups who meet on weekends, in Q4, and have booked within a 20-mile radius of their three biggest competing markets, for example. They couldn’t do that in our existing technology, and we wanted to fix that.
And so the engineering team at Knowland was tasked with improving search results within the platform. We had begun providing scored leads identified and calculated using a set of proprietary algorithms. These leads were created and processed by a rather complex data processing engine. The data produced by the engine was good and the algorithms had been thoroughly vetted and was useful to the customers, but we knew we could do better. Leads should be more dynamic and accessible for all customers, not just those identified by a somewhat narrow engine. The formula used for scoring should be applied to sort and highlight the leads that were the “best-fit” for the customer with every search. We needed to apply our existing scoring formula to every group returned by that search and only to the bookings that match that search. Engineering agreed we could do it but did not think it was achievable against a database of our size and in real-time.
It turns out we were wrong.
The real challenge for engineering was that this wasn’t a normal search problem. This was a filtered data set having an on-demand heuristic function applied dynamically to each aggregation every time. We couldn’t pre-process this data because each search would be completely different. Additionally, for each search, the results would be tailored specifically to the attributes of the property running that search. Every search would have to be done in real-time. Essentially we had to apply our scoring algorithm to every group returned by a search, but only for the bookings that match that search. Then we had to use the collection of each of those bookings to calculate hotel metadata for those specific bookings and compare them to the same type of data of the customer who is running the search. All in real-time. All against our 13.5+million event records.
At the time, we were already in the process of transitioning our application search layer to use a NoSQL document database served through a GraphQL API with a layered retrieval pattern similar to Facebook’s DataLoader. This was going to allow us to open up the types of searches customers could do, as well as make it faster for engineering to develop new features. Because of this existing transition we were able to identify a way to convert our existing Scala scoring functions into a custom aggregation function used by the new database. There was a lot of work required to orchestrate and parallelize these functions so that searches were fast for the customer, but since we had already developed a new search layer, fitting the scoring function in after we had converted it became trivial.
The only remaining challenge we faced was that new scored searches were entirely embedded in a new, vastly improved API that our old solution was not sophisticated enough to express. We had to rebuild our solution from the ground up using the new search layer as the heart of a brand new application. This was when the SmartSearch functionality was born.
Our product, design and engineering teams worked diligently over the past year to put everything together in one new platform that made these new search capabilities accessible and easy to understand for any hotel. We’ve been able to get customers new and better answers to their search queries into our massive dataset that they were not able to see before, enabling them to proactively sell group with greater efficacy.
It’s been a long journey so far and well worth the wait as this new SmartSearch functionality is revolutionizing how our customers proactively sell group and outperform the competition. We look forward to leveraging this technology to introduce even more exciting features in the future.
Click here to request a demo of Knowland’s SmartSearch functionality.
