From data to wisdom, one customer’s journey in Big Data and Cognitive Computing space

About 2.5 years ago, a startup contacted us to help build an ambitious analytic product in the field of Big Data and Cognitive Computing. Open to forming a dual shore team setup of dedicated architects, business analysts, developers, and quality engineers, the client needed us to overcome a steep learning curve across the domains of analytics, machine learning, and natural language processing.
The project was exciting because we had to quickly identify a team, upgrade our skills, and jump into action. The technology stack was quite broad, ranging from near mainstream programming languages such as Scala, to the actor-based message-driven framework Akka, to the analytical libraries such as Spark MLlib, to SparkR and many more.

Finding the right mix
We quickly realized that hiring a team with this expertise was a challenge in a high-demand marketplace. Our Offshore Delivery Center (ODC) in India approached the Human Resources team for help but new candidates were just not available. We were on our own. So we searched internally for help and we found our team. We looked for people with the right attitude toward learning and a focus on delivering value to the customer with every release.

Learn, learn, learn…
The team poured their energy into learning fairly new technologies and concepts such as Scala, Akka, Apache Spark, Apache Mesos, Rancher, NoSQL columnar databases (eg. Cassandra), NLP systems, RDF, Ontology, and many more libraries. It was inspiring to see the team picking up ideas, building proficiency  and coming together as a group.

Product leadership
Product development kicked off and we were committed to delivering value and product leadership. Our product owner along with the customer’s marketing team conceptualized the solution. We also prioritized features based on industry-specific use cases in Retail and Healthcare. This value-add  focused all our energies on building a product that would cater to our client’s customer needs. This significantly improved the market-product fit and we are now able to launch the product at the right time. Knowing how to build is good but knowing what the market needs is invaluable.

Project leadership
In addition to carrying the product vision forward and to ensuring sustained growth without lag, we added shadow resources on our own. These additional resources reduced the risk of ramp-up times that would have severely impacted launch dates. Since the technology was new, we had to train internal resources in parallel with the product being in iterative development. This ensured that skilled resources were available on-demand. Point in time; we introduced DevOps engineers as needed when the product went into beta release phase. This steady pool of skilled resources proved to be vital for building scalable, fault-tolerant and robust deployments.

The story doesn’t end here
The product is currently getting positive reviews from a small group of target audience and the client’s customers are lining up to explore more. There are already opportunities to do a few paid pilot implementations to optimize customer’s business goals. This product is about to launch, so stay tuned as this story unfolds.

Share this post