a bespoke data management system for
that automates data capture, validation and provides a web portal for clients.
Bridgethorne is a specialist in helping suppliers to retail chains understand their sales and maximise revenues through analysis of data. Our bespoke software automates data capture, validation and provides a web portal for clients.
Bridgethorne is a specialist in category management, shopper marketing and customer management for food and non-food, FMCG and CPG. Part of this work involves the retrieval and validation of large data sets from retailers. Data is analysed and then substantial reports are made available to clients. Bridgethorne had a requirement to automate its process for fetching purchase data from retailer websites, to validate that data and for a secure web portal for clients to access their reports.
We spent a few days with Bridgethorne at the start of the project to fully understand their requirements, mapping processes and discussing various possible solutions. The final solution delivers three parts: automated retrieval of detailed product information per store from specified retailer websites; validation of the data; design and setup of secure portal for client access.
To retrieve data from retailers, a .NET application was built, that automatically fetches data and transforms it into a standardised format. There was a requirement to not put retailer sites under unnecessary load, so the application was built with this in mind, maximising efficiency of the retrievals and timing out if necessary.
The system is required to support multiple data formats and sources, outside of the data sourced automatically from the retailers, in that Bridgethorne sources data manually from other sources. The system normalises, processes and cleans all of this data and saves to an MS SQL database. The data importer functions largely via lookup tables, matching the native data with internal data formats, particularly where products have different names and codes across retailers, enabling multiple products in the raw data to be mapped to a single product in the normalised output.
The data is processed further by a 3rd party service, the outputs actionable reports for clients. We setup a secure web portal that enables Bridgethorne to manage each client's access and then automatically populates the each client's page with the latest report from the 3rd party service.
Overall, our bespoke solution has automated many of the previously manual processes that Bridgethorne performed on a weekly basis, offering an end to end solution from data retrieval to reporting, enabling Bridgethorne to spend more of their time doing what they do best - analysing data and delivering value to their clients.