Java 
MySQL 
Spring Boot 
UIPath 
BACKGROUND
Our client, an FMCG major, wanted to automate data scraping, extraction, and analysis to track supply chain metrics and market trends
Unilever is a global manufacturer of fast-moving consumer goods (FMCG), including food, beverages, beauty products and cleaning supplies, operating in more than 190 countries. For a multinational company of this size and scale, raw material sourcing across its global supply chain is a huge, ongoing challenge.
The multi-national was looking to build an application that would automate the data aggregation process, improving accuracy and presenting the results in a highly accessible and straightforward dashboard. It wanted to be able to analyze and present information in graph and chart form, and help its team remain proactive if certain supply chain metrics went above or below a set threshold with email notifications.
It chose Asahi Technologies due to our expertise in creating automation tools, our clear understanding of the project’s scope and definitive timeframe we proposed from the outset. Our standing partnerships with UiPath and Automation Anywhere, two of the most significant automation service providers, also demonstrated we had the tools to help Unilever create the right solution.
CHALLENGE
To extract the correct information from over 70 websites despite dynamic URLs and varying data formats
To gather the data in a more efficient way, Unilever needed to automate the data scraping process from 70 websites, which share information about important market trends. Each website had to be crawled at a different frequency too, ranging from twice a day to once a month, depending on the significance and volatility of the data.
As well as monitoring raw materials, Unilever also needs to consistently source construction materials to build additional factory units to keep up with demand for its FMCG items. Yet understanding the ever-changing cost of these supplies, which can be impacted by factors including production costs, global supply and demand, forex rates, logistics, inflation, commodity rates and fuel rates, proved difficult.
But as the data formats and the URLs were constantly changing on many of the websites, it was difficult to automate the process and ensure it extracted the right information every time. To make matters more complex, lots of the information needed to be downloaded as a file before the information could be analyzed by any automated tool and extracted.
Our client had a dedicated team who manually aggregated data to keep decision makers informed about factors that could change the price of materials. But this manual process was labor-intensive and vulnerable to human error, which resulted in poor decision making that was impacting Unilever’s bottom line.
SOLUTION
A robotic process automation bot and tool to collate data and an open-source library to render charts and graphs
To meet our client’s requirements and timeline, we started by forming two separate teams that would work in parallel on different aspects of the application. Team A focused on the automation script/bot, which would scrape relevant data and send it to the application’s backend. Simultaneously, Group B developed the backend service, application programming interfaces (APIs) and the web-based dashboard using Java and Spring framework.
Following our recommendation, Unilever chose UiPath as its automation platform, which would extract and parse data from its chosen websites. We chose to develop the frontend using Angular JS, with Angular Material and Bootstrap components. This gave the single-page application (SPA) a contemporary appearance and impressive functions. We also used Chart.js, a reliable Open Source library, to render data into eight to 10 different types of chart that would suit a variety of different types of data.
Our team overcame challenges such as dynamic URLs by analyzing the sites over a period of time to identify the varying attributes of each, such as the date, which allowed us to nail a target website URL to scrape data from.
Once our initial work was complete, we deployed the complete system on an AWS EC2 Windows instance. Unilever tested it for a week to make sure the robotic process automation (RPA) bot was collecting the right data, which was then rendered on the dashboard.
TESTIMONIAL
Responsive team
Global Procurement Manager, Unilever
IMPACT
Our app and portal helped save up to 20% in raw materials cost, reduced 16 hours of monotonous work every week, and boosted data accuracy by 40%
By automating this manual process, the information being extracted is now accurate and error-free. Two full-time, Singapore-based employees have been freed from 16 hours of incredibly monotonous work every week, saving Unilever around $50,000 annually. And the addition of automatic trigger notifications, which help Unilever make proactive, informed purchasing decisions well before costs have escalated, has led to savings in raw materials of up to 20%.
Our solution hasn’t just been cost effective. Automating the process of collating data has improved data accuracy by 40%. Employee productivity and morale has soared too, as they are able to focus on their core role rather than getting caught up with manual, monotonous tasks. Most importantly, global decision makers can now access up-to-date information about material pricing in real time, rather than being limited to employee office hours.
