BACKGROUND
Unilever, a global manufacturer of fast moving consumer goods such as energy drinks, processed foods, beverages, ice creams, beauty products, cleaning agents has a strong dominance in over 190 countries around the world. While the company is headquartered in London, it had its Global Procurement team working from Singapore.
For a company like Unilever, considering that they rely on a global supply chain, raw material sourcing is a huge challenge and pricing for raw materials extremely volatile. They have a constant need to source construction materials such as silica, chloro alkalies, and coal to build factory units that manufacture FMCG products. The cost of these construction materials are constantly impacted by various factors like production costs, global supply and demand, bank financing costs, logistics, shipping and docking costs, forex rates, fuel rates, transportation charges, commodity rates and inflation.
At the time, Unilever had dedicated personnel to aggregate data from various sources and present to the decision makers who needed to understand in real time any major change in trend on any of the above factors to the extent it would directly impact the procurement cost of these materials.
This process caused a lot of unnecessary work and overhead, not to mention human-error, which led to poor decision making and ultimately started affecting the company's bottom line.
Unilever decided it was time to build an application that would automate the data aggregation process. This data could then be presented to decision makers in a nicely readable format in a dashboard
They reached out to several companies for the project but finalized the agreement with Asahi Tech as we were able to demonstrate our superior expertise in automation tools, showed the most understanding of the project scope and proposed a definitive timeframe for the development.
Business requirements
- ✓Automate data scraping from 70 different websites, which have information indicative of market trends.
- ✓Analyze the data, make meaningful inferences from them and present them as easily digestible graphs and charts in a dashboard.
- ✓Make this dashboard accessible to decision makers and other relevant members across multiple teams in the organization.
- ✓Receive notification emails whenever a metric (foreign exchange, diesel rates, national repo rate, etc.) breached a certain threshold so that the procurement team could take the necessary measures proactively.
Challenges
- ✓The format of data on the websites monitored by Unilever was constantly changing, making it difficult for an automated process to pinpoint the right information to be extracted.
- ✓Depending on the significance of the data being extracted and its volatility, the frequency of crawling had to be different across the 70 sites ranging form twice a day to once a month.
- ✓In some cases, the URLs themselves could not be fixed in advance for the application to crawl. They had to be determined on the fly as the URLs had the date affixed to them.
- ✓Much of the information to be used was available in files to be downloaded from different URLs. So, it required the files to be downloaded, and information from them parsed before useful information could be extracted from them.
We were perfectly positioned to advise the client on the automation solution most suited for this requirement as we had standing partnerships with UiPath and Automation Anywhere, 2 of the 3 major automation service providers in the market. Based on our recommendation, the client chose UiPath as their automation platform to extract and parse data from the sites.
SOLUTION
- ✓To get this project done in a timely manner, we divided the development team into two groups, working in parallel. Group A worked on the UiPath automation script/bot that would scrape the data from the web and send it to the back-end while Group B developed the backend service, APIs and the web based dashboard using Java & Spring framework.
- ✓As it was a Single Page Application (SPA), the front end was developed using Angular JS, and used Angular Material and Bootstrap components. This gave the appearance of a modern site with impressive styles and animations.
- ✓In cases where the URL themselves were dynamic, we overcame the challenge presented by analyzing the sites over a period of time and identified the varying attributes affixed to the URL (for example, current day/month/year) to nail the target website url to scrape from.
- ✓To present the information on the dashboard using charts/graphs, we rendered the data in 8-10 different types of charts using Chart.js, a robust, reliable, and Open Source library. The type of chart presented was determined based on the type of data and its frequency.
- ✓Our team deployed the complete system on an AWS EC2 Windows instance and had the client test it for a week to ensure that the RPA bot was collecting the right data for the whole week. That same data was then rendered on the dashboard.
The application is currently live and available here and accessible only to users with the right credentials.
RESULTS
- ✓Our team automated the process of fetching commodity information from different websites by building a Robotic Process Automation solution using UiPath. The RPA solution implemented was able to save the client time, effort and cut down overheads significantly.
- ✓Furthermore, since a robot was now extracting the information, the output was accurate and error-free.
BENEFITS
- ✓The solution we developed eliminated 16 hours of daily monotonous work for 2 FT employees in Singapore, which amounted to ~ US $50,000 in aggregate annual savings.
- ✓Because of this RPA solution, decision makers were able to access this information in real time irrespective of office hours
- ✓The automatic trigger notifications we added for threshold breaches helped the client make proactive and informed purchasing decisions well in advance, thus saving them at least 20% in material costs in some cases.
- ✓Lack of a need for manual intervention improved the data accuracy by 40%.
- ✓Employee productivity and morale increased as they could focus on their core function instead of performing manual and monotonous tasks throughout the day.
