The earliest recorded use of data analysis most likely goes back to ancient Egypt and the second millennium BC. The Egyptians introduced a periodic census to gather tax information and ascertain whether there was enough money in the coffers to build the next pyramid.
The census was an idea that eventually took hold all over the civilised world. But collating the information was both laborious and time-consuming. Before the advent of computers, for example, the 1880 census in the US took more than seven years to process.
Today, technology is helping to mine data at an ever-increasing rate. Through the use of mobile, wearable and app technology, businesses are harvesting all kinds of information, not simply about their customers but also about their employees, operations and processes, that all feed into the bottom line.
Increasingly, CEOs are being asked to report to their boards on internal matters that previously haven't come under much scrutiny, such as pay equity, skills gaps and diversity, and often they are not prepared. In the UK last month, 1500 firms failed to meet the deadline to report on gender pay gaps and now face legal fines.
Karin Sanders, a professor and head of the school of management at UNSW Business School, says a grasp of 'big data' has become a recognised part of people's daily jobs.
"When I conduct training sessions, I ask who is collecting data in their business, and 80% of hands go up. But when I ask what they are doing with it, there's a more muted response."
There's a broad understanding that data science offers a more accurate reflection of what's actually happening in a business, rather than relying on the gut instincts of the highest paid executives in the room.
As a region, Asia-Pacific (APAC) organisations are ahead of the rest of the world when it comes to data and analytics maturity, according to advisory firm Gartner.
In the APAC Data Literacy Survey, commissioned by Qlik, most of the Australian workforce (91%) agreed that data helps them to do their job better. However, they also believed that having a better data literacy level would enhance their credibility and value at work. And therein lies the problem.
"Too often there isn't the capability within organisations," says Sanders, and because of that, among rank and file employees, there is widespread anxiety around data.
'There is a duty of care that employers have towards their employees, but are we moving too far?'
– KARIN SANDERS
The same APAC survey, which polled 5288 full-time workers across Australia, Singapore, India, China and Japan, found an escalating skills gap across the workforce.
While nearly three-quarters of Australians use data once a week or more in their present roles, one-third admit to getting overwhelmed when reading, working with, analysing or challenging data. The majority, 82%, don't think they have had adequate training to be data literate.
The speed and proliferation of data has led to a premium for those who understand what it all means. A global shortage of data scientists, predicted by a McKinsey report in 2013, has come to fruition with data scientists now commanding top salaries.
Sanders acknowledges that, for many people, data analytics isn't an easy thing to get your head around.
"It's not as simple as installing a computer program or doing a short course. On the other hand, you don't have to go it alone. What is needed is for HR to take the lead, and raise the important questions at the beginning."
It helps to have a focus on what you want the data for, says Sanders.
For example, a business may wish to understand how customers are using their products, so they can better meet their needs; or, they may want to explain high levels of absenteeism in the workforce, so they can solve the problem and raise production. These kinds of things can be revealed by data.
The application of people analytics is a growing industry. The allure of being able to make objective decisions that remove bias and intuition is compelling, particularly if it can be shown to lead to more effective and just outcomes.
Driven by the widespread adoption of cloud HR systems, companies are investing heavily in programs to use data for all sorts of workforce planning, talent management and operational improvement.
But just moving a human capital management (HCM) system to the cloud doesn't guarantee success without the buy-in of staff and a culture of collaboration.
And there is scepticism about the data itself. Uri Gal, an associate professor in business information systems at University of Sydney Business School, explains how using algorithms to resolve an issue does not produce 'digital solutions' that are as watertight as they may at first seem.
"If you're a manager and you want to assess the performance of your workforce over a certain year, you may look at data points such as how much revenue an employee has generated, or how much feedback they received from colleagues, or how much time they spent with clients, and apply an algorithm to process this data for over 1000 employees so that you arrive at a supposedly rational assessment of who was your best employee over the past year," Gal says
The problem with this, he adds, is why those particular data points were chosen.
"You could be using three other data points or 300. There is no one correct set of data points. Also, what weighting is being used: are they all equally important? The process isn't as objective as it appears."
'[Technology] can tell how fast you log on to your computer right up to the notorious case of Amazon which issues its staff with wrist bands to monitor their actions'
– AARON GOONREY
Sanders notes that data mining is becoming mainstream in monitoring the health of employees simply because a healthy workforce is a more productive workforce. But not everyone is comfortable with their employer looking over their shoulder at their lunch choices or exercise regimen.
"On the one hand, there is a duty of care that employers have towards their employees, but are we moving too far? And what happens to that data when an employer leaves the company?" she asks.
With mental health a widely acknowledged problem in workplaces, and with people unwilling to disclose, Sanders can imagine a code of conduct may be needed in the future, with employees signing agreements to allow their employer to access personal health data.
Aaron Goonrey, a lawyer at Lander & Rogers, can see how employers giving their staff Fitbands or Smartwatches helps them to analyse "how they can reduce stresses and workers compensation claims and other things that are positive for employees".
But the flipside is that it involves a level of intrusion that poses ethical implications.
From the moment someone uses a swipe card to enter a building, to logging on to a computer or using their company-provided smartphone, the business is on to them, Goonrey says.
"There is technology around now that can tell how fast you log on to your computer right up to the notorious case of Amazon which issues its staff with wrist bands to monitor their actions."
Data, of course, is neutral. It's how it is used and abused that has become the issue of the moment. Recent data breaches and covert harvesting of people's personal information at companies such as Facebook and Google, has led to urgent calls for regulation in the US and the UK, and made people think twice about handing over their personal information quite so readily.
The Australian government's response in a climate of growing unease around data breaches has been to introduce new changes to the Privacy Act which came into force in February.
The new rules mean that notifiable data breaches – information that is compromised, stolen or breached – must be reported if it constitutes "serious harm" to customers or employees.
What constitutes serious harm is obviously something organisations will need to work out. Meanwhile, business continues to grasp a data pyramid whose peak is always out of reach. What would the Egyptians have made of it?