Just a few years ago, if someone said that they “worked with data,” you probably would have pictured a dull, tedious existence—crunching numbers and poring over spreadsheets. Now, Google’s chief economist has predicted that statistician will become the “sexiest” career of the next decade. What changed?
“Masses of data are produced as people and businesses go about their daily lives,” says Stan Matwin, director of Dalhousie’s Institute for Big Data Analytics and Canada Research Chair in Visual Text Analytics. “There is this sort of metaphor used: as people live, they leave behind digital crumbs.” Every time you make a phone call, post a photo online or buy something at a shop, you are contributing to “big data,” the term used to describe massive, complex sets of data.
For a statistic that shows just how enormous the field has grown, consider that two years ago, 90 percent of today’s data didn’t exist yet—every piece of information we had from the beginning of time only amounts to a small fraction of what we have now. We are living through the so-called “Industrial Revolution” of data thanks to technologies like social media, cell phones, e-commerce, GPS signals and countless other sources, which altogether collect 2.5 quintillion bytes of data every day.
Although big data has recently become analogous with “Big Brother”—surveillance and privacy concerns have dominated the news this year—there are unlimited positives that can come from mining big data. The practice has great potential to improve the world if significant trends and patterns are discovered.
“We’re trying to use this data that already exists to help people and organizations fulfill their missions better,” explains Matwin. Evaluating data can help us find new sources of economic revenue, fight crime, reduce energy consumption, put a stop to diseases and deliver better health care.
For big data to have any real impact, decision-makers need to understand what it means. Naturally there is a need for people to discover trends and patterns within the data, and communicate it in a way that resonates with people from all backgrounds.
Data skills shortage
A study by the McKinsey Global Institute found that “by 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions.”
“Data is exploding,” says Amir Asif, a computer science and engineering professor at York University and the principal investigator for the Centre for Innovation in Information Visualization and Data-Driven Design, a collaboration between York University, OCAD University and the University of Toronto, funded through the Ontario Research Fund – Research Excellence (ORF-RE) initiative.
“There will be a huge demand of people who can analyze this data, and who can represent the underlying information in a way that people with a non-technical background could understand and interact with,” says Asif. “From my analysis of companies in Ontario there’s a big demand for people with data discovery, design, analytics and visualization skill sets. The banks, media companies, medical companies, environmentalists… they’re all on the lookout for people who have skills in this area. And the need is just going to grow.”
Jobs in big data
The ideal data analyst has skills in mathematics and statistics, computer science and graphic design. However, those working in big data usually work in teams that contribute a combination of these skills. People from these backgrounds come together to mine the data, find the meaning within, and communicate it in a simple way.
At its core, big data analysis and visualization requires computer science skills, but students studying statistics, multimedia and even design can get into the field in varying aspects—data science is interdisciplinary and requires different sets of skills for different parts of the job.
Those working on the computer science side of things are the first in the big data chain—they develop the software and programs that can mine data. Math and statistics students can come into the field of big data as analysts. They are the ones who find significant patterns and trends that, if recognized by decision-makers, have the power to drive change. But first it needs to be visually represented; this is where data visualization comes in.
Data visualization is not just creating tables or infographics—the job is a lot more in-depth than that. It’s combining technology and aesthetics to depict data of massive size. “You need training that would combine the analytical side with the graphical communications side,” says Matwin. “I think that’s an important element of a modern education.” Multimedia and interactive arts programs can help you get into the field.
Because virtually all sectors will need someone to find trends in their data, you must have a natural curiosity to learn about new subjects if you want to work in the field, says Matwin. “It’s not conceivable that you can train someone who can be professionally prepared to interact with people in medicine to ocean science to retail marketing to newspaper publishing. They must have a curiosity about how things work in the world.”
