Spotlight on research: Pulling signal from the noise – mining digital information for smart uses
Professor Noel O'Connor's work on data analytics seeks to capture important information from larger datasets with a view to smarter sports, transport and media.
On September 4th, 2016, the All-Ireland Senior Hurling final in Croke Park was a thrilling clash of titans. The roar of the crowd was immense, but just how loud was each camp? Thanks to sensors and data analytics powered by a unique collaboration between DCU, Intel, Microsoft and the GAA, we know that the Tipperary fans, like their team, were on winning form and out-shouted the Kilkenny supporters.
Detecting the decibels and attributing them to specific groups of fans in the crowd during play is an example of Smart Stadium technology being developed at Croke Park, explains project co-ordinator Professor Noel O’Connor, Director of DCU’s Information Technology and the Digital Society Hub. “It is part of DCU’s broader offering in the ‘Internet of Things’ where devices collect data and enable us to see a bigger picture,” he says.
“We are a trailblazer, a thought leader in Ireland in that space.”
Stadium learning
The Smart Stadium is a crucible for deploying a network of sensors such as microphones, video cameras and wearable devices, then analysing the data. That might mean automatically mining into video footage to track where crowds are moving, so that their flow can be managed more efficiently around the stadium. It might mean watching the all-important grass growing on the pitch over time, and advising the best course of action to optimise it. Or it might mean sending live statistics and updates to fans on their smartphones during the match, such as how far an individual player has run, or how much possession their team has had.
The project at Croke Park, which involves Arizona State University, Microsoft, Intel, the GAA and several other partners, has obvious applications for safety and entertainment at sports events, but it serves as a test bed for technology use in ‘smart cities’, notes Professor O’Connor, who is a Principal Investigator with the Insight Centre for Data Analytics. “The common core here is deep learning,” he says. “We want to train machines to better understand the data that sensors and other devices are giving us.”
Mining images
That goal is a common thread through Professor O’Connor’s research, which includes helping computers sift through mountains of media content.
Professor O’Connor has worked with the BBC, European media outlets and even security companies to help guide them through video archives. “Our software can make all those archives more searchable, so humans can find the information they are actually looking for,” he explains.
Photography collections can also benefit: working with The Irish Times, Professor O’Connor’s group is developing software to find the sweet spot in still images. “If a human has to scan through banks of images and then select one and crop it to highlight the interesting portion, that is time consuming and laborious” he says.
“Our software can analyse images and detect the salient parts, which might be a face or an object, and then automatically crop the images. It doesn’t replace the human eye, but it lightens the workload considerably.” Increasingly, our cars will need to do the seeing as they become more autonomous, and again Professor O’Connor’s research can smooth that process.
“Driverless cars need to detect pedestrians and other cars, road markings and traffic lights,” he explains. “The car’s software needs to learn how to do that, and our research is helping the computer learn how to pull out the relevant information from what its cameras and other sensors detect in its environment.”
Open mind
Professor O’Connor’s approach to digital content analysis also sees him work with researchers who want to monitor the environment for changes and pollution and who are encouraging people to exercise after illness and in older age.
He enjoys collaborating widely, and sees it as a necessary part of making the best use of technology. “You can’t solve societal challenges on your own, you need to work with the exercise physiologists, the ethicists, the environmentalists, the journalists, the enterprises – only by working together can we see how our research and innovations can have a real impact.”
Professor O’Connor is also aware that ideas can come back into vogue when the time is right. “The deep learning nets we are working with now are based around artificial neural networks, which were for a long time considered old hat. But now that computational power has increased, we can use them more efficiently to enable machine learning. So you should never write off an area of research, and you should keep an eye on what people in other fields are doing.”