Corporate IT professionals and executives have descended upon New York City this week for the annual Hadoop World conference to discuss the benefits – and the challenges – of the emerging open source software framework that could reshape the way companies collect and manage data.
According to a 2010 study from IDC and EMC, the "digital universe" is expected to grow 45-fold by 2020. Much of this data will be unstructured, collected from web blogs, social networks, instant messages and online transactions, which poses a challenge for many of today's traditional storage frameworks.
The advantage of Hadoop technology is that is allows companies to collect, share and analyze huge amounts of structured and unstructured data. As a result, more companies, including Yahoo, Facebook and Microsoft, among others, are turning to the framework to manage data in a more effective way at a more affordable cost.
As with any emerging technology, the mere unfamiliarity with Hadoop raises red flags for many businesses. Similar trends have been seen with cloud computing in recent years. As more companies gravitate toward the cloud, many critics question whether the technology has the safeguards in place to protect against data loss and breaches.
The same, to a degree, is true with Hadoop. However, Hadoop has the extra challenge of being open source. According to Larry Feinsmith, managing director of IT operations at JPMorgan Chase, access control and management become major issues when using technology from several different sources, Computerworld reported.
Additionally, Berico Technologies software engineer Richard Clayton stated that analyzing data in a Hadoop environment can create new data sets that also need to be protected.
It's important, Clayton noted, that companies relying on the Hadoop framework utilize security measures that control access to certain data sets and applications. Otherwise, a company may put itself in jeopardy of data theft.
In a 2010 study from research firm Gartner, data center managers identified data growth as the biggest challenge they face today. It is evident that a new solution is needed to address this growth, especially as data becomes increasingly unstructured and difficult to analyze. Hadoop may fill this role – with many professionals now saying the framework is enterprise-ready. However, before committing fully to Hadoop, IT departments must address certain security issues to ensure company information remains protected.
Security News from SimplySecurity.com by Trend Micro