Ramp Up Your Data-Mining Skills

Have you ever thought about what data mining can do for you? My data-mining expertise is limited to spelling the words (and that would be at risk without a spell checker ). I've read SQL Server Books Online (BOL), done a few demonstrations, perused a few Web sites, but have never done an end-to-end data-mining implementation. Data mining sounds cool, but I'm not sure what to do with it. I'm sure most of you feel the same way. So, what is data mining and how does it compare to traditional business intelligence?

Business intelligence basically means OLAP in today's market. I see OLAP as a presentation and data-aggregation technique that lets me visualize and interact with my data in ways that traditional SQL reporting environments often can't match. But I still need to know what questions to ask to make OLAP effective. A great OLAP tool connected to a world-class OLAP cube lets me browse my data in fascinating ways, but it doesn't magically find my data's interesting highlights for me. In my mind, the treasure of data mining is the ability to do just that. Data mining should automatically tell me what's interesting and important about the data so that I can make better decisions. Data mining should help me sift through the data and find the interesting highlights in the first place. OLAP can help me look at the interesting data once I've found it. Data-mining connoisseurs will probably pick that explanation apart, but it's how I categorize the two topics.

Sounds great! Where do I sign up? Unfortunately, data mining is more often fiction than fact as far as most users of real business systems are concerned. Data-mining technology in today's market simply tends to be too hard and or expensive to use. Most serious data-mining environments require a firm foundation in advanced statistical techniques just to make heads or tails of what the applications are doing.

SQL Server 2000 ships with data-mining capabilities, but most of my customers haven't done much with it. The current product isn't feature-rich enough. The new-and-improved data-mining capabilities in SQL Server 2005 might be the first time that data-mining technology from Microsoft will begin to see mass-market adoption and will be practical for many business to use. Time will tell, but now might be the time to start familiarizing yourself with data mining. I recommend starting with Douglas McDowell's commentary "Data Mining in SQL Server 2005" at http://lists.sqlmag.com/t?ctl=AF6E:7B3DB .

Another great resource for ramping up on data-mining skills is http://lists.sqlmag.com/t?ctl=AF77:7B3DB . People from Microsoft directly associated with the data-mining research and development team launched and run this site. You can sign up for a monthly newsletter and the site offers a ton of great content, including demos, white papers, FAQs, links to other sites, and newsgroups. Why isn't this information at http://lists.sqlmag.com/t?ctl=AF7D:7B3DB if Microsoft people run it? Beats me. Sometimes the legal team at Microsoft can make it prohibitively difficult to publish content without spending months and months in review. That might be the reason for a separate site. In any case, it's a great resource--straight from the folks writing the product-that you won't want to miss out on.