Week 15 - Future of Data Warehouse, Data Mining and Data Visualisation

The last of our BI lecture talks about the future of data warehousing, data mining and data visualisation. This week, I have learnt what to expect from the BI world. Understand the future of BI allows us to look forward to the development of BI.

Data Warehouse (DW)

Future of Data Warehouse:

• Coming up with a workable set of rules that ensure privacy of data as well as facilitating the use of large data set (Also a challenge)
• Store unstructured data such as multimedia, maps and sounds

Future of data warehousing will not be a high performance disk storage but rather an array of alternative storage.

There are four reasons of using an alternative storage:

1. Data in a DW are stable. They are placed there once and left alone, hence it does not need to be updated at high speed
   
2. Queries operating a DW often require long streams of data stored sequentially
3. DW is of indeterminate size and is always increasing in volume, requiring flexible capacity
4. When data gets accessed less often as it ages, it can be moved to secondary storage, making access to newer data more efficient

Now the trend of data warehousing is about getting data readily available, accessible, updated and fast implementation. It can be expected in future that more organisation will build Web applications that operate in conjunction with the DW.

Patrick (2005) mentioned that organisation will be trying to integrate DW with their ERP system so that their DW is updated real-time. This is also one of the DW trend going in the corporate world.

Data Mining

Data mining is about discovery insights or underlying business values from the sets of data. It has helped marketers in successfully launching of campaigns, cut costs for the company and so on. However, before mining, data has to be integrated, transformed and cleanse. Besides that, there is also concern of privacy.

Internet has grown rapidly throughout the years, so as the problems of network intrusion. Those network intrusion attacks on the vulnerability of the system and will be able to get hold of the data.

Other than using data mining in business, it can also be used to identify patterns of valid network activity.

The trends of data mining from the lecture slides are:

• Next generation Internet will connect sites 100 times faster than current speeds
• Business will react more quickly and offer better service, do it with fewer people and at a lower cost

However, Information Management Magazine (2004) said that the future of data mining will lies on the predictive analysis. This predictive analysis will be realistic about the required complex mixture of business acumen, statistical processing and information technology support as well as the fragility of the resulting predictive model; but make no assumptions about the limits of predictive analytic.

In addition, it mentioned something very interesting saying that data mining technology has not lived up to its promise because "data mining" is a vague and ambiguous term. It overlaps with data profiling, data warehousing and even such approaches to data analysis as online analytic processing (OLAP) and enterprise analytic applications. I agreed to a certain extend during the process of doing my major project. It surely seems that data mining is overlapping other analysis. Anyhow, data mining is a sub-set of BI, that may be why there is overlapping.

Visualisation

Data visualisation links the critical components and enables the smooth flow of information among the components. It is able to conveys technical or complicated things to the management or even to the whole organisation people. It serves as an important way of communicating.

Future Trends
Bounds between computers, graphics and human knowledge will become blurred. Many advances in technology will be needed to handle the visualisation environment of the future. Intelligent file systems and data management software will contend with thousands of coupled storage devices.

Visualisation has many types of forms. Dashboards, google map, evolved arts and etc. This below shows another way of visualising news in a creative way.

One of an innovative way of displaying BBC News Browser (Taken from lecture slide)

I have found one article (in pdf) written by Stephen Few (2007) who is also the author of "Information Dashboard Design: The Effective Visual Communication of Data" (One of the books used as references by our BIT tutors). He wrote the good and bad trends of data visualisation and also the future direction of visualisation. It is an interesting article which I think everyone should go read it.