Data mining for traders is the process of researching large amounts of historical data to find repeatable price action patterns in financial markets. Data mining can be done on all different time frames and both technical and fundamental data can be used to analyze the movements of markets based on different catalysts.
When you mine for data you try to track past moves in relation to different catalysts and identify these patterns to create repeatable trading signals. This is one of the ways traders look to find edges in their trading based on previous recurring patterns caused by human psychology, seasonal patterns, or capital flows into and out of the market.
A trader must assign the probability of whether the patterns in the data are based on a real cause and effect relationship or they are just random in nature. Data mining can both uncover powerful price patterns and also fool a trader with randomness that has no value going forward. The biggest edge in data mining is in finding an edge through creating a high probability risk/reward ratio with a small risk but the chance of a bigger reward or a high wining percentage signal with small losses when wrong.
Quality control can be created in data mining when looking for patterns in the sample data and then looking for it in new out of sample data that was not previously used or by forward testing to see if the data continues to work in real time implementation.
If a trader or investor is not careful about being fooled by their own bias in data-mining it can lead to a lot of errors and false patterns found by optimizing for past data not looking for real edges based on valid principles.
Data mining is very similar to backtesting but expands to looking at not just price action and technical indicators but all kinds of data sets. Having a powerful computer for analyzing huge amounts of data quickly and also speed of execution can be an edge in itself.
