Experiment

Experiment consists of three parts.

Experiment pipeline

Data preparation

Getting the list of stocks (S&P 500 Index) from Wikipedia.

Receiving prices (High, Low, Close) of each day for last 3 years.

Removing stocks that have missing values in data.

Code block of this part:

Downloading and cleaning the stocks

As a result of this part is the dataset contained 470 stocks.

Brick size optimization and Renko chart evaluation

This part is responsible for Renko chart evaluation. The optimal brick size is determining on train set which contains 70% of days. Renko chart evaluation is carrying out on the rest 30% of days using score function.

Train test split method

These both operations (optimization and evaluation) are performing for two approaches (ATR and score function optimization). In ATR case, the last ATR value of sequence is considered to be optimal brick size.

In score function optimization case, the optimal brick size is which give a maximum value of score function.

Score(brick_size) plot

This approach uses one-dimensional function minimization based on Brent’s method. Score function with minus sign should be used because it is inverse problem.

This code contains the function that should be optimized, and scoring / optimization process:

Optimization process

Brent’s method is a root-finding algorithm combining the bisection method, the secant method and inverse quadratic interpolation. It has the reliability of bisection but it can be as quick as some of the less-reliable methods. The algorithm tries to use the potentially fast-converging secant method or inverse quadratic interpolation if possible, but it falls back to the more robust bisection method if necessary. Boundary points can be given as lowest and highest ATR values on train set, it would be good as an initial approximation. More information about Brent’s method here.

Brent’s method visualization

Brent’s method algorithm

Let’s get the list of ordering stocks by score values for two approaches. Score values are calculated on test set.

Draw the Renko charts of stocks that have best scores (AVY stock for ATR, AFL stock for score function optimization).

Renko charts of highest score stocks

Code block of result visualization:

Result visualization

Analysis of result

This scatter plot demonstrates how price_ratio corresponds to balance:

Scatter plots

Visualization of how price_ratio corresponds to balance

We can conclude that score function optimization approach compresses the data better because price_ratio has the tendency to be greater.

Draw a histogram that shows how many positive / neutral / negative results of stocks on test set: positive: score > 0; neutral: score = 0; negative: score < 0.

Code that counts and visualizes the result:

Positive / Neutral / Negative histogram

Histogram of result distribution for two approaches

Frequencies have some differences, but the shapes of distributions are similar. Let’s show a score histogram of two approaches, and distribution of their difference:

Score histograms

Score histogram for two approaches

Score difference histogram

Will be good to estimate a confidence interval of score difference. This would make it clear which approach on an average gives a better score.

Test the null hypothesis that a sample comes from a normal distribution. The p-value is 1.085e-11, that is very low, it means that sample comes from non-normal distribution:

Test on normal distribution

It is unacceptable to calculate a confidence interval on non-normal distribution.

We can use the bootstrapping to get a confidence interval. Bootstrapping is practical computer method for studying the distribution of statistics based on multiple generations of samples by the Monte Carlo method based on the available sample. Allows you to quickly and easily evaluate a variety of statistics (confidence intervals, variance, correlation, etc.) for complex models.

Boostrapping alogorithm

Bootstrapped sample is shown below. This sample is distributed according to the normal distribution, p-value is 0.9816.

Code block of resampling:

Bootstrapping

Bootstrapped sample of score difference

Almost the all mean values are greater than 0. The p-value of t-test is 0.0 (null-hypothesis: score_diff = 0.0), we reject the null-hypothesis. These calculations confirm that average score difference between score function optimization and ATR approaches is greater for the proposed one.

This is particular example of the difference in approaches (UNH stock).