We end up finding out our strategy is more than 50% accurate in its predictions, but, given our particular example, we also need to know the "degree" of our accuracy. We need to know how well out correct out-perfoming predictions actually perform, and how poorly our incorrect predictions play out.
So now what we're going to do is build a script that will actually play out an investing scenario, using our machine learning algorithm. If you recall earlier, we just trained with most of the samples and tested against some at the end, that's basically what we're going to do here.
Our strategy for investing is pretty simple, and could be improved, but the basics of it go as follows:
Using the machine learning algorithm, we make predictions of companies. The learning was to find companies that were likely to out-perform "the market" (the S&P 500), over the course of one year.
Thus, to back-test this, we first train on a batch of data, then we test against another batch of data that is different than the data we trained on. So we run through some data, and we find companies that the machine learning algorithm suggests we buy. We then take those companies, and measure their outcomes one-year later. We add all of these outcomes up, as if we really had invested, and we see how we did.
import numpy as np import matplotlib.pyplot as plt from sklearn import svm, preprocessing import pandas as pd from matplotlib import style import statistics style.use("ggplot") FEATURES = ['DE Ratio', 'Trailing P/E', 'Price/Sales', 'Price/Book', 'Profit Margin', 'Operating Margin', 'Return on Assets', 'Return on Equity', 'Revenue Per Share', 'Market Cap', 'Enterprise Value', 'Forward P/E', 'PEG Ratio', 'Enterprise Value/Revenue', 'Enterprise Value/EBITDA', 'Revenue', 'Gross Profit', 'EBITDA', 'Net Income Avl to Common ', 'Diluted EPS', 'Earnings Growth', 'Revenue Growth', 'Total Cash', 'Total Cash Per Share', 'Total Debt', 'Current Ratio', 'Book Value Per Share', 'Cash Flow', 'Beta', 'Held by Insiders', 'Held by Institutions', 'Shares Short (as of', 'Short Ratio', 'Short % of Float', 'Shares Short (prior '] def Build_Data_Set(): data_df = pd.DataFrame.from_csv("key_stats_acc_perf_NO_NA.csv") #data_df = data_df[:100] data_df = data_df.reindex(np.random.permutation(data_df.index)) data_df = data_df.replace("NaN",0).replace("N/A",0) X = np.array(data_df[FEATURES].values)#.tolist()) y = (data_df["Status"] .replace("underperform",0) .replace("outperform",1) .values.tolist()) X = preprocessing.scale(X) Z = np.array(data_df[["stock_p_change","sp500_p_change"]]) return X,y,Z def Analysis(): test_size = 1000 invest_amount = 10000 total_invests = 0 if_market = 0 if_strat = 0 X, y, Z = Build_Data_Set() print(len(X)) clf = svm.SVC(kernel="linear", C= 1.0) clf.fit(X[:-test_size],y[:-test_size]) correct_count = 0 for x in range(1, test_size+1): if clf.predict(X[-x]) == y[-x]: correct_count += 1 if clf.predict(X[-x]) == 1: invest_return = invest_amount + (invest_amount * (Z[-x]/100)) market_return = invest_amount + (invest_amount * (Z[-x]/100)) total_invests += 1 if_market += market_return if_strat += invest_return print("Accuracy:", (correct_count/test_size) * 100.00) print("Total Trades:", total_invests) print("Ending with Strategy:",if_strat) print("Ending with Market:",if_market) compared = ((if_strat - if_market) / if_market) * 100.0 do_nothing = total_invests * invest_amount avg_market = ((if_market - do_nothing) / do_nothing) * 100.0 avg_strat = ((if_strat - do_nothing) / do_nothing) * 100.0 print("Compared to market, we earn",str(compared)+"% more") print("Average investment return:", str(avg_strat)+"%") print("Average market return:", str(avg_market)+"%") Analysis()