Only primary features are stored in memory to get around the hard copy issue

Should not affect perfromance that much since test_values are rarely used