“The Ins And Outs Of Shootouts”

I have been doing bike shootouts for 17 years! 17 years!

Lead Photo: Dirt Bike Mag

I have ridden every motocross bike that has been made and have compared them against each other for quite sometime. Now that my time has come and gone with Dirt Rider, I wanted to give you all the real scoop on what it’s like to plan, put one of these suckers on and give you the ins and outs of it all. I have only been planning, running and testing Shootouts for six years (the other 11 years I was just a test rider) and let me tell you, there is a ton of moving parts that come with a shootout.

I have been through three editors in my time at the magazine from the years 2000-2017 (actually four, but I am not counting the last one) and each one of them had their different styles. I learned from each one of them and took away bits and pieces of things they did that I liked. Each one of them had their own unique traits about them and all would be involved in the shootout test results process somehow.

Ken Faught was great guy and although I felt was more of an entrepreneur than a hard core dirt bike rider, he always treated me well. Ken did shootouts like Hollywood does movies. There would be tons of days comparing them all at several different tracks (which was good), but along with that we would have TV crews out there (for a series called Dirt Rider Adventures) that was televised on the Outdoor Channel. It was a huge production, but I felt it took some luster out of the shootout due to trying to get more stuff staged for film than actual testing. Ken wanted our sport to grow and with his connections with important people in the TV world shootouts came to life for all of you at home. It was tough to get all the things done for TV and test in the same day. So in the end sometimes the actual testing suffered somewhat.

Jimmy Lewis came into Dirt Rider and brought more of that hard core racer/rider type of mentality and brought in a lot of technical numbers to our shootouts back then. We would go to four or five tracks and test, but would also do long 4th gear tapped straight radar runs, would have 2-3 lap races on each machine (against all the other test riders), perform dyno runs and we could even change the bike around more for our liking, with different spring rates, gearing and even what tires we preferred. If you weren’t tough than you weren’t going to be invited back to another one of Jimmy’s tests, I can vouch for that. The mood was somewhat too serious at times and could take some of the fun out of what we were doing. I mean riding and testing is fun still right?

Shootout Bike Setups are always full of fresh shiny color. Justin Dawes Photo

Lastly, there is Chris Denison. Chris came from a background of all different aspects of riding. He raced off-road, motocross, did freestyle shows, extreme enduros, and even trials events. There wasn’t one thing this Editor wouldn’t or couldn’t do. If someone asked him to try any type of racing he was ALL IN! Not only did he make testing fun and exciting for me again, but his shootouts were easy to read/follow. After we received our new model test units he made the staff ride all of the bikes until we had 10 hours of engine time on each of them. They would then be parked in the shop until a member of the each manufacturer could come down to our office/shop and perform a tune up (tires, clutch, filter ,etc). This was so that no bike cold be “tinkered” with back in the confines of the manufacturers shop. That’s right, CD wanted to make sure that no bike would get any “extra love” when we weren’t looking. I will have to say that there hasn’t been any of this “cheating” in quite sometime as it was more or less put to bed in the Jimmy Lewis era. In the Ken Fought era though I felt some of our test bikes were better than some of other dealership purchased models I rode back to back.

When I came on staff Chris Denison knew my strengths in the industry, so he immediately gave the responsibility of shootouts to me. I wanted to bring some internal manufacturer testing that I learned (when I was at Yamaha) to some of our magazine tests, so I began making a test sheet. This test sheet was created for each rider so he or she could write all of their notes and give a score to each category, that I created about the machine. This sheet consists of over 70 points of the motorcycle to rank and also has a comment section to add notes on each category. Lastly, each tester has a overall ranking chart and must explain why he or she ranks it in that position. This was the most thorough sheet I could come up with that used a “real manufacturer production evaluation” type of score card. With this type of scoring it’s easier to see why and by how much a certain part of the motorcycle is better than others. It’s a much more tedious testing sheet, but really dissects each bike’s weaknesses. This way I am not worrying about testers notes and if they forget to mention something that is important to the reader. It’s all there and must be filled out. I then must pick a wide variety of test riders that can evaluate, write, ride and are trustworthy. This is not easy people! Seriously! It’s like finding a unicorn. Some people can ride, but they can’t write. Or it’s the other way around. Finding both is tough to do and I have kept some of those people with me at keeferinctesting.com. After finding the right riders I must order six different sets of gear and get it ready for them to shoot photos in, find the correct tracks to go to, rent out a track for photos and video, get a videographer and photographer, make calls to each manufacturer to make sure I am not booking test dates on top of other media outlets, go grocery shopping for the riders, order backgrounds for the test bikes and make sure each unit is prepped and ready to go.

Another vivid shootout setup. Kit Palmer Photo

As far as the test bikes themselves I usually had the manufacturers come to the shop to perform their prep and if they couldn’t make it I would prep the bike myself. A big debate with each manufacturer is if we would use the tires that come on the bike stock or pick one brand of tire and use the same tire for every bike. I have done both, but am a believer that you use what comes on the bike stock. However, in the magazine world you have to please your advertisers and sometimes those advertisers are tire companies. Some of those tire companies that we were required to use didn’t come on any of the test bikes and on one occasion we found ourselves chasing our tail with traction on all of the bikes. Moving forward with my own shootouts I will be using what comes on the bike stock over at Keefer Inc.Testing.

The first day of the shootout is usually dedicated to photos and video so that doesn’t get in the way of actual testing days. It’s a nightmare to get everything done and test in the same day so we make that a separate day. Once that day is over I will dedicate at least four days (at different tracks) to testing the bikes. During those four days only two of those days we will have manufacturer support. Each manufacturer comes out to help with sag settings and any other need the test rider might have. We can change some things around n the bike if it’s necessary, but nothing major will be swapped in the shootout. Each tester WILL NOT talk about their sheet to one another and all the testers I use know if they do talk, they are subject to ejection. It’s like fight club. The first rule of testing is you don’t talk about your testing. If I feel like there are a couple bikes that are close enough to have a fifth day of testing than we will do so. After four days of testing however you get a real good idea on what each bike characters are and can make a educated decision on what you like the best. At the end of our last day we will usually do a wrap up video and get all the testers together to talk about each machine and which bike they thought was the best.

After the shootout is done I will go over pages and pages of notes/scores then tally up the overall ranking. The lowest number score wins. In the 2017 450 Shootout at Dirt Rider I was injured and couldn’t be a part of the testing, but as I was adding the score up the Honda CRF450R was winning with only three sheets to go. In the end, the YZ450F won by two points and that is how close it was for the overall. It didn’t win by me loving or blowing the Yamaha YZ450F, but by other individuals scores and notes. Since my first day at Dirt Rider there was never a “purchased” test or ranking. I know a lot of the rumors are that advertisers buy good reviews and that might be true at some other magazine outlets, but if I was a part of it, there would be none of that. Because I have been vocal about how much I like that bike, I was actually stressing out that the Yamaha YZ450F won the Dirt Rider Shootout. I knew some people in the industry would think that I made that overall score happen. At the end of the day the only thing I have to offer is my integrity on my tests and I am simply not willing to lose that. What might have saved me though was that I had a concussion at the time of the shootout and I never got to ride all of the bikes back to back, until after the shootout was released.

After the scores are tallied up is when the fun 8000 word document begins. Typing and putting together a shootout, so it’s easy to read is somewhat tricky and I usually try to break it up over the course of several days. Weighing the units is also a thing of discussion as some machines have larger fuel capacities, so weights can vary somewhat. In the future, I will be weighing each bike with same amount of fuel, so that the weights will all be apples to apples. I also get people asking my why I don’t believe in dyno graphs. I think dyno results are cool to look at, but do very little to explain the way the bike feels on the track. I have had many test days with engineers and their dyno sheets saying what is “best”. I would go test one set up, come back and tell them it was shit and they looked at me like I was an alien. They would say “well the dyno says this is the best curve there is out of all of the set ups”. I would argue until I am blue in the face and told them to go ride and see for themselves. After riding each configuration, they would come back and agree with a puzzled look on their face, knowing that I was being honest. So with that being said, I feel like dyno graphs are not the end all be all of what bike is the fastest and most rideable on the track. This is why I leave it out completely. Now that is not to say I will never incorporate some bike tests that have dyno findings, but with shootouts I like to go off of actual performance on the track because that is “real world testing”.

If you’re in the market for a new bike, shootouts can be your guide to the right purchase if they are done correctly. If you trust the source you’re getting your information from and you can relate to one of the test riders (in ability, height and weight) it’s a great purchasing tool. It’s good to read what each bike’s character is and how that can relate to your riding style. All of these new bikes are expensive, so take the time to read and trust the source you’re getting your info from. People can hate on Jody at MXA all they want, but I do know that he and his guys ride the shit out of all their test bikes. Routinely it was Dirt Rider and MXA with the most hours on each test bike when they got returned to the manufacturer at the end of the year. Now that doesn’t mean that all of his test riders are good, but at least you know they are actually being ridden. I do like that part of it.

September is just around the corner and with Yamaha and Suzuki coming out with all new 450’s it could be interesting to see how they stack up against a KTM 450 SX-F that is pretty damn good. Honda is also rumored to be coming out with a brand new CRF250R along with Kawasaki refining their ever improving engine character on the KX250F, so Yamaha and KTM might have some competition come 2018 in the 250 class. Stay tuned to pulpmx.com and keeferinctesting.com for the no bullshit tests that I hope you come to check out.