Yellowstone Angler's new shootout: 8 weights
- This topic has 33 replies, 15 voices, and was last updated Feb 14, 2014 at 6:08 am by
.
-
Posts
-
Yes, here they are again:
Yellowstone Angler’s yearly shootout. This time they are looking at 8 weight rods.
http://www.yellowstoneangler.com/gear-review/2014-8-weight-shootout-g-loomis-nrx-orvis-h2-sage-method-sage-one-tfo-bvk-scott-s4s-beulah-opal-best-saltwater-fly-rod-fly-rod-comparisonThe usual suspects are in the top rankings.
What did surprise me is that the Sage Method didn’t do as good as I expected. Probably what I was afraid of: too damn stiff. Another surprise was the Hardy rod. As George says “it sucks!..”Okay, let’s talk!
Hey Jay –
First off, I appreciate the post. I don’t want to seem like I’m trashing you personally and I know you’re just contributing to the discussion.
But, this is my standard response to these tests:
You guys remember the concept of significant digits in chemistry class? The idea was you can’t take a measurement like, say 1.025 liters of water, and multiple that by another measurement of 1.1 liters of water. The actual math there works out to be 1.1275 liters. The problem is, you measured one glass to the thousandth of a liter and the other glass only to the tenth of a liter, and yet your results appear to give you certainty to the ten-thousandth of a liter. You can’t have that kind of certainty based on measurements that are vaguer than your result. Instead you have to throw out the last two digits from your first measurement, because they are not “significant digits,” before you do the math. The only measurements you can rely on are the measurements down to the tenth place, and so this problem works out to a 1.1 liter “significant digit” result.
These tests are the same way: they are only as reliable as their least-reliable component. You can add all the numbers in the world to the end of the results but it doesn’t change the inherent subjectivity of the tests. This is what Stephen Colbert calls “science-y.” I see people try to pull crap like this in litigation a lot, for example in crash test analysis. They extrapolate from speculations or assumed data and then act like the result is hard science. In the law, we have a process called a Daubert test where the judge evaluates the underlying science to determine whether the process is duplicable, peer-reviewed, and makes accurate use of the scientific method. If not, the expert’s testimony isn’t allowed into court.
These shootouts would never pass a Daubert test. They are inherently unreliable. Their only value is to get the opinions of a group of people who presumably (but not necessarily) are expert casters. That has some value, sure, but not enough to rank and divide and name winners. And nothing about George Anderson or his staff of testers is superior to the local fly shop or local guides in whatever area you happen to be in. Most importantly, unless you are a true beginner, none of their opinions are really superior to your own in picking out a rod–you’re better off test casting a bunch and naming your own winner.
And all of that omits the question of intentional bias, since the “tester” also sells some of the rods but not others. I don’t even bother to read these any more, personally.
Zach
Zach,
You’re not being fair here, I don’t think there was any pretense of science in the shootout, it’s a review carried out by four experienced fishermen who express their opinions and score the rods accordingly then add them up and declare a hierarchy. It’s no different than a comparison of cars or any other consumer item where there are a lot subjective issues to deal with. All in all I think they do a pretty good job, they certainly identify some dogs, and when four people unamimously score the top contenders so closely, there has to be some legitimacy in the results. And, for what it’s worth, I don’t think there is any way that fly rods could be scientifically evaluated in a way that would help someone figure out if they were better or worse casting or fishing rods.
JCI agree with Jon on this one. I have never been able to figure out why people are so insistent on bashing the Yellowstone Angler’s shootouts. Sure, there is a lot of subjectivity in them, but at least they have experienced anglers trying them all out at the same time and under the same conditions. Very few people have the resources to try out all 24 rods, plus those that weren’t included. If you were looking for a rod it is sure better to read through the Shootout than it is reading random reviews of all the rods online or reading marketing hype. That way you could narrow it down to the few rods that best fit your criteria and go try them out.
I don’t believe “significant digits” really applies in this case. That is a math convention applied to objectives mathmatical equations. These tests are based on a lot of subjective opinion and should be viewed that way.
It would also be interesting to see if they would pass the Daubert test. Obviously they couldn’t since the testers wouldn’t be qualified as professional experts in a court of law. Putting that aside, I think it might. I do a lot of expert testimony and use subjective opinion in my analysis. As long as the analysis is reasonable and consistent with industry standards, it passes the Daubert test. I think you could make the case that the shootout is based on reasonable criteria and methodology. Sure you could have a different set of experts come to a different opinion, but I face that in court all the time and neither party is questioned on Daubert.
Those are interesting perspectives guys, and they may have ironed out some of the issues with their tests. But historically, these reviews were really, really bad about putting rods *not* sold by the shop at the bottom end of the review scale versus those which were sold by the shop.
The main issue for me was what I guess you would call the insider knowledge that a while back some of the rods in the test were literally the same blanks (imported from Korea) painted in different colors, which were given drastically different scores. Those kinds of results seriously undermine whatever legitimacy the test may have had even as a subjective set of analyses.
To use an analogy: this is like when they test wine experts by putting the exact same wine in two different bottles and the “experts” subjectively score the wine in the nicer Grand Cru bottle a 93 and the exact same wine in a regular bottle a 78. The fact that different manufacturers were using the same blanks was never acknowledged so of course no one could step forward and say, hey, you guys just got duped by a paint job, but that is essentially what was happening.
Zach
Insider information is not exactly vetted reliable info, and could be just a rumor started by a disgruntled loser. In the early tests they did flex profiles of all rods, tracing the arcs on the wall, if there were a bunch of identical blanks, it would have showed up.
JCThey may not pass this Daubert test you speak of, Zach but they could all give expert testimony that would certainly sway a jury. And they are all what you recommend as they are a local fly shop, guides, and really good fishermen. I have personally guided George Anderson and I can assure I was the one learning that day. Plus from the get go they make it abundantly clear it is only their opinion. I think Jay was looking for a fun discussion which is lacking around here these days, bro and you kind of squashed it…
You guys may be right. I am personally very skeptical of these shootouts for the reasons I’ve stated. However mine is just an opinion like any other, and maybe worth less than some.
Zach
I’m on Zach’s side. These tests are interesting and I do usually read them, but if you are using this information to decide on a rod purchase then you are shorting yourself.
Nobody can pick the perfect rod for you just like nobody can pick the perfect shoes for you. You have to pick what fits and feels good to you. I have exactly one rod that I absolutely love out of all of them that I own. It also happens to be an 8wt that is nowhere to be found on this list. I found it because I went searching for the best (for my casting abilities) 8wt that I could find. I cast several rods on this list, 2 in the top 10, but the one I chose outperformed (with my casting abilities) them all by a longshot.
I just don’t think you can judge something like this when everyone has a different expectation from a rod. And if you don’t feel that you should have a different expectation then these guys then maybe you should try to figure out where you went wrong. I don’t think Zach is saying that these guys are doing a poor job, they obviously put some time into it, but it’s just their opinion. Some guys like fast rods, some guys like slow rods, some in between. It’s just makes me sad that someone might buy an $850 dollar rod based off this test when maybe they would have been happier with the $250 rod at #3. Which is an awesome rod by the way and almost won my own comparison.
T I think Jay was looking for a fun discussion which is lacking around here these days, bro and you kind of squashed it…
ZING
Some years ago, three of us did an evaluation in Fly Rod & Reel of 8-wt rods for largemouth fishing. While we didn’t always agree that one specific rod was better than another, we did pretty much agree that the rods could fit into three groups, and which rods fit into those good/great, OK to fish with, and thanks but I’ll pass groups.
What we also knew, and quickly confirmed, is that an 8-wt that casts long, tight loops with a piece of yarn on the end of the leader might cast like a rug beater when a poofy bass bug is tied on – – – particularly at moderate distances (30-70 feet), and when considering the number of false casts needed to get things going when the bug is stripped up to the boat/bank.
And, as George’s shootout noted, some rods performed much better with 9-wt lines.
Same thing with sinking lines/heads: some rods that do well with floaters do just as well with sinkers, but some don’t.
I enjoy reading such comparisons, partly for the information, and yes, partly to see if I can pick out any shortcomings. George, who I’ve known for years and who is a superb and versatile angler (and a fish predator if ever there was one), does really hammer on rods lacking tubes. Yet virtually everybody I know who carries more than a couple of rods on a trip uses one of the many multi-rod carriers. I think the shootout has that backwards. Instead of suggesting those rods ought to be priced $30 higher to be “even”, I’d suggest all the others are overpriced by $30, because most anglers would rather buy the rod w/o a tube.
Good entertainment!
Buzz
I like rod shoot-outs.
Maybe it’s hard to sort out bias or commercial interests, but, if nothing else, it starts the conversation.
We should be discussing fly rods no-stop.
Also, fly shops should IMHO try before you buy and have an idea about what they’re selling beyond margins.http://www.flylife.com.au/library/tackrev/49/FL49-%20Rod%20Rage.pdf
Here’s a six weight rod shoot-out I organised a few years ago with over 30 six weights and 100 testers.
I found it a very interesting exercise not so much for winners and losers, but seeing how casters of different skill levels judged things.www.dsaphoto.com
A picture is thousand words that takes less than a second while a thousand words is a picture that takes a month.
Zach, so with you on this one…
There is absolutely no like for like in this test, NONE what-so-ever.
eg 1. The Method is 7w loaded with an 8 weight line???
eg 2. Then they use a Method Elite for the actual 8w, and mark it down because it is so expensive??Swing weight??
This is a completely subjective thing, one persons preferred swing weight is different from another’s…What is the “perfect 8” score?? I do a lot of casting lessons, and after you’ve been casting all day, you can pick up any rod and just chuck it… does that make it the perfect rod? no, it means you have been working on timing and form all day, so you suddenly cast a stick and it is a perfect rod… no man.
And whats this fun to fish/got to have??? its just a point system so you can say that one is better than the other… HUH??
Let alone the brands they stock, recommend and sell and fish…
I have more, but I must rather stop… my 2 cents.
@David Anderson, much win in your article!!
I read them for confirmation that the enormously expense rod I just bought is, in fact, the best rod in the world. If they rate my rod low, then I attribute the rating to the author’s bias and stupidity, especially when I know I can find a comparison article somewhere else that says my rod is the best. In either event, the magazine succeeded in getting my attention and probably my money.
David, I did enjoy your article. Thanks for posting.
<div class=”d4p-bbt-quote-title”>Joel Thompson wrote:</div>
T I think Jay was looking for a fun discussion which is lacking around here these days, bro and you kind of squashed it…ZING
I should apologize for that comment. Probably a bit harsh. Sorry Zach.
Joel
I think these shootouts are a joke. I think Zach hit the nail on the head. To me they are a commercial for whatever they want to sell that year. One needs only to look at their deflection charts from shootouts past to see rods with the exact same blank scoring different. I don’t care how good a fisherman He is…this is about sales
What really disturbs me (most of the replies) is how commercial interest of a fly shop comes into the discussion rather than ‘just’ to discuss what we all love: talk about gear.
First Yellowstone Angler is trying to gain/keep attention to their shop to survive. Anyone remember how one after the other fly shop closed their doors the past few years?…
Second, what is wrong points going to brands which put a lot of R&D (no guarantee by the way…) and marketing to make the best rods that can make? And what’s wrong fly shops trying to sell the best rods they can get while still looking out for that one special rod which doesn’t cost $600+?… No fly shop will sell gear which they don’t believe in or did you think they will sell you gear which sucks and expect to consider doing the right job?…
Finally, George is no novice when it comes to rod design and casting. The conclusions in the shootout is his opinion. He and his men who did the testing are probably a few levels in terms of casting above the average fly fishermen out there…
One last point:
We owe George a big thank you for his efforts in the fly fishing history.So much for my effort to have a lively discussion here… I’ll think twice before doing such a thing next time.
- You must be logged in to reply to this topic.