Wes Boyd: The idea [of a review standard] has potential, but there is a big pitfall -- even if we try to account for it, there still isn't any way we can account for the non-level playing field of various reviewers. We have in the past talked about the Paddling.net reviews. The problem is that you have an experienced and highly skilled paddler reviewing, say, a Nordkapp, just to pull a boat out of mid air. He has some bones to pick with it, maybe stability, maybe something else, but thinks it's pretty good, so gives it a 7. An inexperienced paddler buys his first boat, say, a Perception Vizcaya, and thinks its the greatest thing since canned beer, so rates it as a 10. The end result is that the Vizcaya is skewed high, and the Nordkapp low. Swap paddlers -- the Nordkapp now gets a 2, because the inexperienced guy can't stay in it. The experienced guy gives the Vizcaya a 2, by comparing it to the Nordkapp. Ralph Hoehn: In my experience the only successful method of neutralizing the above described inevitably skewed subjectivity, is to have each reviewer evaluate certain standard criteria comparatively (initial stability, final stability ... however defined ... speed, glide, acceleration, tracking, maneuverability -- nothing unnecessarily / highly scientific required!) and to rank the tested boats for each criterion independently. In order to get a somewhat objective picture, further processing of the results is then required, again by ranking. This procedure does not solve the subjectivity problem completely, if merely because comparisons by feel will not be accurate. The effect of this shortcoming is reduced the larger the sample, though. This approach at least tends in the right direction. It does leave us with another problem: Who will undertake to do the processing? This kind of comparative reviewing works best if more than two boats are included for each reviewer per review session (but, then again, not too many or this highly subjective process gets too hazy!!). Also, it allows comparison of normally incompatible boats since we are not comparing the whole boat, but only certain individual characteristics. How does this lead to providing a "potential buyers' guide"? By weighting, in subsequent processing of the raw data, certain criteria in relation to their applicability to certain envisioned end-uses. In very much simplified terms the product of the weighting and the ranking will produce a ranked list of all the reviewed boats' suitability for certain purposes. In reality this is a fairly labor intensive process. Is this worth the effort to enough people to get it off the ground? Does anyone have a simpler approach (without sacrificing quality and validity of the results)? Best regards, Ralph Ralph_at_PouchBoats.com www.PouchBoats.com *************************************************************************** PaddleWise Paddling Mailing List - Any opinions or suggestions expressed here are solely those of the writer(s). You must assume the entire responsibility for reliance upon them. All postings copyright the author. Submissions: PaddleWise_at_PaddleWise.net Subscriptions: PaddleWise-request_at_PaddleWise.net Website: http://www.paddlewise.net/ ***************************************************************************Received on Fri May 11 2001 - 15:37:05 PDT
This archive was generated by hypermail 2.4.0 : Thu Aug 21 2025 - 16:30:42 PDT