Image provided by: University of Oregon Libraries; Eugene, OR
About Eugene weekly. (Eugene, Oregon) 1993-current | View Entire Issue (Jan. 25, 2018)
BY MORGAN THEOPHIL QUESTIONABLY MEASURING SUCCESS LANE COUNTY AREA SPRAY INFORMATION • Roseburg Resources, 541- 679-3311, plans to aerial and ground spray atrazine, clopyralid, Cleantraxx, glyphosate, hexazinone, imazapyr, indaziflam, sulfometuron methyl, triclopyr with ester, triclopyr with amine, oxyfluorfen and penoxsulam, Forest Crop Oil, W.E.B. Oil, brush & basal oil, Conquer, Crop Oil Concentrate, Forest Crop Oil, petroleum oil, emulsifier, MSO Concentrate, Surface (non-ionic surfactant), Insist 90 and/or No Foam on 1,883.2 acres in many areas southwest of Veneta. See ODF notifications 2018-781- 00809, 2018-781-01090 and 2018-781-01345, call Dan Menk at 541-935-2283 with questions. • Roseburg Resources, 541- 679-3311, plans to hack and squirt and/or spray glyphosate, imazapyr, triclopyr with ester, W.E.B. Oil and/or Forest Crop Oil on 1,542.1 acres in many areas southeast of Florence and 111.3 acres near Canary Road north of Woahink Lake. See ODF notifications 2018-781-00963, 2018-781-00989 and 2018- 781-01075, call Quincy Coons at 541-935-2283 with questions. • Seneca Jones Timber Company, 541-689-1011, plans to spray roadsides near Ferguson Road and Wolf Creek Road with 2,4-D with acid, 2,4-D with amine, 2,4-D with choline, 2,4-D with ester, triclopyr with ester, triclopyr with acid, triclopyr with amine, triclopyr with choline, aminopyralid and/ or MSO Concentrate. See ODF notifications 2018-781-01393 and 2018-781-01542, call Robin Biesecker or Brian Peterson at 541-935-2283 with questions. Compiled by Gary Hale, Forestland Dwellers: forestlanddwellers.org 8 The University of Oregon is consistently evaluated by Academic Analytics, a group critics say uses inaccurate measures of academic performance cademic Analytics, a company that measures the performance, the production and ultimately the success of faculty and overall in- stitutions at the university level, has been collecting data on the University of Oregon since 2013, despite widespread controversy about the group’s accuracy and merit. Under a contract the UO and Academic Analytics signed in 2015, the UO pays near- ly $100,000 annually for Academic Analyt- ics data. Since the company was founded in 2005, a debate has emerged on whether data from Academic Analytics is accurate or useful. In early 2015, the American Association of University Professors (AAUP) released a statement about Academic Analytics and research metrics warning universities to be cautious about the group. The AAUP cited a 2015 study by the Higher Education Funding Council for England that found “indicators can be mis- used or ‘gamed,’ that the data infrastructure underpinning use of metrics and informa- tion about research remains fragmented, with insufficient interoperability between systems; and that ‘it is not currently feasible to assess research outputs or impacts … us- ing quantitative indicators alone.’” Academics Analytics gathers data from universities worldwide. The mission, ac- cording to Tricia Stapleton, the company's chief communications officer, is “to provide university administrators and faculty mem- bers with high quality, accurate data and tools for decision making, collaboration and research promotion to the public.” Among other measurements, Academic Analytics works one-on-one with universi- ties to measure success in terms of produc- tivity among individual faculty members and departments. Data gathered are put into its comparative database — which report- edly has information on more than 270,000 faculty members associated with more than 9,000 Ph.D. programs and 10,000 depart- ments at more than 385 universities global- ly — for universities to compare themselves against others. Brad Shelton, UO executive vice pro- vost of academic operations, says Academic Analytics measures faculty productivity by considering several factors: How many re- search papers has this faculty member pub- lished, where were the papers published, how many times have the papers been cited, and so on. “Those are a set of metrics that very accurately measures the productivity of a math professor, for example,” Shelton says. “It’s not easy to collect, but it’s straightfor- ward. But another discipline, such as mu- A January 25, 2018 • eugeneweekly.com sic, is on a completely different landscape for measuring productivity, and Academic Analytics ends up missing a lot.” Bill Harbaugh, UO economics professor and creator of the blog UO Matters, says he believes the way Academic Analytics measures data “encourages the wrong kind of science. It encourages faculty to publish quickly, publishing in a way not to have greatest intellectual impact but to get the best numbers,” he says. “Faculty will pur- posely split something into two papers that would normally have been one, just to get more numbers of publications.” According to the AAUP, Rutgers Uni- versity signed a $492,500 four-year con- tract with Academic Analytics in 2013. By Even though the data centers significant- ly around individual faculty members, the UO restricts the information to department heads alone, something Harbaugh calls “a major transparency issue.” Robert Berdahl, a longtime UO profes- sor and the interim president of the univer- sity in 2012, works with Academic Ana- lytics on a consulting basis as one of four “senior academic advisors,” Stapleton says. Berdahl is also former president of the As- sociation of American Universities, a sepa- rate group from the AAUP. In early 2016, Gildea presented his in- formation to various faculty and adminis- trators around the university until greater attention was brought to the matter, and ‘I found that the Academic Analytics data were embarrassingly wrong.’ — UO PROFESSOR SPIKE GILDEA 2015, the school disregarded all use of the data, saying they “hardly capture the range and quality of scholarly inquiry, while utter- ly ignoring the teaching, service, and civic engagement that faculty perform,” and “the Academic Analytics database frequently undercount, overcount, or otherwise mis- represent the achievements of individual scholars.” In late 2015, Spike Gildea, a linguistics professor at the UO, also questioned the accuracy of Academic Analytics data, and spent time checking the data that Academic Analytics provided compared with what individual faculty in the linguistics depart- ment had measured of themselves. “I found that the Academic Analytics data were embarrassingly wrong, so I did an analysis of the kinds of errors I saw, em- phasizing that these didn’t look like random errors that happened to hit a lot of linguists, but rather they potentially revealed system- atic problems in their data-collection algo- rithms,” he says. “This was especially important because, at that time, ours was the only department that had tested the data, and the results were so bad that somebody else needed to do some due diligence,” Gildea adds. “If the numbers for other departments were any- where near as bad as they were for our de- partment, then we were paying good money for bad information — which might mislead us into making bad decisions.” the provost and academic affairs office ulti- mately decided to conduct another study to check the accuracy of Academic Analytics’ data. The experiment, Shelton says, looked at individual curriculum vitae versus the Aca- demic Analytic data for 30 randomly cho- sen faculty members. “What we found is that Academic Ana- lytics data is very accurate — it’s always accurate. If there are small errors, they fix them right away,” Shelton says. “The ques- tion becomes: How close does that repre- sent the faculty in that field represent them- selves? And as we move further from the sciences, that mismatch grows and the data doesn’t reflect so well what the faculty is doing.” Shelton says the UO does not use the data to make hard and fast decisions — rather, the overall expected productivity of individual departments is always consid- ered, and the data is only looked at in con- junction with traditional faculty measure- ments and reviews — a standard practice among universities. The AAUP included in its statement that universities and faculty members should “exercise extreme caution in deciding whether to subscribe to external sources of data like Academic Analytics and should al- ways refrain from reliance on such data in tenure, promotion, compensation or hiring decisions.”
