Street roots., September 01, 2017, Page 9, Image 9

R n n U R a v i AVA/
Page 10
The
weapons <
x
of
destructionz
Street Roots • September 1-7, 2017
v
*<* y«X>-
w
i w
«»<»<>.«» &■£$$$
WEAPOHS
F
PONS O
OF/
<^MAT» OESTROCTIO«^
XOO ó.t ikíx >HSr<ítSÍ<
• ■■♦««« f»»»XÍ**í ««««»tv
CAIRI O’HEÌl
p « **» y»»z r;w?s
»«<
èksé SB è io
Weapons of Math
Destruction: How
Big Data
Increases
Inequality and
Threatens
Democracy, by
Cathy O’Neil
BY MEGAN WILDHOOD
S
/ -/.M +
I
M o s t of
u s w a n t a fa ire r an d
m o re e q u ita b le society; th is is likely w hy
e n titie s fro m F a c eb o o k to th e g o v e rn m e n t
to e m p lo y ers have c o m e to rely so heavily
o n th e s e a lg o rith m s. O ’N eil e n c o u ra g e s u s
to im ag in e th a t th e 1950s-era b a n k e r w as
decid in g w ho to give a loan to b a se d on h is
employer has started charging higher health
insurance premiums for not using a Fitbit or
the number of cops in your neighborhood is
personal knowledge of the applicant’s
church attendance, the company they kept
and, likely, their race, instead of their
assets, spending habits and income. We all
C O N T R IB U T IN G C O L U M N IS T
' f y o u ’ve b e e n d e n ie d jo b a fte r job, even
so m e of th o s e entry-level, low-wage
p o sitio n s, C ath y O ’N eil can explain. If
th e r a te s y o u ’re finding fo r a u to in su ra n c e
a re sky high, C ath y O ’N eil know s why. If
y o u r am azing te a c h e r w as ju s t fired o r y o u r
grow ing a t a ra te d isp ro p o rtio n a te to actual
c rim e, allow C ath y O ’N eil to in tro d u c e you
to th e o p aq u e, self-reinforcing w orld of
w e ap o n s of m a th d e s tru c tio n (W M D s). If
y o u ’r e c o n fu sed a b o u t w hy p e o p le on th e
o th e r sid e of th e political d e b a te from you
th in k v e ry stro n g ly th e w ay th e y do, t h e r e ’s
a t le a s t o n e c o n c re te re a s o n an d C athy
O ’N eil know s it. If you th in k u sin g d a ta
an aly se s and m a th e m a tic s rem o v e s p e sk y
h u m a n b iases, C ath y O ’N eil’s c o n v ersatio n al
“W eapons of M a th D e s tru c tio n ” will
d isa b u se you of th a t notion.
O ’N eil h o ld s a d o c to ra te in m a th and,
a fte r a s tin t te a c h in g a t B a rn a rd C ollege in
N ew York City a n d w orking as a d a ta an aly st
for a h ed g e fund, s h e ’s a w ak en ed from w h a t
sh e h e rs e lf ad m its w as a tech n o -u to p ian
slum ber. S he h o p e s h e r aw akening is early
en o u g h to sto p W M Ds, w hich a re already
targ e tin g u s and com pletely codifying
injustice.
W hat sh e m e a n s by th e te r m “w eap o n s of
m a th d e s tru c tio n ” in h e r e p o n y m o u s book
is th e a lg o rith m s and m ac h in e -in te rp re te d
d a ta se ts th a t a re trolling and analyzing
e v ery a sp e c t of o u r lives, from how w e m ake
o u r w ay th ro u g h cities and cyber-highw ays,
to o u r sle e p p a tte rn s an d c o n su m e r
b eh av io rs. W hen th e y c a n ’t accu rately
m e a su re so m eth in g , o ften b e c a u se it’s
e ith e r illegal - su c h as ra c e - o r b e ca u se
h u m an b ein g s a re finicky an d th e r e will
alw ays be individuals w ho confound any
so-called n o rm s, th e y ’re p ro g ra m m e d to u se
proxies. T h u s, w e a re m e a su re d and
m y sterio u sly sc o red b a sed on o u r F acebook
frie n d s’ activities, th e Z IP code w e live in
and th e ro u te s and tim e s w e ch o o se to
com m ute.
w a n t an eq u al s h o t a t th e b asic n e c e ssitie s
o f m o d e rn life, su c h a s jo b s, in su ra n c e an d
an e d u catio n . B ut, as “W eapons o f M a th
D e s tru c tio n ” in fu riatin g ly d etails, all w e ’ve
d o n e is tra n s la te p e rs o n a l b ia se s an d blind
sp o ts in to a lg o rith m s th a t d e te rm in e m o re
a n d m o re o f o u r lives.
N o t all d a ta o r d a ta sy ste m s a re bad;
O ’N eil d o e s d isc rim in a te b e tw e e n d a ta
analysis an d w eap o n ized d a ta se ts. B u t
w h e n w e tr e a t th e re s u lts of a c o m p u ter,
w hich d o e s n o t have th e cap acity e ith e r to
b e m o ral o r tell u s w hy it is so rtin g p eo p le
an d in fo rm atio n th e w ay it is, like an
u n q u e stio n a b le god, w e p u n ish th e p o o r an d
in su la te th e w ealthy.
S o m e tim es th e p ro b lem is th a t th e
a lg o rith m u se d to give an e x c e lle n t te a c h e r
a low ratin g , fo r ex am p le, d o e sn ’t g e t th e
feed b ack th a t th is te a c h e r w e n t o n to w in
T e a c h e r of th e Y ear aw ard a t th e ir n e x t
p o sitio n . So its in sc ru ta b le an d clearly
e rro n e o u s m e th o d of sc o rin g te a c h e rs
re m a in s unfixed.
Of c o u rse tw eaking m o d els can b e ju s t as
dam aging. C rim e-tracking sy ste m s w ith
p red ictiv e e le m e n ts, for ex am p le, can c re a te
th e s e self-fulfilling feed b ack lo o p s th a t
c re a te th e v e ry rea lity th e m o d els u se to
validate th e ir own d ata. A nd it’s n o t as
stra ig h tfo rw a rd as ju s t b an n in g o r disabling
th e m . W M D s a re o ften in place b e c a u se
th e y m ake th e c o m p an ies th a t u se th e m a
lo t of m oney, o r b e c a u se th e y g e t d e sire d
re su lts. F o r in sta n c e , political cam p aig n s
em ploy W M D s to s o r t p e o p le in to g ro u p s
b a sed on how likely th e y a re to v o te and, if
so, for w hich c an d id ate. T h e y can c re a te
en tirely d ifferen t ad s to show to th e v ario u s
se g m e n ts of p e o p le th e ir a lg o rith m s
p ro d u ced .
B eyond p a ssin g o u t th is b o o k to th e
e m p lo y ers w ho m ay have re je c te d you b a sed
on a lack of keyw ords in y o u r re s u m e o r th e
re s u lts o f a p e rso n a lity te s t (th e legality of
which is questionable but the use of which
is wide and widening), I’m not left with a lot
of options. O’Neil proposes some abstract
so lu tio n s in th e fo rm of p latitu d e -esq u e
v alu e s ta te m e n ts - “w e sh o u ld re in th e s e
W M D s in ,” “w e m u s t
m a s te r th e m so th e y d o n ’t
m a s te r u s ” - b u t sh e saves
th e m all fo r th e
co nclusion. A nd m o st of
th e m a re lacking actual,
c o n c re te s te p s th a t
citizen s, esp ecially th o s e
w ho a re n o t d a ta sc ie n tists
can ta k e to sto p th e
dam ag e sh e so e lo q u e n tly
e la b o ra te s c h a p te r by
m ad d e n in g c h ap ter.
O n o n e h an d , it’s
validating fo r th e jo b
s e e k e r w h o ’s b e e n
se a rc h in g fo r m o n th s, o r
th e y oung d riv e r b ein g
When we treat the results of
a computer? w hich dees not
h a w the capacity either to
be m oral or te ll ns why it is
sorting people and
Inform ation the way It Is,
lik e an nngnestlonable god,
we punish the poor and
Insulate the wealthy.
g o u g ed o n th e ir c a r
in su ra n c e , esp ecially if
th e y ’ve b e e n to ld so m e v e rsio n o f “ju s t k e e p
lo o k in g ” o r “you ju s t have to find a b e tte r
ra te . O n th e o th e r h a n d , I ’m still w aitin g
fo r th e b o o k th a t can d e s c rib e so lu tio n s fo r
re g u la r p e o p le , a s co n cretely , th o ro u g h ly
and specifically a s it can d e s c rib e th e
p ro b lem s.
T h e w ay to d isa rm a W M D , C a th y O ’N eil
su g g e sts, is to u s e th e c o lle c te d d a ta to
“ta r g e t” individuals fo r s e rv ic e s th e y n e e d ,
r a th e r th a n to ex clu d e o r g au g e th e m . I ’m
u n n e rv e d th a t sh e a p p a re n tly le ft it u p to
m e, a m a th e m a tic a lly c h a lle n g e d a n d m ildly
te c h n o p h o b ic re a d e r, to fig u re o u t how.
Reprinted from Street Roots’ sister paper,
Real Change News, Seattle, Wash.