Who's Training Whom?

by XCM

In the last few years of the 1990s, around the time of a series of compulsive visits to the local bookstore (see my previous article in 37:2), I happened to purchase a volume on Visual Basic 6 out of boredom.

I convinced myself that it was about time I moved away from QBasic and its visual reincarnation made a decent fit.  Or so I thought at the time.

I called one of my first tests with VB6 "Windows TD (Total Domination)."  It was a parody of the installation process of a successful operating system.

While listening to the comforting music "borrowed" from the original OS, the user would be presented with a series of improbable splash screens enunciating the exciting new features they would soon benefit from.

I remember one of these boasting something along the lines of: "With Windows TD you do not have to worry about emails any longer.  Windows will turn your PC on at night and independently interact with your recipients."

A couple of decades later, or a few weeks ago, I found myself composing a document using the editor endorsed by the company I work for.  It is a web-based editor owned by an organization which also runs a very successful search engine.

This program helpfully employs auto-completion and it appears to be using a model that learns and adapts to the user's writing style.  The more I wrote, the more the program would suggest words or ways to complete a phrase.  The more I wrote, the more the model and I agreed on what should be written next.

I opened up the corporate chat service and joked that we are all unwittingly training a machine learning model which, one day, might completely do without us fragile humans.

Regardless of opinions on when/if the singularity will ever be reached, or whether we have already surpassed that milestone, one thing holds immediately true:

By training a Machine Learning (ML) model, I am indirectly cementing my biases.  These might ultimately present themselves in any future write-ups of mine even when I would not have done so autonomously.  Additionally, depending on how the algorithm is designed, my biases might ultimately spill out and influence other writers.

This conundrum is nothing new.  There is an abundance of academic papers that highlight the reality of bias in machine learning models, even though these typically focus on bias intrinsic to the algorithm, rather than in the data.

The primary challenge is not just the fact in itself, as all humans are biased to begin with.  Neither are all the privacy implications of an ML-powered text editor, important though they are.

What I find most concerning is the unexplored and unpredictable territory of a machine-evolved human bias.

After reading my post in the chat room, a colleague of mine was quick to point out: "Have you considered that while you believe it's predicting what you are thinking, it might be telling you what it wants you to write?"

I must admit that until then I had foolishly thought I had been training the model, rather than entertaining the possibility of a premature inversion of roles.

I spent a few minutes pondering on a scenario I was not ready for.

Ultimately, denial and self-preservation kicked in.  After stretching on my chair and gazing out of the window, I comforted myself that this was clearly just the product of a momentary lapse of reason.

Exactly what I thought back in the 1990s while joking about self-completing emails.