Race from an X-ray?

Good morning!

We got stuff to learn! Is your brain awake? Well, grab some warm-up stretches and meet me below.

Today's agenda:

• AI can predict self-reported race, but...the question it how?

• Rad-Memes

What do you see?

When I typically think about AI use cases, it is mostly involves AI doing things that humans can do, but better. Games such as chess, poker, and Go, all now have accompanying AI algorithms coded to beat any human. But at least the human can at least put up a fight. However, in this study described below, we are stepping into an area where AI is doing things humans can't...and maybe shouldn't.

The study: AI recognition of patient race in medical imaging: a modelling study

• The Findings: AI deep learning models can predict the self-reported race of a patient. What does that mean? AI will analyze an radiology image like the one below and be able to determine what someone wrote down on their intake form when asked, "Race: Please mark which best describes you."

• The Good: This is super impressive. Physicians are unable to this but AI is able to do it from a variety of image modality types. This isn't a better chess-playing-robot. This is a Collatz Conjecture solving-robot. I have to give props where props are due.

• The Puzzling: This is self-reported data. That means the data is pretty messy (I would know, I'm a data-guy). Yet, the model is still able to be successful. How is it accomplishing this feat?

• The Bad: Bias and discrimination exist in healthcare so it's no surprise (at least to me) it also exists in the models that we build on top healthcare data. What is concerning is that if we cannot determine how the model is predicting race, we cannot control for it. In other words, this racial bias could exist in other AI models and we're not aware of it.

This study has already been cited over 40 times and I feel as though that number is going to continue to grow until we figure out how this algorithm was able to do this. For now, we will need to keep this in the back of our minds when we are deploying AI solutions.

Now some Rad Memes for making it this far!

That’s all for today!

Thanks for reading and I'll see you next week for more With and Without AI.

-Mitch

If you liked this piece, please subscribe below: