Online retailers aren't the only ones digitally spying on you. Brick-and-mortar stores are, too

The holiday shopping season is in full swing and there’s lots of talk about how online shoppers are being tracked. If that creeps you out, you might be tempted to hit the mall instead. But artificial intelligence is tracking you there, too.

It's hard to get stores to talk about this, so KUOW met up with Robert Frederick, an industry insider, in a mall just east of Seattle.

He develops tracking hardware and AI software for retail stores. Now he runs a tech company in Seattle called Sirqul which makes high-tech devices that track you in stores.

He paraphrases his clients' motivation this way: "I want to get money out of your wallet. I want to get you to charge whatever you can afford when you're going into my store."

He may sound like a supervillain in saying that, but he's just voicing the deepest desire of every major retail store.

Frederick's philosophy is that brick-and-mortar stores must learn from online stores in order to survive. Online retailers gather tons of information about you, including your search history or what you've hovered your mouse over but didn't buy. When you're in a physical store, you also leave a trail that can be tracked.

Hear the whole Booming episode on spying stores:

From the moment you walk into a store, a wide variety of sensors begin tracking you in different ways. Some are invisible, hidden behind the ceiling tiles. Some hang below, in clusters, like mushrooms growing out of the ceiling.

Together, they’re compiling a picture of who you are, where you're going, what you're doing, what you're changing your mind about, and ultimately, what you buy. And they're feeding it into an AI system that helps store owners determine which of their strategies are working.

For instance, let's say you're looking for button down shirts and stop at a clearance rack. It's got some button down shirts on it so you start looking at those, and then next to it, there are some slightly discounted shirts. Perhaps you pick up one at each price point because you think, "Well, I got a good deal on a clearance item, so maybe I'll pay a little bit more for the second shirt."

Next to the shirts, perhaps, is a table with a really expensive sweater on it that would match well with them. Then behind that, there are jackets, and these lure you toward the most expensive items: suits.

"There's a traffic pattern that you're trying to get: Clearance, sales, regular, most expensive," Frederick said.

You may think security cameras are for catching thieves, and that's a big part of it. But they're also for capturing your likeness as a paying customer. They want to recognize your face and your walking patterns.

"So if we walk past 50 cameras in the store, it's going to be stitching together all of the references of you, and it's going to basically say, 'This person — wearing these clothes with this facial identity — walked along this path, and these are the places that they stopped.'"

But cameras are just one piece of the puzzle. There are many other sensors that AI can combine to understand who you are, including your smartphone — the biggest source of information for retailers, especially if you use one's app.

Those apps can get you perks including discounts and loyalty rewards. But many of their privacy policies also opt you into in-store surveillance. For example, the Macy's app allows the company to collect your face, your ethnicity, your location, and you opt in when you agree to the privacy policy.

Even if you don't use those apps, companies like Frederick’s capture cellphone leakage and other kinds of signals that come off you.

Smartphones put out something called a unique device identifier. Recently, Apple made this harder to track, but it's still there. And our phones are always searching for WiFi, which a store's network can see. And if you connect to a retailer's WiFi network, some of your digital activity while on it could become visible to the business.

The Bluetooth on your phone, as it attempts to pair with other devices, is another point of leakage. But Bluetooth signals don't go as far, so you have to put the sensors in more places to pick that up.

To a human being, all these channels of information from different sensors are just noise. But AI is the brain that puts together the puzzle pieces to form the whole picture of who you are.

While Frederick is all in on surveillance technology, he's uncomfortable with cameras, and draws a hard line at facial recognition, especially when that biometric data is held for longer than two weeks or shared with police departments.

"If you are someone who looks like me, you might actually be misidentified and then have a pretty bad experience when you walk into a mall," said Frederick, who is Black. "I know it sounds ironic because I have sensors that we manufacture and do stuff with. But there's a line, and you don't cross the line."

Still, he says, many retailers do cross that line. He said there are some malls he will not set foot in, for that reason. He refused to say which ones.

So, is all this surveillance legal? We put this question to Ryan Calo, a law professor at the University of Washington and the founder of the school's tech policy lab.

It's gray, according to Calo: The laws that the Federal Trade Commission has to work with at the federal level are squishy; they don't define what's illegal very clearly. That's why many states, including Washington, have stepped up and written their own rules.

"The Washington consumer is relatively well protected," Calo said. "But not as well protected as consumers in California, Massachusetts, or Illinois," which has a law banning the collection of biometric data, such as your face or the way you walk.

Calo said his main concern is how these practices can become unfair to consumers.

"What I get really worried about is the advantage-taking — the fact that consumers may not appreciate how prices are creeping up," he said. "They're being offset, apparently, by sales, but those sales are not landing evenly across the population."

In other words, you can get left out of the best sales or discounts if you don't or can't raise the value of your profile high enough for a company to pay attention to you. If you're not a big spender, you could be left out — a practice that could widen wealth gaps in the U. S., according to Calo.

"Systemically speaking, when companies study consumers for their own benefit, consumers can get harmed by having fewer opportunities and paying more than they need to," he said.

So what can you do if online shopping isn't your thing but you don't want to be tracked in brick-and-mortar stores? You can shop secondhand, or go to local artisan markets and craft fairs. They're everywhere in the Seattle area this time of year.

To hear more about what to know before you shop, listen to Booming on the KUOW app or wherever you get your podcasts.

Special thanks to Joseph Turow, author of “The Aisles Have Eyes,” who provided valuable background information for this story.