Saw the first episode of Black Mirror, Season 7 yesterday. I feel like I need to offer a gentle reminder to anyone reading this. Black Mirror is a dystopian nightmare, it’s supposed to be a warning of what NOT to do, not an inspiring product roadmap.
Something about the deceptive patterns being delivered by the company representative with a smile, made me more queazy that I wanted. Deceptive patterns (also known as “dark patterns”) are tricks used in websites and apps that make you do things that you didn’t mean to, like buying or signing up for something. Should go without saying - spoilers below.
Welder Mike (Chris O’Dowd) and schoolteacher Amanda (Rashida Jones) have been married for three years and are trying for a baby. One day while teaching, Amanda collapses and doctors discover she has an inoperable brain tumor.
Bait-and-Switch
Rivermind Technologies offers Mike a “free” surgery to save Amanda’s life, but the essential functionality of her new synthetic brain requires a subscription. This gets more and more expensive to prevent her from involuntarily speaking advertisements. This wasn’t clear to them this was even possible at sign up.
Reminds me of when Google Photos offered storage unlimited photos for free, trained their AI models with this data then changed their mind asking users to pay a subscription.
Enshittificaiton or Planned Obsolescence
The base “Common” tier for Rivermind actively makes Amanda’s life worse (constant advertising, fatigue from updates), essentially forcing them to “upgrade” to a minimally acceptable level of functioning.
Reminds me of when Netflix standard, cost £5.99 compared to now, when £5.99 is the advert tier.
Hidden Costs
The initial agreement doesn’t clearly outline the extent to which Amanda will be controlled by the subscription tiers or the severity of the consequences (constant advertising) for not paying.
Reminds me of when low budget airlines sneak in costs during purchases flows.
Exploitation of Vulnerability
Rivermind preys on Mike’s desperation to save his dying wife, even when those services severely compromise Amanda’s autonomy, health, privacy and their financial stability.
Reminds me of when diet pills containing banned substances were sold on Instagram or when Uber was accused to increase prices when someone has low-battery.
Coercive Upselling
Once the couple is dependent on the Rivermind technology for Amanda’s basic functioning, they are pressured to upgrade to more expensive tiers (Rivermind Plus, Rivermind Lux) to alleviate the disruptive advertising and gain any semblance of a normal life.
Reminds me of when HP ink cartridges that stop working when users cancel their subscription.
Privacy Violations
Amanda’s thoughts and speech start being used for advertising without warning. She doesn’t even remember when she speaks them and ends up losing her job as she can’t advertise around kids. Complete violation of her privacy control over her own mind and speech.
Reminds me of when private mental health company Better Help shared consumers’ health data for advertising. Or when period app Flo also shared data without user consent.
Gamification of Degradation
The “DumDummies” platform, which Mike end up using to for income, uses a system of payment based on increasingly extreme acts of self-harm, gamifying his desperation under the “gig economy”
Reminds me a little of when Duolingo put free users in the lowest league unless they “level up” with a paid plan. Reminds more of Dirty Sanchez if it was behind a live stream.
Resource Hijacking
Her mind is used without her consent for resources, similar to mining cryptocurrency. She is put into sleep mode but never feels rested as they are using her brain for processing power.
Reminds me of when thousands of PostgreSQL servers were hijacked to mine crypto
Manufacturing Need
Rivermind creates a need for normal brain function without advertising, that they themselves control and charge a premium for.
Reminds of when Ring Doorbell users can only watch their camera live, without a subscription. They can’t view the recorded clips.
Hope on the horizon?
Maybe. There are upcoming laws against deceptive patterns that might help.