Fordham Law Review Fordham Law Review
Volume 88 Issue 1 Article 4
2019
Regulating Habit-Forming Technology Regulating Habit-Forming Technology
Kyle Langvardt
University of Detroit Mercy School of Law
Follow this and additional works at: https://ir.lawnet.fordham.edu/6r
Part of the Commercial Law Commons, Constitutional Law Commons, Consumer Protection Law
Commons, and the First Amendment Commons
Recommended Citation Recommended Citation
Kyle Langvardt,
Regulating Habit-Forming Technology
, 88 Fordham L. Rev. 129 (2019).
Available at: https://ir.lawnet.fordham.edu/6r/vol88/iss1/4
This Article is brought to you for free and open access by FLASH: The Fordham Law Archive of Scholarship and
History. It has been accepted for inclusion in Fordham Law Review by an authorized editor of FLASH: The Fordham
Law Archive of Scholarship and History. For more information, please contact tmelnick@law.fordham.edu.
129
REGULATING HABIT-FORMING TECHNOLOGY
Kyle Langvardt*
Tech developers, like slot machine designers, strive to maximize the user’s
“time on device.” They do so by designing habit-forming products—
products that draw consciously on the same behavioral design strategies that
the casino industry pioneered. The predictable result is that most tech users
spend more time on device than they would like, about five hours of phone
time a day, while a substantial minority develop life-changing behavioral
problems similar to problem gambling.
Other countries have begun to regulate habit-forming tech, and American
jurisdictions may soon follow suit. Several state legislatures today are
considering bills to regulate “loot boxes,” a highly addictive slot-machine-
like mechanic that is common in online video games. The Federal Trade
Commission has also announced an investigation into the practice. As public
concern mounts, it is surprisingly easy to envision consumer regulation
extending beyond video games to other types of apps. Just as tobacco
regulations might prohibit brightly colored packaging and fruity flavors, a
social media regulation might limit the use of red notification badges or
“streaks” that reward users for daily use.
It is unclear how much of this regulation could survive First Amendment
scrutiny; software, unlike other consumer products, is widely understood as
a form of protected “expression.” But it is also unclear whether well-drawn
laws to combat compulsive technology use would seriously threaten First
Amendment values. At a very low cost to the expressive interests of tech
companies, these laws may well enhance the quality and efficacy of online
speech by mitigating distraction and promoting deliberation.
I
NTRODUCTION .................................................................................. 130
I.
THE RISE OF HABIT-FORMING DESIGN ......................................... 134
A. Why Drive Compulsive Use? ............................................ 134
1. Time on Device: The Advertising Model ................. 135
* Associate Professor, University of Detroit Mercy School of Law. The author thanks Khaled
Beydoun, Erin Carroll, Julie Cohen, Andrew Moore, G. Michael Parsons, Alan Rozenshtein,
and the participants at the Georgetown Technology Law and Policy Colloquium and the
Loyola University Chicago School of Law Constitutional Law Colloquium for their valuable
comments and insights at various stages of the process. The author thanks Samantha Buck
and Alex Choi for their valuable research assistance.
130 FORDHAM LAW REVIEW [Vol. 88
2. The Microtransactions Model .................................... 137
B. An Overview of Habit-Forming Design ............................ 141
C. What’s the Harm? ............................................................ 146
1. Extreme Cases of Problem Use .................................. 146
2. Social Norms and Lifestyle ........................................ 147
3. The Public Sphere ...................................................... 148
II.
PROSPECTS FOR REGULATION ..................................................... 152
A. Strategies for Regulation .................................................. 154
1. Labeling Requirements .............................................. 154
2. Responsible Use Devices ........................................... 154
3. Bans on Dangerous Features ...................................... 156
4. Counter-Addictive Design ......................................... 159
B. Tools for Regulation ......................................................... 160
1. Existing Tools for Regulation .................................... 160
a. Gambling Law ...................................................... 160
b. Consumer Protection ........................................... 164
c. Common Law Torts .............................................. 166
2. New Tools for Regulation .......................................... 168
a. Piecemeal Legislation .......................................... 169
b. A General Mandate .............................................. 169
c. Dry-Up-the-Market Approaches .......................... 170
III.
FIRST AMENDMENT CHALLENGES ............................................. 171
A. Coverage ........................................................................... 172
1. The Holistic Approach ............................................... 173
a. Games .................................................................. 174
b. Social Apps and Platforms ................................... 175
c. Computer Code .................................................... 177
2. The Particular Approach ............................................ 178
a. Addictive Design as Conventional
Message-Bearing Speech ................................... 178
b. Addictive Design as Abstract Expression ............ 179
c. Addictive Design as an Auxiliary to Speech ......... 181
B. Degrees of Scrutiny .......................................................... 182
C
ONCLUSION ..................................................................................... 184
I
NTRODUCTION
Tech products from Facebook to Candy Crush apply well-tested
behavioral techniques to make their products as habit-forming as possible.
Industry gurus, until recently, bragged openly about these practices in
industry bestsellers with titles like Hooked: How to Build Habit-Forming
2019] REGULATING HABIT-FORMING TECHNOLOGY 131
Products
1
and Evil by Design.
2
A tech consultancy called Dopamine Labs
went so far as to advertise that: “Dopamine makes your app addictive. Lift
your engagement and revenue up to 167% by giving your users our perfect
[hits] of dopamine. . . . High ROI. PhD tested, CEO approved.”
3
These efforts appear to have paid off. One recent study discovered that
average Americans use their smartphones 76 times a day and touch their
phones 2617 times per day.
4
Among young adults, five hours of phone use
a day is typical.
5
And in 2018, the World Health Organization (WHO) took
an unprecedented step by including “gaming disorder,” a behavioral
addiction, among its International Classification of Diseases.
6
Regulators have begun to pay attention. Washington State’s gambling
laws have been interpreted to cover certain casino-themed mobile games.
7
Congress and several state legislatures are exploring legislation to restrict the
use of “loot boxes,”
8
a de facto slot machine that drives revenue in video
games,
9
and the Federal Trade Commission (FTC) has promised to
1. NIR EYAL WITH RYAN HOOVER, HOOKED: HOW TO BUILD HABIT-FORMING PRODUCTS
(2014).
2. C
HRIS NODDER, EVIL BY DESIGN: INTERACTION DESIGN TO LEAD US INTO TEMPTATION
(2013).
3. Jonathan Shieber, Meet the Tech Company That Wants to Make You Even More
Addicted to Your Phone, T
ECHCRUNCH (Sept. 8, 2017), https://techcrunch.com/2017/
09/08/meet-the-tech-company-that-wants-to-make-you-even-more-addicted-to-your-phone/
[https://perma.cc/LS2A-NGZJ].
4. Michael Winnick, Putting a Finger on Our Phone Obsession,
DSCOUT (June 16, 2016),
https://blog.dscout.com/mobile-touches [https://perma.cc/6GKR-ZFPU].
5. Carolyn Gregoire, You Probably Use Your Smartphone Way More Than You Think,
H
UFFINGTON POST (Nov. 2, 2015, 4:13 PM), https://www.huffingtonpost.com/entry/
smartphone-usage-estimates_us_5637687de4b063179912dc96 [https://perma.cc/L5HA-
SWRQ].
6. The WHO defines “gaming disorder” as a “pattern of gaming behavior (‘digital-
gaming’ or ‘video-gaming’) characterized by impaired control over gaming, increasing
priority given to gaming over other activities to the extent that gaming takes precedence over
other interests and daily activities, and continuation or escalation of gaming despite the
occurrence of negative consequences.” Gaming Disorder, W
ORLD HEALTH ORG. (Sept. 2018),
https://www.who.int/features/qa/gaming-disorder/en/ [https://perma.cc/4P6D-UPQZ].
7. Kater v. Churchill Downs Inc., 886 F.3d 784, 785 (9th Cir. 2018). Other states’
gambling laws, however, have been interpreted more narrowly. See generally Mason v. Mach.
Zone, Inc., 851 F.3d 315, 319–20 (4th Cir. 2017) (applying Maryland law); Phillips v. Double
Down Interactive LLC, 173 F. Supp. 3d 731, 737–39 (N.D. Ill. 2016) (applying Illinois law);
Soto v. Sky Union, LLC, 159 F. Supp. 3d 871, 877–81 (N.D. Ill. 2016) (applying California
law).
8. See, e.g., Press Release, Josh Hawley, U.S. Senator, Senator Hawley to Introduce
Legislation Banning Manipulative Video Game Features Aimed at Children (May 8, 2019),
https://www.hawley.senate.gov/sites/default/files/2019-05/2019-05-08_Video-Game-
Bill_One-Pager_0.pdf [https://perma.cc/RVX4-PXTD]; see also Assemb. 2194, 2017–2018
Leg., Reg. Sess. (Cal. 2018); S. 3024, 29th Leg., Reg. Sess. (Haw. 2018); H.R. 2727, 29th
Leg., Reg. Sess. (Haw. 2018); H.R. 2686, 29th Leg., Reg. Sess. (Haw. 2018); H.R. 4460, 90th
Leg., Reg. Sess. (Minn. 2018). Other bills have called for investigations into the practice by
gambling and law enforcement authorities. See, e.g., S. 333, 120th Gen. Assemb., 2d Reg.
Sess. (Ind. 2018); S. 6266, 65th Leg., Reg. Sess. (Wash. 2018).
9. See infra Part II.A.3.
132 FORDHAM LAW REVIEW [Vol. 88
investigate that same practice.
10
International regulators have gone further,
with South Korea requiring online gaming platforms to deter extended play
and to kick minors off after midnight.
11
In principle, these efforts might someday extend beyond gaming to a much
wider world of habit-forming technology. A growing number of tech insiders
have expressed strong reservations about social media’s addictive potential,
with some calling for the industry to be regulated “like tobacco or alcohol.”
12
It is easy to imagine a legislative push to protect minors, at least, from tech
products that are understood to encourage compulsive use. And as tech
products become more sophisticated and intrusive, we may eventually see
calls to protect adults from certain aggressive forms of behavioral
manipulation.
This Article outlines the various forms that these regulatory interventions
might take. These include light-touch, limited-range policies that are already
under official consideration in American jurisdictions.
13
They also include
aggressive policies that seem unthinkable today, but that over the long-term
may appear proportionate as tech companies become more sophisticated at
manipulating user behavior.
14
Even the more modest policies will raise quick constitutional challenges if
enacted. Social media,
15
video games,
16
and even computer code itself
17
have received First Amendment protection in past case law. One might
10. Brian Crecente, Federal Trade Commission Chairman Pledges to Investigate Video
Game Loot Boxes, V
ARIETY (Nov. 27, 2018, 3:01 PM), https://variety.com/2018/gaming/
news/ftc-loot-box-investigation-1203038178/ [https://perma.cc/23C3-YBQ3].
11. See Sohn Ji-Young, WHO’s Labeling of Gaming Addiction as a Disorder Sparks
Concern in Korea, J
AKARTA POST (Jan. 4, 2018, 2:01 PM), https://
www.thejakartapost.com/life/2018/01/04/whos-labeling-of-gaming-addiction-as-a-disorder-
sparks-concern-in-korea.html [https://perma.cc/ZZP8-LGG8]; Carolyn Sun, South Korea’s
Video Game Addiction, N
EWSWEEK (Oct. 17, 2011, 1:00 AM), https:// www.newsweek.com/
south-koreas-video-game-addiction-68309 [https://perma.cc/Y2CR-5R8X]. South Korea’s
online gaming platforms must screen users by requiring them to submit a national ID number.
Huu Kim Le, Internet Gaming Shutdown Law aka “Cinderella Curfew,” CGI
CLINIC (July 18,
2015), https://cgiclinic.com/south-korea-special-shutdown-law-aka-cinderella-curfew/
[https://perma.cc/3HHF-Z5ET].
12. Alex Hern, Facebook Should Be “Regulated Like Cigarette Industry,” Says Tech
CEO, G
UARDIAN (Jan. 24, 2018, 7:32 AM), https://www.theguardian.com/technology/2018/
jan/24/facebook-regulated-cigarette-industry-salesforce-marc-benioff-social-media
[https://perma.cc/RQ34-GLGN] (“[T]echnology has addictive qualities that we have to
address, and that product designers are working to make those products more addictive, and
we need to rein that back as much as possible.” (quoting Marc Benioff, chief executive officer
of Salesforce)); Roger McNamee, Why Not Regulate Social Media Like Tobacco or Alcohol?,
G
UARDIAN (Jan. 29, 2018, 4:32 AM), https://www.theguardian.com/media/2018/jan/
29/social-media-tobacco-facebook-google [https://perma.cc/P6BG-Q9CN].
13. See infra Parts II.A.1, II.A.3 (proposing labeling strategies and bans on video game
“loot boxes”).
14. See infra Part II.A.4 (proposing mandatory design features to slow down and de-
intensify the user experience).
15. See, e.g., Packingham v. North Carolina, 137 S. Ct. 1730, 1735 (2017).
16. See, e.g., Brown v. Entm’t Merchs. Ass’n, 564 U.S. 786, 790 (2011).
17. See, e.g., Universal City Studios, Inc. v. Corley, 273 F.3d 429, 446–52 (2d Cir. 2001).
2019] REGULATING HABIT-FORMING TECHNOLOGY 133
assume from this record that even the most modest attempts to regulate habit-
forming design should fail First Amendment scrutiny.
Instead, the question is wide open—courts have hardly begun to address
the First Amendment status of software’s technical and nonexpressive
components. Is the choice to use an “infinite scroll” in the Facebook News
Feed a form of “speech”? What about Twitter’s strategic deployment of a
“loading” wheel to build anticipation after a Twitter user “pulls to refresh”?
And if they are speech, then how should courts balance the equities? It is
forbidden to burden speech on the basis that the speech is too persuasive.
18
Is it any better to burden habit-forming design—sometimes called
“persuasive technology”
19
—because it succeeds in building user habits?
Courts’ answers to these questions will carry heavy consequences for
media policy at large. On the one hand, most free content online is monetized
by a huge behavioral advertising ecosystem that includes Google, Facebook,
and scattered smaller advertising networks.
20
The business model requires
enormous amounts of data, which in turn requires users to spend enormous
amounts of time on their devices. Policies that cut into that time may improve
users’ health while impairing their access to information.
On the other hand, the addiction-driven nature of social media probably
harms the quality of public discourse and deliberation. First, developers’
efforts to drive constant user engagement encourage an emotional, hair-
trigger style of public expression.
21
Second, by training users to respond to
stimuli out of habit rather than by considered choice, social media engineers
prime the public discussion for manipulation by “trolls” and saboteurs.
22
A
18. See Sorrell v. IMS Health Inc., 564 U.S. 552, 578 (2011) (“That the State finds
expression too persuasive does not permit it to quiet the speech or to burden its messengers.”).
19. See, e.g., B.J.
FOGG, PERSUASIVE TECHNOLOGY: USING COMPUTERS TO CHANGE WHAT
WE THINK AND DO 1 (2003) (defining “persuasive technology” as “any interactive computing
system designed to change people’s attitudes or behaviors”).
20. See infra Part I.B.
21. See M. J. Crockett, Moral Outrage in the Digital Age, 1 N
ATURE HUM. BEHAV. 769,
769 (2017); Maksym Gabielkov et al., Social Clicks: What and Who Gets Read on Twitter?,
P
ERFORMANCE EVALUATION REV., June 2016, at 179, 182 (reporting that “a majority (59%) of
the URLs mentioned on Twitter are not clicked at all”); Michela Del Vicario et al., Echo
Chambers: Emotional Contagion and Group Polarization on Facebook, S
CI. REP. 8–9 (Dec.
1, 2016), https://www.nature.com/articles/srep37825.pdf [https://perma.cc/BM9U-DRHY]
(demonstrating that users on Facebook tend to organize into echo chambers where higher
involvement produces more negative emotion); Rui Fan et al., Anger Is More Influential Than
Joy: Sentiment Correlation in Weibo, PLOS
ONE (Oct. 15, 2014), http://journals.plos.org/
plosone/article?id=10.1371/journal.pone.0110184 [https://perma.cc/RN8E-FHP6] (finding
that connected users of Weibo, a Twitter-like platform in China, share significantly more
sentiment correlation along lines of anger than of joy or sadness).
22. See D
IPAYAN GHOSH & BEN SCOTT, NEW AM., #DIGITALDECEIT: THE TECHNOLOGIES
BEHIND PRECISION PROPAGANDA ON THE INTERNET 17 (2018); Craig Timberg & Elizabeth
Dwoskin, Russians Got Tens of Thousands of Americans to RSVP for Their Phony Political
Events on Facebook, W
ASH. POST (Jan. 25, 2018), https://www.washingtonpost.com/
news/the-switch/wp/2018/01/25/russians-got-tens-of-thousands-of-americans-to-rsvp-for-
their-phony-political-events-on-facebook/ [https://perma.cc/392P-9Q3G].
134 FORDHAM LAW REVIEW [Vol. 88
population better able to resist its news and social media interfaces may well
produce a political culture that is hardier and more stable.
23
Part I discusses the economic incentives that have made addictive design
such a pervasive phenomenon in the tech world. Then, it briefly discusses
some techniques that tech designers use to encourage habit formation, along
with the harms, both mundane and profound, that these habits cause. Part II
lays out the prospects for regulation: what the range of possible strategies
would be and which institutions may be equipped to implement some of
them. Part III turns to a broad discussion of the First Amendment
considerations.
I.
THE RISE OF HABIT-FORMING DESIGN
A. Why Drive Compulsive Use?
Over the past twenty years or so, consumers have come to expect low
prices—and more often no price at all—for most of the software and media
they consume online. This is basic economics: the internet has reduced the
marginal cost of distributing (and copying) content to practically zero, and
prices have followed close behind.
Zero-price markets force content producers and distributors to extract
value from users in ways that do not offend the zero-price expectation up
front.
24
Most firms have two options: micropayments and advertising.
These techniques make up for a lack of up-front payment by collecting value
over a long and limitless series of consumer interactions. Hence the
obsession at companies like Facebook and Twitter with “driving
engagement”—more interactions lead to more user “engagement,” which
leads to more value the firm stands to capture.
25
23. Hunt Allcott et al., The Welfare Effects of Social Media 20–22 (Stanford Inst. for Econ.
Policy Research, Working Paper No. 19-002, 2019), https://siepr.stanford.edu/sites/default/
files/publications/19-002.pdf [https://perma.cc/7RVR-LMX7] (finding that study subjects
who quit Facebook for one month indicated less attention and knowledge of news but also
demonstrated less political polarization).
24. Rafet Sifa et al., Profiling in Games: Understanding Behavior from Telemetry, in
S
OCIAL INTERACTION IN VIRTUAL WORLDS 337, 345 (Kiran Lakkarahu et al. eds., 2018) (“The
rise of F2P games has notably added to the industry’s focus on behavior analysis. These are
games with no up-front cost to the customer, with revenue depending on In-App Purchases
(IAPs) (and sometimes associated branded products). Revenue is thus dependent on the ability
of the developer to convince some portion of the customer base to purchase virtual items for
real money. In order to be successful as a business model, these games require continued
analysis of player behavior . . . .).
25. See The State of Intellectual Freedom in America: Hearing Before the Subcomm. on
the Constitution & Civil Justice of the H. Comm. on the Judiciary, 115th Cong. 6 (2018)
(statement of Ari Ezra Waldman, Director, Innovation Center for Law and Technology, New
York Law School) (“The more we like, the more we comment, the more pictures we upload,
and the more we simply browse the internet, the more Facebook knows about us, . . . and the
more money it can charge those who want to target us for advertisements.”);
SIVA
VAIDHYANATHAN, ANTI-SOCIAL MEDIA: HOW FACEBOOK DISCONNECTS US AND UNDERMINES
DEMOCRACY 56–62 (2018) (describing Facebook’s data-driven advertising model).
2019] REGULATING HABIT-FORMING TECHNOLOGY 135
In the casino industry—another sector where few would pay up front for
what they get at the end of the day
26
—the imperative to drive the slot
machine gambler’s engagement for hours on end has a more straightforward
name: “time on device.”
27
1. Time on Device: The Advertising Model
Advertising is a time-worn technique for drawing in revenue without
charging a high price to the customer. The newspaper and magazine
industries collect most of their revenue from advertisers, and broadcasters
depend on them.
28
The concept today is ultimately the same as in the mid-
nineteenth century when the “penny presses” invented it: content producers
draw consumers’ attention and “sell” their ad exposure to the merchant.
29
Advertising-based business models have always valued “time on device,”
whether the device is a television or a magazine. Twentieth-century mass
media strove to keep viewers’ and listeners’ attention through cliff-hangers,
promotions, call-in contests, and the like.
30
Advertisers targeted audiences
demographically. TV networks would use soap operas to drive demand for
soap and other household products that women of the day were more likely
to purchase.
31
Rolling Stone magazine, at least in its early days, ran
advertisements aimed at young adults.
32
Today’s online advertising industry has grown much more sophisticated.
Modern ad networks target individuals based on real-time behavioral
profiling.
33
In many cases, ad networks monitor the individual’s response to
26. See Slot Machine Payback Statistics, AM. CASINO GUIDE,
https://www.americancasinoguide.com/slot-machine-payback-statistics.html
[https://perma.cc/YLT2-7YE6] (last visited Aug. 22, 2019) (showing that slot machines
around the country have 85 to 97 percent rates of return per pull).
27. Brad Plumer, Slot-Machine Science: How Casinos Get You to Spend More Money,
V
OX (Mar. 1, 2015, 6:27 AM), https://www.vox.com/2014/8/7/5976927/slot-machines-
casinos-addiction-by-design [https://perma.cc/P44F-T9ZU].
28. See generally Newspapers Fact Sheet, P
EW RES. CTR. (June 13, 2018),
https://www.journalism.org/fact-sheet/newspapers/ [https://perma.cc/PX6X-UDLB]; Matt
Stieb, One Company Is Making Money from News and It Rhymes with ‘Shmoogle,’ N.Y.
MAG.
(June 9, 2019), http://nymag.com/intelligencer/2019/06/google-posts-usd4-7-billion-profit-
in-2018-from-news-industry.html [https://perma.cc/Y7C6-N5T9].
29. See Sonja R. West, The “Press,” Then & Now, 77 O
HIO ST. L.J. 49, 93 (2016).
30. See Emily Nussbaum, Tune in Next Week: The Curious Staying Power of the
Cliffhanger, N
EW YORKER (July 22, 2012), https://www.newyorker.com/magazine/2012/07/
30/tune-in-next-week [https://perma.cc/4XE7-5AAM].
31. Soap Opera, A
D AGE (Sept. 15, 2003), https://adage.com/article/adage-
encyclopedia/soap-opera/98883/ [https://perma.cc/F9GM-BHGR].
32. Nick Ravo, Rolling Stone Turns a Prosperous 20, N.Y.
TIMES (Aug. 23, 1987),
https://www.nytimes.com/1987/08/23/business/rolling-stone-turns-a-prosperous-20.html
[https://perma.cc/76PU-UUUV].
33. See Adknowledge Rolls Out Enhanced Behavioral Targeting Model to Increase
Advertiser Conversions, B
US. WIRE (July 24, 2007), https://www.globenewswire.com/news-
release/2007/07/24/1050231/0/en/Adknowledge-Rolls-Out-Enhanced-Behavioral-Targeting-
Model-to-Increase-Advertiser-Conversions.html [https://perma.cc/KBZ5-WEZB]
(“Adknowledge, a CPC auction-style ad network, analyzes behavioral data in real time,
uncovering interest and response patterns to deliver targeted advertisements more likely to
match consumer preferences.”); Following $6m Investment, SALESmanago Brings Next Gen
136 FORDHAM LAW REVIEW [Vol. 88
an ad: Did the individual follow through by visiting the merchant’s website,
viewing the advertised items, putting the item in a “cart” or a “save for later”
list, or consummating the purchase?
34
This fine-grained control over consumer behavior vastly improves the
“yield” of an online ad placement,
35
and it has to. Many web users use ad-
blockers.
36
Users who do not use ad-blockers are not generally inclined to
click; studies have demonstrated that internet ads are among the least
effective forms of advertising.
37
And if nobody clicks, then advertisers must
strive harder to put “relevant” ads in front of consumers.
Marketing Automation to eCommerce & B2C, PR NEWSWIRE (June 6, 2016, 9:00 AM),
https://www.prnewswire.com/news-releases/following-6m-investment-salesmanago-brings-
next-gen-marketing-automation-to-ecommerce--b2c-300279516.html [https://perma.cc/
D2N6-WP5J] (“The complete individual profiles of customers with such behavioral and
transactional data are then used in real time to build automatic predictive offers that are
delivered individually in all marketing channels . . . .”); Michael Learmonth, Search-Titan
Google Makes Display Play with ROI Tool: Campaign Insights Aims to Give Click King
Bigger Share of $4.7B Business, A
D AGE (Oct. 19, 2009), https://adage.com/article/
digital/google-search-titan-makes-display-play-roi-tool/139756 [https://perma.cc/BBA4-
PY4Q] (describing Google’s “Campaign Insights” program, which relies on automated A/B
testing to determine what kinds of ad displays prompt users to search for a brand or product);
Proving There Is More to Location Targeting in Mobile, xAd Launches Proprietary
SmartFencing™, SmartLocation™ Technologies, B
US. WIRE (Nov. 8, 2012, 8:00 AM),
https://www.businesswire.com/news/home/20121108005317/en/Proving-Location-
Targeting-Mobile-xAd-Launches-Proprietary [https://perma.cc/NZB3-6L2U] (“xAd’s local
search and display ad network reaches over 85 million mobile consumers each month and is
the only network with the ability to not only ensure accurate local targeting—but enhance it
through the use of real-time and historical local search behaviors and other available data
variables.”).
34. Tim Peterson, Google Tool Can Calculate What That Mobile Ad Is Really Worth,
A
DWEEK (Mar. 26, 2013), https://www.adweek.com/digital/google-tool-can-calculate-what-
mobile-ad-really-worth-148155/ [https://perma.cc/2TEC-N29M]. This progression of steps
toward a final transaction is called a “conversion funnel.” Id. Data analytics tools help
merchants to optimize it by locating at which point in the funnel consumers are jumping out
and then fixing the “leak.” See Create and Manage Funnels, F
ACEBOOK ANALYTICS,
https://www.facebook.com/help/analytics/935921203105136 [https://perma.cc/AKU4-
9TTE] (last visited Aug. 22, 2019).
35. See H
OWARD BEALES, THE VALUE OF BEHAVIORAL TARGETING 3 (2009), https://
www.networkadvertising.org/pdfs/Beales_NAI_Study.pdf [https://perma.cc/B5AX-CRLR]
(showing behaviorally targeted advertising has over twice the clickthrough rate of “run-of-
network” advertising, and significantly greater conversions). Some critics have dismissed this
account as “investor storytime,” arguing that ad networks excel not at driving sales but at
showing ads to people who are just about to make a purchase anyway. See Ethan Zuckerman,
The Internet’s Original Sin, A
TLANTIC (Aug. 14, 2014), https://www.theatlantic.com/
technology/archive/2014/08/advertising-is-the-internets-original-sin/376041/
[https://perma.cc/NFH7-KVZV?type=image]. But note that in either case, the incentive to
drive compulsive engagement is the same.
36. Lauren Johnson, IAB Study Says 26% of Desktop Users Turn On Ad Blockers,
A
DWEEK (July 26, 2016), https://www.adweek.com/digital/iab-study-says-26-desktop-users-
turn-ad-blockers-172665/ [https://perma.cc/2MYC-8SXB].
37. Derek Thompson, A Dangerous Question: Does Internet Advertising Work At All?,
A
TLANTIC (June 13, 2014), https://www.theatlantic.com/business/archive/2014/06/a-
dangerous-question-does-internet-advertising-work-at-all/372704/ [https://perma.cc/NHF9-
NPMT].
2019] REGULATING HABIT-FORMING TECHNOLOGY 137
Today’s advertisers therefore have two reasons to drive time on device.
First, the user must be exposed to as many ads as possible. The more time
on device, the more ad exposures. The second requirement—one that
emerged in the internet era—is to target the ads as closely as possible to the
individual user. This targeting requires advertisers to collect as much data as
possible about the user—not only demographic data, but minute-by-minute
data about the user’s location, mood, and desires.
38
The more time on device,
the richer the dossier.
Behaviorally targeted advertising underwrites free online media on the
web and in apps. Facebook and Google run the largest advertising networks,
tracking user behavior and using the data to place ads on their own sites and
across the web.
39
They may also buy or otherwise obtain data from other
sources, such as brick-and-mortar customer reward programs,
40
credit
bureaus,
41
or from other app developers who collect data for the purpose of
selling it.
42
Advertisers pay Facebook, Google, and the other ad networks to place ads,
and the ad networks split this payment with the owners of the website or app
where the ad is placed. Everyone’s revenue stream—that of the content
producers, app developers, and social and search platforms that place ads—
depends on maximizing time on device.
2. The Microtransactions Model
Only a few years ago, the video game industry captured most of its revenue
at the point of sale. Gamers might purchase a game in the store on a physical
medium, or they might pay up front for a downloadable copy from an online
gaming platform, often after having the chance to play through the beginning
38. See Kurt Wagner, This Is How Facebook Uses Your Data for Ad Targeting, VOX (Apr.
11, 2018, 6:00 AM), https://www.vox.com/2018/4/11/17177842/facebook-advertising-ads-
explained-mark-zuckerberg [https://perma.cc/5L6V-32YX].
39. See Nathan Ingraham, Facebook Buys Data on Users’ Offline Habits for Better Ads,
E
NGADGET (Dec. 30, 2016), https://www.engadget.com/2016/12/30/facebook-buys-data-on-
users-offline-habits-for-better-ads/ [https://perma.cc/LG76-XUDR].
40. Brick-and-mortar retailers can use an automated process, Facebook’s Offline
Conversions API, to pair their own point-of-sale data on a person with Facebook’s
geolocational and other data on the same person. Facebook can then provide the retailer with
a “lookalike” audience of potential buyers who might be susceptible to similar marketing
strategies. See Thorin Klosowski, How Facebook Uses Your Data to Target Ads, Even Offline,
L
IFEHACKER (Apr. 11, 2013, 11:00 AM), https://lifehacker.com/5994380/how-facebook-uses-
your-data-to-target-ads-even-offline [https://perma.cc/9FQK-4AS2]; Tim Peterson, Facebook
Will Target Ads to People Based on Store Visits, Offline Purchases, Calls to Businesses,
M
ARKETING LAND (Sept. 21, 2017, 11:00 AM), https://marketingland.com/facebook-will-
target-ads-people-based-store-visits-offline-purchases-calls-businesses-224668
[https://perma.cc/4AAN-2QFX].
41. Ingraham, supra note 39 (listing Experian and TransUnion as U.S. data partners with
Facebook).
42. Julia Angwin et al., Facebook Doesn’t Tell Users Everything It Really Knows About
Them, P
ROPUBLICA (Dec. 27, 2016, 9:00 AM), https://www.propublica.org/article/facebook-
doesnt-tell-users-everything-it-really-knows-about-them [https://perma.cc/7Q66-LK2U].
138 FORDHAM LAW REVIEW [Vol. 88
part of the game—a “demo”—for free.
43
For the most part, this business
model continued the straightforward retail model for books and other
physical media that held through the twentieth century. Prices for console
cartridge video games in the 1990s ran from roughly $60 to $80 in 2019-
adjusted terms.
44
Similar products at similar price points continue to exist for traditional
gamers who are willing to pay up front for serious console or PC games—
2018’s Red Dead Redemption 2, for instance, a big-budget production that
sells for nearly $60.
45
A new category of casual mobile gamers has emerged
as well. Light “free-to-play” (F2P) games now make up most of the market.
46
Rather than charging up front, these games’ developers monetize their
products mostly through in-app purchases or advertising.
47
In principle, they
could sell in-game behavioral data to third-party advertising networks as
well, though it is unclear whether such a market has emerged.
48
There is a clear need for F2P casual games to maximize user engagement.
A banner ad in a video game will typically pull only about $2 or $3 per
thousand user impressions.
49
Making any significant amount of money from
ad impressions requires users to spend massive amounts of time on the game,
and even then, an ad-based strategy may not always be lucrative.
Most of the gaming sector has therefore moved to a hybrid monetization
model that rests at least partially on in-app “micropayments” from players—
small fees that players spend over time to gain small in-game advantages.
50
43. See generally Patrick Klepek, The Real Stories Behind E3’s Glossy Game Demos,
K
OTAKU (June 10, 2016, 3:28 PM), https://kotaku.com/the-real-stories-behind-e3-s-glossy-
game-demos-1710169104 [https://perma.cc/KFP2-34JB].
44. Kyle Orland & Jonathan Gitlin, Why Retail Console Games Have Never Been
Cheaper, Historically, A
RS TECHNICA (June 30, 2013, 4:00 PM), https://arstechnica.com/
gaming/2013/06/why-retail-console-games-have-never-been-cheaper-historically/
[https://perma.cc/ZW49-P5LK].
45. Red Dead Redemption 2—PlayStation 4, A
MAZON, https://www.amazon.com/Red-
Dead-Redemption-2-PlayStation-4/dp/B01M5DZ525?th=1 [https://perma.cc/PS27-DH95]
(last visited Aug. 22, 2019).
46. TTT, The Future of Gaming: How Micro-Transactions Are Disrupting the Industry,
M
EDIUM (Aug. 20, 2018), https://medium.com/trustnote/the-future-of-gaming-how-
microtransactions-are-disrupting-the-industry-c7f5bab1081b [https://perma.cc/WE2T-
RN6U].
47. “Reward-based monetization,” a third strategy, is arguably distinct from advertising
and in-app purchases, but the differences are irrelevant for purposes of this Article. Under
this strategy, developers dispense in-game rewards to players who complete some action that
is valuable to a paying merchant. These actions include installing a separate app, watching a
video ad, or completing a survey. See Michael Sprague, New Study: The Data Behind Reward-
Based Monetization in F2P Games, M
EDIUM (May 31, 2016), https://medium.com/
tapresearch/new-study-the-data-behind-reward-based-monetization-in-f2p-games-
66bb31d3ac7e [https://perma.cc/88GU-HGLF].
48. See generally Sifa et al., supra note 24, at 350.
49. Yaniv Nizan, 42 Ways to Monetize Your Mobile Game, G
AMEANALYTICS (Oct. 9,
2013), https://gameanalytics.com/blog/42-ways-to-monetize-your-mobile-game.html [https://
perma.cc/R6EW-FS43].
50. See Andrei Klubnikin, Microtransactions in Games: The Good, the Bad, and the
Ugly, G
AMEANALYTICS (Feb. 14, 2018), https://gameanalytics.com/blog/microtransactions-
games-good-bad-ugly.html [https://perma.cc/HFT5-Y6PS]; see also S
WRVE, MONETIZATION
2019] REGULATING HABIT-FORMING TECHNOLOGY 139
F2P games lean heavily on micropayments, and certain big-budget traditional
games do as well, typically as a means to lower the retail price.
51
In recent
years, the video game industry has captured most of its overall revenue from
these post-sale or post-download micropayments.
52
The usual approach is to allow the user to play for a few minutes or even
a few days without any kind of sales pitch.
53
Then some goal is dangled in
front of the player, such as the chance to win a new in-game item or to
upgrade the player’s character.
54
Achieving the goal usually requires the
player to cash in some significant amount of in-game currency that can be
either purchased with cash or earned through time on device.
55
The latter
option may require hours of tedious work from the player,
56
and the game is
often designed to make it impossible to get this work done in one sitting.
57
The user is only allowed to play for ten minutes or so at a time, for example,
REPORT 2016 6 (2016), https://www.swrve.com/images/uploads/whitepapers/swrve-
monetization-report-2016.pdf [https://perma.cc/W5W5-AUMR] (showing that 73.5 percent
of mobile in-app purchases range from $0 to $10).
51. See Sifa et al., supra note 24, at 345; see also Ben Lindbergh, Battlefront, Reddit: The
Video Game Pricing Wars That Might Reshape the Industry, R
INGER (Dec. 2, 2017, 8:00 AM),
https://www.theringer.com/features/2017/12/2/16725196/reddit-fighting-microtransaction-
exploitation-in-video-games [https://perma.cc/HT8J-5VRN] (“Rather than raise the sticker
price and scare away customers on the front end, publishers are, logically enough, opting to
tack on costs post-purchase, via paid downloadable content, ‘season passes’ that grant access
to all future DLC, and, more problematically, microtransactions.”).
52. See Rob Thubron, Microtransactions from Free-to-Play Games Almost Three Times
the Revenue of Paid Game Releases, T
ECHSPOT (Nov. 30, 2017, 6:15 AM), https://
www.techspot.com/news/72104-microtransactions-ftp-pc-games-generate-22-billion-
2017.html [https://perma.cc/9F7W-YYNY]; TTT, supra note 46 (showing that 82 percent of
game revenue in 2017 came from F2P games containing in-app purchases).
53. See, e.g., Damion Schubert, Respecting the Player’s Wallet,
GAMASUTRA (Apr. 29,
2013), https://www.gamasutra.com/view/news/191264/Respecting_the_players_wallet.php
[https://perma.cc/Q68N-DD97] (noting that in Star Wars: The Old Republic, “[p]layers are
rarely, if ever, prompted to spend money in the first 10 levels, and the player is likely to forget
that doing so is even possible”).
54. On June 12, 2019, the author played through the opening stages of Candy Crush Saga
and Minion Rush. In Candy Crush Saga, the player runs out of moves and is then given the
opportunity to keep playing using “gold bars” and a “Piggy Bank” purchasable for $2.99. In
Minion Rush, apparently a children’s game, the player is eventually introduced to a set of slot-
machine-like “prize pods” and character upgrades called “costumes” that can be purchased
through a set of in-game currencies. Revives and prize pods can also be earned by watching
a video advertisement.
55. See TTT, supra note 46.
56. Blizzard’s Overwatch allows users to save about an hour of completing repetitive in-
game goals per dollar paid. See Paul Tassi, The Math Behind Why Overwatch’s Loot Boxes
Are Exhausting to Unlock, F
ORBES (June 10, 2016, 11:28 AM), https://www.forbes.com/
sites/insertcoin/2016/06/10/the-math-behind-why-overwatchs-loot-boxes-are-exhausting-to-
unlock [https://perma.cc/Z3SC-54XX].
57. In the mobile game Harry Potter: Hogwarts Mystery, for example, the player
becomes trapped in a “Devil’s Snare,” which appears to slowly suffocate the player’s avatar.
At this point, the player runs out of the “energy” that allows them to complete tasks within the
game. The player may either pay for some quick energy and escape the snare or wait for hours
for the energy to recharge. Julia Alexander, Harry Potter: Hogwarts Mystery’s In-App
Purchases, Wait Times Turning Players Away, P
OLYGON (Apr. 27, 2018, 12:08 PM),
https://www.polygon.com/2018/4/27/17290168/harry-potter-hogwarts-mysterys-devils-
snare-purchases-wait-time [https://perma.cc/X6XG-3U8K].
140 FORDHAM LAW REVIEW [Vol. 88
or the user may be required to “check in” every twenty-four hours, or even
every four hours, to collect a bonus toward the goal.
58
At some point, an
invested player will be worn down to the point where it appears rational to
spend, say, $2.99 and skip ahead to the sought-after goal.
59
At this point, of
course, another in-game goal appears and the cycle begins again.
Most players never pay, and most of those who do pay very little. Instead,
most revenue from micropayments is highly concentrated among a small
group of apparent addicts who individually spend thousands of dollars on in-
app purchases. One study showed that 0.15 percent of mobile gamers
account for 50 percent of the industry’s revenue from micropayments.
60
About 1.9 percent make up 90 percent of revenue.
61
These gamers are called
“whales” within the video game industry, and they compensate for the
overwhelming majority of gamers who either never pay or who pay very
little.
62
In this respect, the video game industry’s revenue structure is coming
to resemble that of the gambling industry and the alcohol industry, where
addicted customers also account for most of the profits.
63
The term “whale,”
in fact, originated in the casino industry.
64
And as in the casino industry, the
whale-centered model can be highly profitable. The freemium
65
mobile
game Game of War, despite its poor critical reception, draws a yearly average
58. William Grosso, The Science & Craft of Designing Daily Rewards—and Why FTP
Games Need Them, G
AMASUTRA (June 13, 2016, 5:32 PM), http://www.gamasutra.com/
blogs/WilliamGrosso/20160613/274759/The_Science__Craft_of_Designing_Daily_Rewards
__and_Why_FTP_Games_Need_Them.php [https://perma.cc/593G-DQ4Y].
59. See, e.g., Keza MacDonald, Harry Potter: Hogwarts Mystery Review: A Shameless
Shake-Down, G
UARDIAN (May 4, 2018, 2:00 PM), https://www.theguardian.com/
games/2018/may/04/harry-potter-hogwarts-mystery-review [https://perma.cc/5WHU-Y4NV]
(“Hogwarts Mystery pulls the old trick of hiding the true cost of its purchases behind an in-
game ‘gem’ currency, but I worked out that you’d have to spend about £10 a day just to play
Hogwarts Mystery for 20 consecutive minutes.”).
60. Swrve Finds 0.15% of Mobile Gamers Contribute 50% of All In-Game Revenue,
S
WRVE (Feb. 25, 2014), https://www.swrve.com/company/press/swrve-finds-015-of-mobile-
gamers-contribute-50-of-all-in-game-revenue [https://perma.cc/4QBG-UU3H].
61. Klubnikin, supra note 50.
62. Paul Tassi, Why It’s Scary When 0.15% Mobile Gamers Bring In 50% of the Revenue,
F
ORBES (Mar. 1, 2014, 4:28 PM), https://www.forbes.com/sites/insertcoin/2014/03/01/why-
its-scary-when-0-15-mobile-gamers-bring-in-50-of-the-revenue/ [https://perma.cc/VK7A-
HBER].
63. See generally Tiffany Hsu, Video Game Addiction Tries to Move from Basement to
Doctor’s Office, N.Y.
TIMES (June 17, 2018), https://www.nytimes.com/2018/06/17/
business/video-game-addiction.html [https://perma.cc/6Y8Z-B4VW].
64. See Lisa Fletcher et al., Biggest Loser?: Gambler Dropped $127M in a Year, ABC
NEWS (Dec. 8, 2009), https://abcnews.go.com/GMA/gambler-dropped-127-million-vegas-
blames-casino-losses/story?id=9272730 [https://perma.cc/5FCN-8QP4].
65. “Freemium” is a portmanteau of “free” and “premium”: free for now, and you pay
later for more content. See Brian X. Chen, Playing at No Cost, Right into the Hands of Mobile
Game Makers, N.Y.
TIMES (Mar. 18, 2012), https://www.nytimes.com/2012/03/19/
technology/game-makers-give-away-freemium-products.html [https://perma.cc/7ZQX-
CQ4E].
2019] REGULATING HABIT-FORMING TECHNOLOGY 141
of $550 per player—a remarkable figure given the overwhelming majority of
players who pay very little.
66
This unbalanced revenue structure may give game developers strong
incentives to encourage addiction-driven, whale-like purchases. But gaming
companies often have a collateral interest in addicting nonpaying players as
well. First, a player who keeps playing might pay later on; a player who
walks away will not. Second, many mobile games contain a social dimension
that is enhanced by widespread participation. In Design Home, for example,
players rate and review each other’s attempts at interior design.
67
In Marvel
Strike Force, players join together in “alliances” to fight major battles and to
compare victories—a social dynamic that creates a sense of obligation.
68
Nonpaying players who participate in these activities enhance the games’
allure for the whales and potentially promote the games to others.
69
For both
of these reasons, it is in game developers’ interest to ensure that their
customers, both paying and nonpaying, spend as much “time on device” as
possible.
B. An Overview of Habit-Forming Design
Developers drive time on device by producing user interface (UI) and user
experience (UX) design practices that draw the user into compulsive
behavior. Natasha Dow Schüll, a leading researcher of the casino industry,
has commented that UX design is “about looping people into these flows of
incentive and reward. Your coffee at Starbucks, your education software,
your credit card, the meds you need for your diabetes. Every consumer
interface is becoming like a slot machine.”
70
Insiders in the tech industry share Schüll’s assessment. In a recent BBC
documentary, for instance, several former Facebook executives stepped
forward to explain how core design features they had personally invented
66. Andrew Handley, What They Won’t Tell You About Game of War, LOOPER,
https://www.looper.com/34210/untold-truth-game-war/ [https://perma.cc/QP9A-PPF3] (last
visited Aug. 22, 2019).
67. Leslie Katz, This Game Will Make You Obsessed with Rattan Ottomans and
Watercolor Whales, CNET
(Sept. 6, 2018, 6:00 AM), https://www.cnet.com/news/design-
home-game-has-me-obsessed-with-rattan-ottomans-watercolor-whales/
[https://perma.cc/D6V8-9FKZ].
68. Brian Shea, Opinion—Marvel Strike Force Is One of the Most Addictive Games I’ve
Played All Year, G
AME INFORMER (June 8, 2018, 8:58 PM), https://www.gameinformer.com/
opinion/2018/06/08/opinion-marvel-strike-force-is-one-of-the-most-addictive-games-ive-
played-all [https://perma.cc/4JL8-E54U].
69. See Schubert, supra note 53 (“[T]he game is served well by having a large free-to-
play population. . . . Even if only a fraction [of players] are paying for your game, having a
free population of a couple million means that you have a couple million people potentially
evangelizing the game to their friends and family.”).
70. Ian Leslie, The Scientists Who Make Apps Addictive, E
CONOMIST: 1843 (Oct./Nov.
2016), https://www.1843magazine.com/features/the-scientists-who-make-apps-addictive
[https://perma.cc/DK52-M22V].
142 FORDHAM LAW REVIEW [Vol. 88
exploited weaknesses in human psychology.
71
These design features include
the “Like” button, the News Feed’s endless scroll, which enables the user to
see more posts without stopping to ask for more, and even the choice to use
the color red in notification badges.
72
Sean Parker, the inventor of Napster and later the first company president
at Facebook, told interviewers that UI designers have “exploit[ed] a
vulnerability in human psychology” by “giv[ing] users ‘a little dopamine
hit’” when doing so would help the company “consume as much of [the
users’] time and conscious attention as possible.”
73
Aza Raskin, inventor of
the infinite scroll at Mozilla, told interviewers that behind every phone
screen, there are about a thousand engineers who work on increasing
addictiveness.
74
Leah Pearlman, inventor of the “Like” button, admitted that
the feature had her “kind of addicted to the feedback.”
75
“It’s . . . exactly the
kind of thing that a hacker like myself would come up with,” said Sean
Parker.
76
“The inventors [and] creators,” he continued, “understood [the
addictive potential] consciously. And we did it anyway.”
77
The techniques designers use to produce these effects vary, but Nir Eyal’s
2014 bestseller, Hooked: How to Build Habit Forming Products, helpfully
boils them down to a four-step process. “The ultimate goal of a habit-
forming product,” he writes, “is to solve the user’s pain by creating an
association so that the user identifies the company’s product or service as the
source of relief.”
78
The ideal is “unprompted user engagement, bringing
users back repeatedly, without depending on costly advertising or aggressive
messaging.”
79
In the first step, the user is “triggered” to use the product.
80
Early on, this
trigger is “external.”
81
It comes from advertising, or from an invitation from
a friend to join Facebook or Instagram.
82
Or perhaps the user is “triggered”
to download a game from iTunes or the App Store as a response to what Eyal
terms the “pain”
83
of boredom. After repeated use, the user picks up “internal
71. See Hilary Andersson, Social Media Apps Are “Deliberately” Addictive to Users,
BBC NEWS (July 4, 2018), https://www.bbc.com/news/technology-44640959 [https://
perma.cc/27T4-GD9U].
72. See id.
73. Olivia Solon, Ex-Facebook President Sean Parker: Site Made to Exploit Human
“Vulnerability,” G
UARDIAN (Nov. 9, 2017, 3:11 PM), https://www.theguardian.com/
technology/2017/nov/09/facebook-sean-parker-vulnerability-brain-psychology
[https://perma.cc/WEN7-TKU6].
74. Andersson, supra note 71.
75. Id.
76. Solon, supra note 73.
77. See Thuy Ong, Sean Parker on Facebook: “God Only Knows What It’s Doing to Our
Children’s Brains,” V
ERGE (Nov. 9, 2017, 10:21 AM), https://www.theverge.com/
2017/11/9/16627724/sean-parker-facebook-childrens-brains-feedback-loop
[https://perma.cc/Q6FU-UZAN].
78. E
YAL, supra note 1, at 52.
79. Id. at 5.
80. Id. at 39–60.
81. Id. at 41.
82. Id. at 44–47.
83. Id. at 52.
2019] REGULATING HABIT-FORMING TECHNOLOGY 143
triggers” that prompt the user to take action without any need for outside
intervention.
84
Simple habit formation may do the job, as the user comes to
rely on a phone app as a quick cure for boredom.
85
But many apps set goals
for the user which encourage the process of internal trigger formation.
86
Snapchat’s “streak” feature, for instance, counts the number of consecutive
days that two friends have used the platform to communicate with each
other.
87
If either friend misses a day, then the streak is lost.
88
In step two, the triggered user takes action by downloading, registering for,
or using the application.
89
Scrolling through the Facebook News Feed or
“pulling down” to see recent tweets constitutes action, and Eyal advises that
successful developers make the barriers to action as low as possible; ideally,
the user should be able to act without stopping to think before doing so.
90
On
Pinterest, for example, users can simply scroll forever without ever needing
to hit a “See More” button.
91
The user can simply keep scrolling, and the
service will keep serving up more pinned items.
92
If the user stops scrolling,
the site displays some images as cut off at the fold, indicating that there is
still more to see.
93
The endless scroll monetizes the same human tendency that was illustrated
by the famous “bottomless bowl of soup” experiment.
94
One group was
served a normal bowl of tomato soup.
95
The other group was served a bowl
of tomato soup that was literally “bottomless”: a tube under the table kept
pumping in more soup.
96
The group with the bottomless bowls of soup ate
on average about 73 percent more soup seemingly because, without having
to stop and ask for seconds, they had no opportunity to reflect on how hungry
84. Id. at 47–51.
85. Id. at 28–31.
86. See Göran Wågström, Is Social Media Addiction Worse Than Cigarettes?, F
ORBES
(Nov. 21, 2018, 7:45 AM), https://www.forbes.com/sites/forbestechcouncil/2018/11/21/is-
social-media-addiction-worse-than-cigarettes [https://perma.cc/Y86U-CDJ7].
87. Social Media Must Be Made Less Addictive, Says Watchdog, BBC
NEWS (June 12,
2018), https://www.bbc.com/news/technology-44452265 [https://perma.cc/BM5B-RN68].
88. Id. In 2018, Anne Longfield, Children’s Commissioner for England, called for
Snapchat to drop the streak feature. Id.
89. E
YAL, supra note 1, at 61.
90. Id. at 67.
91. Id. at 79.
92. Id.
93. Id. at
110.
94. See generally Brian Wansink et al., Bottomless Bowls: Why Visual Cues of Portion
Size May Influence Intake, 13 O
BESITY RES. 93 (2005). Note, however, that apparent
methodological deficiencies have brought the paper under scrutiny as many other papers to
emerge from Professor Wansink’s Food and Brand Lab at Cornell University have been
retracted. Pete Etchells & Chris Chambers, Mindless Eating: Is There Something Rotten
Behind the Research?, G
UARDIAN (Feb. 16, 2018, 6:21 AM), https://www.theguardian.com/
science/head-quarters/2018/feb/16/mindless-eating-brian-wansink-is-there-something-rotten-
behind-the-research [https://perma.cc/RY9V-MTCZ].
95. Wansink et al., supra note 94, at 95.
96. Id. at 96.
144 FORDHAM LAW REVIEW [Vol. 88
they actually were.
97
This bottomless approach to the scroll is widespread
among social apps today.
98
In step three, after the user takes action, the system serves up a “variable
reward”—a reward that, as on a slot machine, varies on a random basis.
99
The core of the addiction machine, variable rewards’ addictiveness was first
discovered in a series of experiments on pigeons and rats.
100
These animals
were given a food-pellet dispenser with a lever.
101
Predictable results meant
that these animals would push the lever when they wanted a snack.
102
But
when the rewards were randomized, the animals would press endlessly on
the lever, hungry or not.
103
The variable reward is overt in many gaming applications. As explained
in Part II, many online games offer “loot boxes” to players in exchange for
micropayments of a dollar or two.
104
After paying, the loot box opens in an
animated sequence to reveal a pseudorandomized set of in-game items such
as character upgrades.
105
Some items are rare, and some are junk; the
similarity to the ups-and-downs appeal of the slot machine is impossible to
miss.
Other applications involve a subtler variable reward component. Perhaps
the most common application of the variable reward principle is in the “pull
to refresh” feature found in most phone apps.
106
“Pull down” in Facebook
or in your podcasting app, for example, and the app will load a new, and
unpredictable, set of updates. The feature contributes nothing to
functionality; apps such as Facebook could just as easily notify the user when
an event occurs. Instead, it is designed, like the pull of a slot machine or the
press of a food-dispenser lever, to engage the user.
107
And as for the
“loading”? It is usually a contrivance—nothing but a visual animation
displayed to whet the appetite.
108
97. Id.
98. Facebook, Instagram, Twitter, Quora, Snapchat, and Pinterest all employ the infinite
scroll. Vinit Joshi, The Problem with Infinite Scrolling Feeds, M
EDIUM (July 18, 2017),
https://blog.newtonhq.com/the-problem-with-infinite-scrolling-feeds-e3d1aad2c078
[https://perma.cc/CQ3V-H7FH].
99. E
YAL, supra note 1, at 95.
100. Id. at 99.
101. Id.
102. Id.
103. Id.
104. See infra Part II.A.3.
105. See Senate Environment and Communications References Committee, Parliament of
Australia, Gaming Micro-Transactions for Chance-Based Items
(Report, November 2018) 2–
3
[1.11]–[1.13].
106. See Tristan Harris, The Slot Machine in Your Pocket, S
PIEGEL ONLINE (July 27, 2016,
5:25 PM), http://www.spiegel.de/international/zeitgeist/smartphone-addiction-is-part-of-the-
design-a-1104237.html [https://perma.cc/V3AA-ZBWM].
107. See id.; Julian Morgans, The Secret Ways Social Media Is Built for Addiction, V
ICE
(May 21, 2017, 4:30 AM), https://www.vice.com/en_uk/article/vv5jkb/the-secret-ways-
social-media-is-built-for-addiction [https://perma.cc/BHV7-XQ6E].
108. Morgans, supra note 107 (“You know when you open Instagram or Twitter and it
takes a few moments to load updates? That’s no accident. Again, expectation is part of what
makes intermittent variable rewards so addictive. This is because without that three-second
2019] REGULATING HABIT-FORMING TECHNOLOGY 145
Small design tweaks can be extremely effective at enhancing the sense of
reward.
109
T. Dalton Combs of Boundless Mind (formerly known as
Dopamine Labs) explains how his firm might enhance a product’s addictive
neural “kick”: “When you text a friend a ‘Congratulations’ iMessage, you
get confetti falling from the top—those little elements are called UI sugar. . . .
Then, on a user-by-user basis, we change that sugar to make it more or less
enticing to build that habit over time.”
110
In step four, after collecting the variable reward, the user is encouraged to
invest somehow in the product by doing a “bit of work” on it—for example,
by updating a Facebook profile or posting a photo to Instagram or Pinterest.
This investment is then used to restart the cycle.
111
After sinking this
investment into the product, the user becomes “internally triggered” to come
back and check on its performance: Who commented? What did they say?
How many likes?
112
The user’s investment can be leveraged to trigger and reward others as
well. Post a photo to Facebook with one of your friends in it, and Facebook
will send an external trigger to your friend with a push notification.
113
Your
friend will then take action by opening the Facebook app, and in return
Facebook will show your friend the variable reward of a photo that may be
flattering or may be embarrassing.
114
The sweet spot for Facebook, of
course, is for your friend’s social anxiety to internally trigger them to check
the photo later and see how other Facebook users have evaluated their
appearance. Those users, through their own investments, will have variably
rewarded or punished your friend with likes, comments, or silence.
115
Along the way, your friend’s actions and investments will have helped
Facebook “improve the product” by giving the algorithms information about
what “engages” your friend. Facebook’s algorithms will then incorporate
this information into the schedule of “triggers” and “rewards” that it serves
delay, Instagram wouldn’t feel variable. There’s no sense of will I win? because you’d know
instantly. So the delay isn’t the app loading. It’s the cogs spinning on the slot machine.”).
109. Shieber, supra note 3 (noting that “Dopamine Labs predicts they can add 10 percent
to a company’s revenues”).
110. April Glaser & Will Oremus, Fighting Tech with Tech, S
LATE (Nov. 24, 2017),
https://slate.com/technology/2017/11/dopamine-labs-t-dalton-combs-on-tech-addictions-
and-habits-on-if-then-podcast.html [https://perma.cc/P6QU-PSAY].
111. E
YAL, supra note 1, at 143–47.
112. Id. at 154–61.
113. See generally What Types of Notifications Does Facebook Send?, F
ACEBOOK,
https://www.facebook.com/help/1668906000006551 [https://perma.cc/JQX7-YK5T] (last
visited Aug. 22, 2019).
114. See generally Britney Fitzgerald, One in Four Women Deliberately Posts Unflattering
Pictures of Facebook Friends, Survey Finds, H
UFFINGTON POST (July 3, 2012, 1:21 PM),
https://www.huffpost.com/entry/one-in-four-women-unflattering-photos-
facebook_n_1646499 [https://perma.cc/8BBZ-6QAU]; What Is Tagging and How Does It
Work?, F
ACEBOOK, https:// www.facebook.com/help/124970597582337/ [https://perma.cc/
446W-2VVQ] (last visited Aug. 22, 2019).
115. See Morgans, supra note 107.
146 FORDHAM LAW REVIEW [Vol. 88
to your friend—particularly in the News Feed, the endless scroll of ads and
variable rewards that greet users every time they open the app.
116
C. What’s the Harm?
Habit-forming design causes at least three types of harm: addiction, strain
on social norms, and degradation of public discourse.
1. Extreme Cases of Problem Use
First, there is acute and severe harm that occurs to a relatively small group
of “problem users”—time and money losses generally similar to those
suffered by problem gamblers. In one extreme case, an anonymous poster
on Reddit wrote that the F2P mobile game Final Fantasy Brave Exvius
appealed to him as “[a] short format game that I could play for 5 or 10
minutes and put it away.”
117
He said, “I am currently $15,800 in debt. My
wife no longer trusts me. My kids, who ask me why I am playing Final
Fantasy all the time, will never understand how I selfishly spent money I
should have been using for their activities.”
118
This “problem gaming disorder,” as defined by the WHO,
119
“affects only
a small proportion of people who engage in digital- or video-gaming
activities.”
120
The situation is similar to that in the gambling industry, where
only a small percentage of the population develops a serious habit.
121
Industry leaders in both the tech and the gambling sectors emphasize the
behavioral nature of the problem, and they suggest that they are not
responsible for the small minority’s problems with impulse control.
122
People can develop behavioral addictions to any product, they say, or any
behavior, and individual propensities are often a determining factor.
123
116. See Josh Constine, How Facebook News Feed Works, TECHCRUNCH (Sept. 6, 2016),
https://techcrunch.com/2016/09/06/ultimate-guide-to-the-news-feed/
[https://perma.cc/CP5X-D45H].
117. Nothing024, A Whale of a Tale, R
EDDIT (Dec. 13, 2017), https://www.reddit.com/
r/FFBraveExvius/comments/7jmezv/a_whale_oo_a_tale/ [https://perma.cc/HZ5R-E5WL]
(posting anonymously).
118. Id.
119. See supra note 6 and accompanying text.
120. See Gaming Disorder, supra note 6; see also Charlotte Thorensen Wittek et al.,
Prevalence and Predictors of Video Game Addiction: A Study Based on a National
Representative Sample of Gamers, 14 I
NT’L J. MENTAL HEALTH & ADDICTION 672, 672 (2015)
(showing 1.4 percent “addicted gamers” and 7.3 percent problem gamers among a randomly
selected sample of Norwegian gamers).
121. See supra note 64 and accompanying text.
122. N
ATASHA DOW SCHÜLL, ADDICTION BY DESIGN: MACHINE GAMBLING IN LAS VEGAS
18 (2012).
123. Howard Shaffer, director of the Harvard Medical School Division on Addiction, was
one of the first to call slot machines “the crack cocaine of gambling.” Id. But after forming
close ties with the American Gaming Organization, an industry group, Shaffer has qualified
the “crack cocaine” statement considerably. Id. at 262–65. In an interview with CBS’s 60
Minutes, he told interviewer Leslie Stahl that even actual crack cocaine should not be
considered addictive because “[o]nly a small minority” of cocaine users develop a cocaine
addiction, “and the same would be true with gambling.” 60 Minutes: Slot Machines: The Big
2019] REGULATING HABIT-FORMING TECHNOLOGY 147
But the industry plays a strong contributory role. Both the casino and the
mobile gaming industries draw most of their revenue from that same small
percentage of vulnerable users.
124
Developers have strong incentives to drive
problem use, just as casinos do, and they make every effort to do so.
The similarity between tech addiction and gambling addiction is less
striking outside of gaming, where in-app purchases are less common. But
the heavy “gamification” that pervades these applications, together with the
developers’ shared imperative to drive engagement at all costs, suggests that
the phenomenon of “problem use” extends beyond the gaming sector.
2. Social Norms and Lifestyle
The clinical debate around “gaming addiction” overlooks a more
significant consumer protection issue: the average person’s compulsive
attachment to smartphones and, in particular, the social media and gaming
applications that run on them. One study shows that twentysomethings check
their phones eighty-five times a day on average, and that more than half of
these check-ins last less than thirty seconds.
125
The study shows that average
Americans use their phones for five hours in total every day.
126
Still, another
study shows that users touch their phones 2617 times per day.
127
Norms of constant distraction have emerged to feed these habits. It is now
a social norm, for example, for two people eating together to keep their
phones face up on the table and check incoming messages during their
meal.
128
It is increasingly common to stop in the middle of a conversation to
respond to a text.
129
One survey by Verizon reports that 77 percent of users
turn to their phones first thing in the morning and nearly 90 percent use their
phone in the bathroom.
130
Governments in liberal societies should not generally take activities off the
table simply because they are a waste of time. But it is unclear whether
regulation of addictive design would necessarily substitute the government’s
judgment for consumers’ judgment. People may be spending much more
Gamble (CBS television broadcast Jan. 7, 2011), https://www.cbsnews.com/news/slot-
machines-the-big-gamble-07-01-2011/2/ [https://perma.cc/D4QR-B4Q7].
124. See supra notes 60–66 and accompanying text.
125. Sally Andrews et al., Beyond Self-Report: Tools to Compare Estimated and Real-
World Smartphone Use, PLOS
ONE (Oct. 28, 2015), https://journals.plos.org/plosone/
article?id=10.1371/journal.pone.0139004 [https://perma.cc/7RDX-E9MN].
126. Id.
127. Winnick, supra note 4.
128. See Sandee LaMotte, Smartphone Addiction Could Be Changing Your Brain, CNN
(Dec. 1, 2017, 11:16 AM), https://www.cnn.com/2017/11/30/health/smartphone-addiction-
study/index.html [https://perma.cc/CY95-UGFK].
129. Emma Seppälä, Are You “Phubbing” Right Now?: What It Is and Why Science Says
It’s Bad for Your Relationships, W
ASH. POST (Oct. 13, 2017), https://
www.washingtonpost.com/news/inspired-life/wp/2017/10/13/are-you-phubbing-right-now-
what-it-is-and-why-science-says-its-bad-for-your-relationships [https://perma.cc/2U99-
NNQV].
130. Edward Cardenas, Study: 9 Out of 10 People Use Their Cell Phone in the Bathroom,
CBS D
ET. (June 30, 2015, 3:06 PM), https://detroit.cbslocal.com/2015/06/30/study-9-out-of-
10-people-use-their-cellular-phone-in-the-bathroom/ [https://perma.cc/3D75-SU99].
148 FORDHAM LAW REVIEW [Vol. 88
time checking their phones than they realize or prefer. If the regulatory goal
is simply to make product design less manipulative, then regulation in
principle exists to enhance rather than diminish tech users’ freedom of
choice. And of course, any concerns that restrictions on habit-forming design
are overly paternalistic can be ignored insofar as the regulation is targeted at
children’s products.
It is also worth considering the miscellaneous external harms that flow
from compulsive tech use. Early studies indicate drops in productivity,
empathy, and general intelligence when smart devices are in the room.
131
High-income workers and executives in the tech industry disproportionately
prefer low-tech and no-tech private schooling for their children.
132
Even to the extent that some of these harms might be called “aesthetic” or
“cultural,” recall that governments regulate simple nuisances all the time.
Think of the do-not-call list,
133
or the law that attempts to regulate spam
email,
134
or the local ordinances that regulate shrubbery and house paint.
135
In each of these cases, the government has stepped in to regulate a mostly
aesthetic harm in order to enhance general quality of life.
3. The Public Sphere
The third harm is less focused but more severe: addictive tech’s distortive
effect on public discourse and ultimately the democratic process.
There are reasons to suspect that the old marketplace of ideas is undergoing
renovation as a state-of-the-art casino. Just like twenty-first-century casinos,
131. See Robinson Meyer, Your Smartphone Reduces Your Brainpower, Even If It’s Just
Sitting There, A
TLANTIC (Aug. 2, 2017), https://www.theatlantic.com/technology/
archive/2017/08/a-sitting-phone-gathers-brain-dross/535476/ [https://perma.cc/6ZKW-
TH3U]; see also Robert Rosenberger, Yes, Smartphone Use Is Probably Behind the Spike in
Driving Deaths. So Why Isn’t More Being Done to Curb It?,
SLATE (Dec. 28, 2017, 5:57 AM),
https://slate.com/technology/2017/12/yes-smartphone-use-is-probably-behind-the-spike-in-
vehicle-related-deaths-so-why-isnt-more-being-done-to-curb-distracted-driving.html
[https://perma.cc/8BK9-ZTUV].
132. See Nellie Bowles, The Digital Gap Between Rich and Poor Kids Is Not What We
Expected, N.Y.
TIMES (Oct. 26, 2018), https://www.nytimes.com/2018/10/26/style/digital-
divide-screens-schools.html [https://perma.cc/EBP5-JPLK]; Matthew Jenkin, Tablets Out,
Imagination In: The Schools That Shun Technology, G
UARDIAN (Dec. 2, 2015, 2:00 AM),
https://www.theguardian.com/teacher-network/2015/dec/02/schools-that-ban-tablets-
traditional-education-silicon-valley-london [https://perma.cc/25ZT-ATQQ]; Chris Weller, An
MIT Psychologist Explains Why So Many Tech Moguls Send Their Kids to Anti-Tech Schools,
B
US. INSIDER (Nov. 7, 2017, 12:42 PM), https://www.businessinsider.com/sherry-turkle-why-
tech-moguls-send-their-kids-to-anti-tech-schools-2017-11 [https://perma.cc/SSZ2-R24Y].
133. The FTC bars calls to any telephone number in the “‘do-not-call’ registry” without
express written consent or an established business relationship. 16 C.F.R. § 310.4(b)(1)(iii)(B)
(2019).
134. Controlling the Assault of Non-Solicited Pornography and Marketing Act of 2003, 15
U.S.C. §§ 7701–7713 (2012).
135. See Members of the City Council v. Taxpayers for Vincent, 466 U.S. 789, 794 (1984)
(upholding a content-neutral ban on signs and posters justified as an effort to avoid “clutter
and visual blight”); Davis v. City of Worthington, No. C-2-00-65, 2001 WL 175262, at *4
(S.D. Ohio Feb. 13, 2001) (holding that “[p]laintiff’s ‘natural landscaping’ lacks ‘the sufficient
communicative elements to bring it within the protections afforded by the First
Amendment’”).
2019] REGULATING HABIT-FORMING TECHNOLOGY 149
social media platforms surveil their users to ensure that the state of play is
always adapting to the players’ emotional needs. For example, “luck
ambassadors” in a casino use real-time data from slot machines to anticipate
the point when a disappointed or fatigued gambler might be ready to walk
away from the machine.
136
When that happens, it is time to approach the
gambler with a free drink, or tickets to a show, or some other pick-me-up.
137
Facebook executives bragged about an eerily similar ability to a group of
Australian bankers: the platform can detect when teenage users “feel
‘insecure,’ ‘worthless’ and ‘need a confidence boost.’”
138
This is not to say that online platforms necessarily drive engagement by
boosting mood. To the contrary, it seems that the most reliable engagement
drivers are messages that stimulate feelings of outrage and group
identification.
139
Many recommendation algorithms—YouTube’s most
prominently—have been shown repeatedly to send users along a
“radicalizing” path.
140
These platforms in essence have devised a guided
daydream that monetizes our weaknesses for tribalism and negative
rumination.
Social media platforms’ constant pandering to these emotions has likely
informed, if not driven, the content of today’s angry tribal politics. But the
engagement model has also warped the underlying physics of public
discussion. Finding personally interesting content in the pre-internet era
required some amount of deliberation or at least intention—a person would
find new sources either through friends or through store owners, or maybe
by browsing in a bookstore or library. Today’s recommendation algorithms,
on the other hand, keep users always “on the rails,” to borrow a term from
136. See Evan Selinger, Inside the Casino, the House is Always Watching, CHRISTIAN SCI.
MONITOR (June 3, 2015), https://www.csmonitor.com/World/Passcode/Passcode-Voices/
2015/0603/Inside-the-casino-the-house-is-always-watching [https://perma.cc/P9NR-KJTP];
see also Andrew Thompson, Engineers of Addiction: Slot Machines Perfected Addictive
Gaming. Now, Tech Wants Their Tricks, V
ERGE (May 6, 2015), https://www.theverge.com/
2015/5/6/8544303/casino-slot-machine-gambling-addiction-psychology-mobile-games
[https://perma.cc/8HT9-7TDQ].
137. See Leslie, supra note 70; Plumer, supra note 27; Selinger, supra note 136; see also
John Rosengren, How Casinos Enable Gambling Addicts, A
TLANTIC (Dec. 2016),
https://www.theatlantic.com/magazine/archive/2016/12/losing-it-all/505814/
[https://perma.cc/ZR3P-AZ29].
138. Sam Levin, Facebook Told Advertisers It Can Identify Teens Feeling “Insecure” and
“Worthless,” G
UARDIAN (May 1, 2017, 3:01 PM), https://www.theguardian.com/
technology/2017/may/01/facebook-advertising-data-insecure-teens [https://perma.cc/X6ME-
KF3D].
139. See supra note 21 and accompanying text.
140. See Mathew Ingram, YouTube’s Secret Life as an Engine for Right-Wing
Radicalization, C
OLUM. JOURNALISM REV. (Sept. 19, 2018), https://www.cjr.org/
the_media_today/youtube-conspiracy-radicalization.php [https://perma.cc/2Y67-X4LS];
Rebecca Lewis, Alternative Influence: Broadcasting the Reactionary Right on YouTube,
D
ATA & SOC’Y (Sept. 18, 2018), https://datasociety.net/output/alternative-influence/
[https://perma.cc/PG6E-UYA8]; Zeynep Tufekci, YouTube, the Great Radicalizer, N.Y.
TIMES (Mar. 10, 2018), https://www.nytimes.com/2018/03/10/opinion/sunday/youtube-
politics-radical.html [https://perma.cc/QF89-9A8K].
150 FORDHAM LAW REVIEW [Vol. 88
video gaming.
141
A user with a budding interest does not need to seek it
out—the News Feed will ensure that this content finds the user.
Importantly, the user may not realize that they are predisposed to engage
with a given kind of content; the algorithm takes the work of self-discovery
off the user’s hands. In some cases, these recommendations are helpful. But
they can also be harmful—for instance, when YouTube helps users
“activate” authoritarian tendencies that might otherwise have lain
dormant.
142
By serving users’ “revealed preferences” rather than stated preferences,
user engagement algorithms largely crowd out the individual’s role in
cultivating a set of interests and values.
143
But as on a slot machine, the
device must cultivate an illusion of control to maintain the compulsion. Slot
machines carry this principle to an extreme; users are led to believe that they
know exactly when to stop the spinning reels, when in truth the payout has
already been calculated.
144
On YouTube, the effect is more subtle; the user
may choose to discriminate among a seemingly endless universe of sources,
when in truth the menu of options has already been rigorously pruned to fit
the user’s own preexisting temptations.
145
Facebook users who “like,” share,
and comment on what they see are given an apparent opportunity to express
themselves in public discussions, though the recommendation algorithm
determines who will see it and when.
146
141. See Will Hindmarch, Sword-Fighting on a Roller-Coaster: Railroading for the Best
in RPG Play, M
EDIUM (Nov. 13, 2014), https://medium.com/gameplaywright-presents/sword-
fighting-on-a-roller-coaster-railroading-for-the-best-in-rpg-play-547333c80359
[https://perma.cc/F4LC-6Y7C] (describing how “railroading” in video game design leaves the
player a small sphere of agency while marching the player through a series of large-scale
objectives).
142. See Dave Hanley & Alicia Hatch, Social Activation,
DELOITTE INSIGHTS (Feb. 21,
2014), https://www2.deloitte.com/insights/us/en/focus/tech-trends/2014/2014-tech-trends-
social-activation.html [https://perma.cc/2KB4-LYB8] (“The power of social activation is
unleashed when others advocate an organization’s message in their own words to their
network.”). See generally K
AREN STENNER, THE AUTHORITARIAN DYNAMIC (2005) (positing
that authoritarianism is a predisposition that activates when the authoritarian person is exposed
to conditions of normative threat in the form of social difference or disobedience of authority).
143. Julie E. Cohen, What Privacy Is for, 126 H
ARV. L. REV. 1904, 1926–27 (2013)
(“Stimuli tailored to consumptive preferences crowd out other ways in which preferences and
self-knowledge might be expressed . . . .”).
144. S
CHÜLL, supra note 123, at 82–86.
145. See Aaron Smith et al., Many Turn to YouTube for Children’s Content, News, How-
To Lessons, P
EW RES. CTR. (Nov. 7, 2018), https://www.pewinternet.org/2018/11/07/many-
turn-to-youtube-for-childrens-content-news-how-to-lessons/#an-analysis-of-random-walks-
through-the-youtube-recommendation-engine [https://perma.cc/M26K-RRWB] (“[T]he site’s
recommendation engine is responsible for more than 70% of users’ time spent watching videos
on the platform. These sorts of recommendation systems seek to draw viewers to content that
is more engaging to them, potentially keeping them on the site for longer periods of time.”).
146. See Tim Herrera, What Facebook Doesn’t Show You, W
ASH. POST (Aug. 14, 2014),
https://www.washingtonpost.com/news/the-intersect/wp/2014/08/18/what-facebook-doesnt-
show-you/ [https://perma.cc/HYU9-C8VS] (finding that Facebook showed the editor 29
percent of all posts users in his network made during six hours of scrolling through his News
Feed).
2019] REGULATING HABIT-FORMING TECHNOLOGY 151
Speed reinforces the compulsive flow of a slot machine, and social apps
thrive on the same phenomenon.
147
Social platforms, craving data, strive to
make posting as quick and spontaneous as possible; they do not want users
to think for too long before they speak or share.
148
The obvious and
unfortunate result is a more hot-blooded and emotionally reflexive media
environment than was possible on slower media.
149
Inevitably, speakers say
ill-considered things every day, often triggering cycles of mass indignation
and recrimination that enlighten no one and reverberate over multiple news
cycles.
150
“Online mobs” are a commonplace,
151
and they can easily become
real mobs in markets, such as Sri Lanka’s or Myanmar’s, where platforms
have not invested heavily in top-down censorship.
152
The “fire” of violence and mass hatred that concerned the U.S. Supreme
Court in its early free speech opinions appears to catch far more quickly and
easily on social media.
153
Much of this may result from the simple fact that
speakers have been placed in such close and constant contact with each other.
But it also may be that outside behavioral manipulation has compromised the
147. See Anna Johansson, Your Social Media Apps Are as Addictive as Slot Machines—
Should They Be Similarly Regulated?, N
EXT WEB (Mar. 25, 2018),
https://thenextweb.com/contributors/2018/03/25/social-media-apps-addictive-slot-machines-
similarly-regulated/ [https://perma.cc/2RLG-KG8M].
148. E
YAL, supra note 1, at 67–74 (discussing Facebook login prompts and embeddable
Tweet buttons as devices that “prompt users to move quickly” by reducing the necessary
“brain cycles—the level of mental effort and focus required to take an action”).
149. See Jayson DeMers, 59 Percent of You Will Share This Article Without Even Reading
It, F
ORBES (Aug. 8, 2016, 2:27 PM), https://www.forbes.com/sites/jaysondemers/2016/08/08/
59-percent-of-you-will-share-this-article-without-even-reading-it [https://perma.cc/U5L9-
PYHX]; supra note 21 and accompanying text.
150. See Anna Johansson, Social Media Mobs Are Worse for Society Than the People They
Shame, N
EXT WEB (Oct. 31, 2018, 4:00 AM), https://thenextweb.com/contributors/
2018/10/31/social-media-mobs-are-worse-for-society-than-the-people-they-shame/
[https://perma.cc/EJ3U-HJGS].
151. See Amanda Hess, Some Online “Mobs” Are Vicious. Others Are Perfectly Rational.,
N.Y.
TIMES MAG. (Aug. 7, 2018), https://www.nytimes.com/2018/08/07/magazine/some-
online-mobs-are-vicious-others-are-perfectly-rational.html [https://perma.cc/TQS2-7BND]
(“Twitter . . . incentivizes people with no true investment in a controversy to weigh in
anyway.”).
152. See Vindu Goel et al., In Sri Lanka, Facebook Contends with Shutdown After Mob
Violence, N.Y.
TIMES (Mar. 8, 2018), https://www.nytimes.com/2018/03/08/technology/sri-
lanka-facebook-shutdown.html [https://perma.cc/M27T-P5GV]; Paul Mozur, A Genocide
Incited on Facebook, with Posts from Myanmar’s Military, N.Y.
TIMES (Oct. 15, 2018),
https://www.nytimes.com/2018/10/15/technology/myanmar-facebook-genocide.html
[https://perma.cc/K7CS-UPKV].
153. See Ben Popken, How WhatsApp Became Linked to Mob Violence and Fake News—
and Why It’s Hard to Stop, NBC
NEWS (Nov. 2, 2018, 4:31 AM),
https://www.nbcnews.com/tech/tech-news/how-whatsapp-became-linked-mob-violence-
fake-news-why-it-n929981 [https://perma.cc/F395-F5A2]; Sunita Sohrabji, Social Media
Platforms Should Not Encourage Mob Violence: India IT Minister Ravi Shankar Prasad in
Silicon Valley, I
NDIA W. (Aug. 30, 2018), https://www.indiawest.com/news/
global_indian_social-media-platforms-should-not-encourage-mob-violence-india-
it/article_8792ead0-acac-11e8-88ac-e3e297231f7b.html [https://perma.cc/S732-CZBA] (“In
an eerie pattern, the victims, largely Muslim male visitors, are branded ‘child abductors’ and
brutally beaten, often to death. WhatsApp groups have been accused of promoting fake
news—falsely accusing the victims of violence—to foment mob violence.”).
152 FORDHAM LAW REVIEW [Vol. 88
role of individual deliberation as the public sphere’s most effective natural
safeguard against disaster.
154
And this is to say nothing of the unprecedented
opportunities that platforms’ engagement-driving techniques have given
saboteurs who look to spread misinformation and engineer social conflict.
The dominant prescription for these problems today is either to encourage
or require platforms to take down dangerous content. The fiat-based solution
called “content moderation” would be condemned as an unconstitutional
prior restraint if it were instituted by a state or the federal government. It is
a “private” solution to the mess, but this formalism masks a fundamental
betrayal of the free speech concept. Justice Louis Brandeis wrote of free
speech both as “an end and as a means” to preserve liberty.
155
Addictive
social media architecture threatens free speech in both senses.
II.
PROSPECTS FOR REGULATION
Today, the problem of compulsive tech use is not widely understood, or
widely acknowledged, as either a serious public problem or a serious private
problem for any but the most desperately addicted individuals. There is little
public or institutional drive to address it. Scattered K–12 schools require
students to put cell phones away during the school day (even as many more
have chosen to “innovate” by providing all students with branded, internet-
capable tablet devices under discounted contracts with manufacturers).
156
Private nonprofits have run ad campaigns encouraging parents to put their
cell phones away at the family dinner table.
157
But the lack of any strong or
consistent public response seems to reflect an equivocal position about
humanity’s deepening absorption into its “smart” devices: the times and the
norms are understood to be changing, some ways good, some ways bad, but
almost entirely within a rubric of personal or parental responsibility.
Much of this laissez-faire attitude is rooted in the underlying technologies
being so new. There has been no high-profile addiction crisis—nothing in
the ballpark of the opioid epidemic, for example—to capture public attention.
Even to the extent that the public is aware of the problem, it can at times
154. See Cohen, supra note 143, at 1917–18 (“The modulated society . . . continually
adjust[s] the information environment to each individual’s comfort level. Liberal democratic
citizenship requires a certain amount of discomfort—enough to motivate citizens to pursue
improvements in the realization of political and social ideals. The modulated citizenry lacks
the wherewithal and perhaps even the desire to practice this sort of citizenship.”).
155. Whitney v. California, 274 U.S. 357, 375 (1927) (Brandeis, J., concurring).
156. See Carolyn Thompson, More US Schools Consider Allowing Students to Use
Cellphones, C
HRISTIAN SCI. MONITOR (Apr. 3, 2018), https://www.csmonitor.com/USA/
Education/2018/0403/More-US-schools-consider-allowing-students-to-use-cellphones
[https://perma.cc/5DM2-6MTM]; Leo Versel, As Cell Phones Proliferate in K–12, Schools
Search for Smart Policies, E
DUC. WEEK (Feb. 8, 2018, 11:21 AM), https://
blogs.edweek.org/edweek/DigitalEducation/2018/02/smartphones_student_learning_classro
oms_K12_education.html [https://perma.cc/QTS7-P3GG].
157. See, e.g., Tim Nudd, Will Ferrell’s Darkly Comic New Ads Show How Our Devices
Are Ruining Family Time, A
D WEEK (Oct. 19, 2017), https://www.adweek.com/brand-
marketing/will-ferrells-darkly-comic-new-ads-show-how-our-devices-are-ruining-family-
time/ [https://perma.cc/MJD5-Z77M].
2019] REGULATING HABIT-FORMING TECHNOLOGY 153
appear unlikely that strong regulation is even possible. Too many lawmakers
lack basic understanding of the subject matter, as Facebook CEO Mark
Zuckerberg’s 2018 congressional hearings revealed,
158
and tech industry
lobbyists have eagerly offered their own expertise to compensate.
159
The state of the art, moreover, is still primitive and unthreatening
compared with what may be coming in the next ten or twenty years. As
transformative as early twenty-first-century smartphones have been, the
major tech companies are moving toward products that are significantly more
invasive and omnipresent.
In 2017, for example, Facebook announced that it plans to roll out its
product in “augmented reality”: wearable products that overlay a consumer
interface onto the user’s perception of real-world environments.
160
And
around the same time, the company announced that it was developing a
wearable product that allows the user to type by thinking.
161
At a minimum,
these extensions of Facebook’s product will expand surveillance, targeted
advertising, and gamification into the entire phenomenological space. At a
maximum, user dependency and stimulus response will empower Facebook
to transcend the whole concept of “advertising” and sell user stimulus
response itself to clients. Much, as always, will depend on Facebook’s own
internal culture of ethics and self-restraint.
If future developments along these lines raise public concern, then the
regulatory response should escalate with the sense of urgency. Light-touch
responses would try to help consumers make good decisions. More dramatic
responses would simply disable products’ most dangerous features. Some or
all of these protective measures may apply to minors or young children
exclusively or, if the sense of emergency is especially pronounced, the
protections may apply across the market.
Part II.A discusses the requirements that might be imposed on app
developers and distributors as part of an effort to police addictive tech. These
run the gamut from simple labeling and disclosure requirements to total
product bans, as well as a few more novel intermediate options. Part II.B
158. See Chris Cillizza, How the Senate’s Tech Illiteracy Saved Mark Zuckerberg, CNN
(Apr. 11, 2018, 3:58 PM), https://www.cnn.com/2018/04/10/politics/mark-zuckerberg-
senate-hearing-tech-illiteracy-analysis/index.html [https://perma.cc/FJA5-QUCJ].
159. See S
HOSHANA ZUBOFF, THE AGE OF SURVEILLANCE CAPITALISM: THE FIGHT FOR A
HUMAN FUTURE AT THE NEW FRONTIER OF POWER 105 (2019) (“Google and Facebook
vigorously lobby to kill online privacy protection, limit regulations, weaken or block privacy-
enhancing regulation, and thwart every attempt to circumscribe their practices . . . .”).
160. See Patrick Moorhead, Facebook F8 Day 2: 10 Year Roadmap for VR/AR, Wireless,
Brain Interfaces and Open-Sourced AI, F
ORBES (Apr. 20, 2017, 3:03 PM),
https://www.forbes.com/sites/patrickmoorhead/2017/04/20/facebook-f8-day-2-recap-10-
year-roadmap-for-vr-comms-brain-interfaces-and-open-sourced-ai/ [https://perma.cc/FC4J-
G3LY].
161. See James Titcomb, Mark Zuckerberg Confirms Facebook Is Working on Mind-
Reading Technology, T
ELEGRAPH (Apr. 19, 2017, 8:09 PM), https://www.telegraph.co.uk/
technology/2017/04/19/mark-zuckerberg-confirms-facebook-working-mind-reading-
technology/ [https://perma.cc/LY5A-TLB2].
154 FORDHAM LAW REVIEW [Vol. 88
describes how these regulatory requirements might come into existence,
whether through existing regulatory authority or through new law.
A. Strategies for Regulation
1. Labeling Requirements
As in the field of tobacco regulation, the government may use product
labeling to encourage consumers to make good choices.
162
A traditional
labeling strategy might require tech products to display general messages
about risks associated with tech addiction and overuse. These messages
could be tailored to the specific risks that a given product presents. A
warning label for Snapchat, for example, could point the user to resources on
goal-setting and social obligation as means of driving engagement.
Minnesota’s proposed loot box law would require certain games to carry the
following label: “Warning: This game contains a gambling-like mechanism
that may promote the development of a gaming disorder that increases the
risk of harmful mental or physical health effects, and may expose the user to
significant financial risk.”
163
But the adaptable nature of tech products would also enable regulators to
move beyond crude general-purpose appeals. Apps might be required to post
the amount of time the user has logged, whether in a given session or over a
longer period such as a day or a week. Other usage indices—such as the
number of posts, likes, logins, swipes, or taps—might be displayed as well.
2. Responsible Use Devices
One step beyond simple disclosure requirements might require apps to
look for patterns of problem use and intervene with warnings or invitations
to take a break at appropriate times. Some casinos have voluntarily adopted
“responsible gaming solutions” such as iView’s Intelligent Gaming Analysis
Platform, or iGap.
164
These casinos, Natasha Dow Schüll explains, have
adopted the software to insure against burnout and maintain lifelong
customers.
165
The concept could easily be adapted to social media platforms,
for instance, which have access to a much richer suite of data-analytics tools
than casinos do.
After Dopamine Labs (now rebranded as Boundless Mind) came under fire
for a 60 Minutes profile describing its ability to make outside firms’ software
162. See, e.g., 15 U.S.C. § 1333 (2012).
163. H.R. 4460, 90th Leg., Reg. Sess. (Minn. 2018); cf. Assemb. 2194, 2017–2018 Leg.,
Reg. Sess. (Cal. 2018); S. 3024, 29th Leg., Reg. Sess. (Haw. 2018); H.R. 2727, 29th Leg.,
Reg. Sess. (Haw. 2018); H.R. 2686, 29th Leg., Reg. Sess. (Haw. 2018).
164. Responsible Gaming Solutions,
IVIEWSYSTEMS, https://www.iviewsystems.com/
responsible-gaming [https://perma.cc/M7HD-WZ7G] (last visited Aug. 22, 2019).
165. S
CHÜLL, supra note 123, at 283.
2019] REGULATING HABIT-FORMING TECHNOLOGY 155
more addictive,
166
the firm released a product called Space that is designed
to help users break away from patterns of compulsive app use.
167
Space
monitors the phone user’s device use for patterns of addiction and then
intervenes between the user and the most alluring apps by forcing the user to
wait for several seconds before the app opens.
168
During this interval, Space
invites the users along for a brief breathing exercise called the “moment of
Zen.”
169
Other major firms have also introduced anti-addiction functionality into
their own products in what appears to be the Silicon Valley equivalent of
carbon offsetting. Although these features are rarely, if ever, turned on by
default,
170
they appear to have real practical value for those who are
interested in them. Facebook and Instagram now allow users to check in on
their daily use, to turn off notifications temporarily, and to set a timer that
goes off when the user exceeds self-imposed limits.
171
It appears that these firms want to frame the addiction matter as a personal
choice and set themselves up as willing helpers for however their customers
wish to live their lives. Importantly, however, the anti-addiction measures in
apps like Facebook and Instagram are almost always turned off by default
and buried in settings menus.
172
But that could change if regulators required
app developers to put anti-addiction functionality in the foreground of the UI
rather than in an out-of-the-way settings menu, and even more so if the anti-
addiction functionality was turned on by default.
166. See Jonathan Shieber, Dopamine Labs Slings Tools to Boost and Reduce App
Addiction, T
ECHCRUNCH (Feb. 13, 2017), https://techcrunch.com/2017/02/13/dopamine-labs-
slings-tools-to-boost-and-reduce-app-addiction/ [https://perma.cc/XF3B-4UTC].
167. See S. C. Stuart, This AI Wants to Give You Space (from Your Apps), PC
MAG. (June
11, 2018, 7:00 AM), https://www.pcmag.com/news/361725/this-ai-wants-to-give-you-space-
from-your-apps [https://perma.cc/Z4G7-PSC7?type=image].
168. See Glaser & Oremus, supra note 110.
169. See id. Helpfully, Boundless Mind designer T. Dalton Combs has suggested to Slate
that the company’s mainline addiction products could be put to great use on actual Zen
meditation apps if any of their designers were interested in a consultation. See id. “Any
meditation app is leaving a lot of Zen on the table by not thinking about how to make a mindful
practice a habit of mind,” he mused. Id.
170. See, e.g., Alfred Ng, Google, Android P Take On Phone Addiction with Android
Dashboard, CNET (May 8, 2018, 11:27 AM), https://www.cnet.com/news/
google-creates-new-android-controls-to-take-on-phone-addiction/ [https://perma.cc/96UQ-
K74A?type=image]; Richard Nieva, Facebook, Instagram Add Tools to Limit Time Spent on
the Apps, CNET (Aug. 1, 2018, 4:00 AM), https://www.cnet.com/news/facebook-instagram-
add-tools-to-limit-time-spent-on-the-apps/ [https://perma.cc/ETA3-SX2F?type=image];
Marguerite Reardon & Shara Tibken, Apple Helps You Kick iPhone Addiction with iOS 12,
CNET
(June 4, 2018, 10:50 AM), https://www.cnet.com/news/apple-helps-you-kick-iphone-
addiction-with-ios-12/ [https://perma.cc/S89X-WU7Q].
171. Nieva, supra note 170.
172. See Alex Hern, Facebook and Instagram to Let Users Set Time Limits, G
UARDIAN
(Aug. 1, 2018, 7:18 AM), https://www.theguardian.com/technology/2018/aug/01/facebook-
and-instagram-to-let-users-set-time-limits [https://perma.cc/K69V-ZRTR].
156 FORDHAM LAW REVIEW [Vol. 88
3. Bans on Dangerous Features
Regulators could ratchet up the pressure further by requiring app
developers to disable various features that are known to drive compulsive
use. Over the past year, for example, international regulators have
investigated, regulated, and, in some cases, criminalized loot boxes—the
pseudo-slot machine alluded to earlier.
173
A pending bill in the U.S. Senate
would prohibit their use in video games marketed to minors.
174
A loot box
is an in-game item—usually portrayed as some sort of closed container—that
gives randomly selected in-game items to the player.
175
The loot box itself
is usually purchased with in-game virtual currencies that can be earned
slowly through play or bought quickly for real cash.
176
One or two dollars is
typical.
177
Loot boxes’ resemblance to slot machines is widely
acknowledged.
178
First, they distribute rewards in a pseudorandom scatter
that is well-known to drive compulsive use.
179
Second, loot boxes often
exploit the same behavioral fallacies as slot machines by exposing users to
artificially frequent “near misses.”
180
These near misses create the mistaken
impression that the user is closing in on a jackpot and encourage further
use.
181
Finally, both loot boxes and slot machines are often operated with
tokens rather than actual currency.
182
The exchange rate from dollar to token
is typically unwieldy by design so that the user lacks any intuitive grasp of
173. See infra notes 216–38 and accompanying text.
174. The bill, sponsored by Senator Josh Hawley, would also prohibit “pay-to-win” games
marketed to minors. Press Release, Josh Hawley, supra note 8.
175. See Cecilia D’Anastasio, Why Opening Loot Boxes Feels Like Christmas, According
to Game Devs, K
OTAKU (Mar. 20, 2017, 2:00 PM), https://kotaku.com/why-opening-loot-
boxes-feels-like-christmas-according-1793446800 [https://perma.cc/9V5U-Z3VS].
176. See, e.g., Daniel Friedman, Destiny 2 Should Steal Overwatch’s Loot Box System or
Shut Down the Eververse, P
OLYGON (Jan. 8, 2018, 10:15 AM), https://www.polygon.com/
2018/1/8/16855180/overwatch-skins-loot-box-prices-destiny-2-crate-analysis
[https://perma.cc/AM59-TEW7] (comparing the sale of loot boxes in Destiny 2 and
Overwatch).
177. See id.
178. See D’Anastasio, supra note 175; see also Benjamin Pu, What Are Loot Boxes? FTC
Will Investigate $30B Video Game Industry, NBC
NEWS (Nov. 28, 2018, 4:27 PM),
https://www.nbcnews.com/tech/tech-news/loot-boxes-gambling-video-games-ftc-look-it-
n941256 [https://perma.cc/HL6L-VPJM]; Alex Wiltshire, Behind the Addictive Psychology
and Seductive Art of Lootboxes, PC
GAMER (Sept. 28, 2017),
https://www.pcgamer.com/behind-the-addictive-psychology-and-seductive-art-of-loot-
boxes/ [https://perma.cc/Z7MV-GQ5B].
179. Wiltshire, supra note 178 (“[T]he dopamine system, which is targeted by drugs of
abuse, is also very interested in unpredictable rewards. Dopamine cells are most active when
there is maximum uncertainty, and the dopamine system responds more to an uncertain reward
than the same reward delivered on a predictable basis . . . .” (quoting Dr. Luke Clark, director
at the Center for Gambling Research at the University of British Columbia)).
180. Id. (describing the role of near-misses in the video game Counter-Strike: Global
Offensive).
181. See id. (“A moderate frequency of near-misses encourages prolonged gambling . . . .
Problem gamblers often interpret near-misses as evidence that they are mastering the game
and that a win is on the way.” (quoting Dr. Clark)).
182. See id.
2019] REGULATING HABIT-FORMING TECHNOLOGY 157
how much “one more pull” or one more in-game purchase will cost.
183
Making financial cost an abstraction removes a significant cognitive speed
bump.
Loot boxes represent only one tool among a diverse kit of addictive design
strategies that might come under scrutiny. Users seem to be drawn in easily
by goal-setting, such as in the pursuit of “badges” representing the user’s
“achievements.”
184
Early goals are met quickly, and they gradually become
more demanding as the user progresses.
185
One particularly effective goal-
setting tactic is to encourage users to maintain a “streak” of daily use.
186
The
streak increases by one for each day that the user checks in and performs the
required action, and many apps bestow badges on users who maintain the
streak for some given amount of time.
187
Apps also drive engagement by preying on social obligation.
188
Facebook
users, to give the most familiar example, are summoned daily to wish their
many “friends” a happy birthday. On Instagram, some conscientious users
“Like” all of their friends’ posts and expect the same in return. The platform
guards this economy of mutual obligation zealously.
189
When Rameet
Chawla released Lovematically, “a platform that automatically likes every
183. Kyle Orland, ESRB Defends “Fun” Loot Boxes as It Starts Labeling All “In-Game
Purchases,” A
RS TECHNICA (Feb. 27, 2018, 11:01 AM), https://arstechnica.com/
gaming/2018/02/esrb-defends-fun-loot-boxes-as-it-starts-labeling-all-in-game-purchases/
[https://perma.cc/BT3B-GAVL] (“[Quake Champions] has two kinds of coins: ones that can
only be earned, and ones that can only be purchased with real money. A lot of loot box games
do the same confusing thing.”).
184. Snapchat, for example, gives out trophies such as “Panda Bear” and “Monkey
Covering Ears” for sending fifty snaps with the black-and-white filter and one snap without
sound, respectively. John-Michael Bond, How to Fill Your Snapchat Trophy Case Like a
Champ, D
AILY DOT (Mar. 12, 2017, 5:00 AM), https://www.dailydot.com/debug/snapchat-
trophies/ [https://perma.cc/D3WT-Y6AT].
185. See CodeFights Unveils Code Arcade to Make Honing Your Skills a Thrilling Game,
PR N
EWSWIRE (Sept. 21, 2016, 4:46 AM), https://www.prnewswire.com/news-releases/
codefights-unveils-code-arcade-to-make-honing-your-skills-a-thrilling-game-
300332143.html [https://perma.cc/JL95-JFDS] (“Arcade games are beloved and addictive
because players are prompted to complete tasks of increasing difficulty by mastering certain
skills.”).
186. See Sarah E. Needleman, How Mobile Games Aim to Keep You Coming Back, W
ALL
ST. J. (Jan. 8, 2017, 7:18 PM), https://www.wsj.com/articles/how-mobile-games-aim-to-keep-
you-coming-back-1483790404 [https://perma.cc/Z2GE-584S] (“This year, 18 out of the top
25 most downloaded games in the U.S. offer some form of daily-reward system . . . .”).
187. See Taylor Lorenz, Teens Explain the World of Snapchat’s Addictive Streaks Where
Friendships Live or Die, B
US. INSIDER (Apr. 14, 2017, 1:58 PM), https://
www.businessinsider.com/teens-explain-snapchat-streaks-why-theyre-so-addictive-and-
important-to-friendships-2017-4 [https://perma.cc/5YKD-A5UT] (“Snapchat rewards longer
streaks with special emojis, such as the ‘100’ emoji for streaks lasting 100 days, or a mountain
emoji for an extremely long streak.”).
188. See Needleman, supra note 186 (“A particularly effective tactic is team-based
objectives, since players tend to nudge one another to participate, said Harlan Crystal,
technology chief at Pocket Gems Inc. In its strategy game ‘War Dragons,’ for instance, the
more players who show up for a daily team battle, the better a team’s chances to win and for
members to earn free virtual currency.”).
189. See Brendan O’Connor, Lovematically: The Social Experiment Instagram Shut Down
After Two Hours, D
AILY DOT (Feb. 17, 2014, 4:22 PM), https://www.dailydot.com/
debug/lovematically-auto-like-instagram-shut-down/ [https://perma.cc/48ZJ-92MW].
158 FORDHAM LAW REVIEW [Vol. 88
single post that passes your Instagram feed,” Facebook—which owns
Instagram—stopped the threat within two hours and “[a]ll posts on
Facebook . . . that included links to Lovematically.com were deleted, and any
attempts to post these links are still met with an error message.”
190
Other platforms lure users in by “notifying” them of tangential events
involving people that they know.
191
This may suggest that either the friend
directed this news to the user or that the user is under some social obligation
to weigh in—or some combination of the two. But social media posts, unlike
email or direct messages, are generally not directed to anyone in particular.
192
And to the extent that there is a social obligation to communicate about some
event—an acquaintance’s birthday, for instance—the obligation may exist
largely or entirely because the platform has cooked it up as an engagement
driver.
193
Sometimes simple aesthetics can drive compulsive use. It is well
documented, for instance, that app developers display notifications in bright
red because the color inspires a sense of urgency.
194
Developers use
animations of fireworks, explosions, and bursts of light to amplify the sense
of reward when a user reaches a goal.
195
Many applications draw out loading
times artificially and accompany them with animations to create a sense of
anticipation when users pull down and refresh their screens.
196
Snapchat’s “streak” feature combines these various techniques into a
particularly potent cocktail of goal-setting, social obligation, randomized
rewards, and aesthetic enticement.
197
Users, typically teenagers, continue
their streak with various friends by sending them each at least one “snap” a
day.
198
Breaking the streak is typically received among these younger users
as a lack of commitment to the friendship.
199
190. Id.
191. See Louise Matsakis, Facebook Notification Spam Has Crossed the Line, W
IRED (Feb.
15, 2018, 2:16 PM), https://www.wired.com/story/facebook-notification-spam-two-factor/
[https://perma.cc/A9JQ-X6ZP] (“It might be somebody that you want to hear from, but you
don’t know, the only way to know is to check it.”).
192. See id. (“Now Facebook will nag you when an acquaintance comments on someone
else’s photo, or when a distant family member updates their status.”).
193. See Glenn Fleishman, How Facebook Devalued the Birthday, F
AST CO. (Apr. 6, 2018),
https://www.fastcompany.com/40550725/how-facebook-devalued-the-birthday
[https://perma.cc/H38B-76UA].
194. See John Herrman, How Tiny Red Dots Took Over Your Life, N.Y.
TIMES (Feb. 27,
2018), https://www.nytimes.com/2018/02/27/magazine/red-dots-badge-phones-
notification.html [https://perma.cc/Y4RY-4ZMR].
195. See Wiltshire, supra note 178 (displaying several of these animations).
196. See Kaveh Waddell, Why Some Apps Use Fake Progress Bars, A
TLANTIC (Feb. 21,
2017), https://www.theatlantic.com/technology/archive/2017/02/why-some-apps-use-fake-
progress-bars/517233/ [https://perma.cc/JFC5-BNWG]; Mark Wilson, The UX Secret That
Will Ruin Apps for You, F
AST CO. (July 6, 2016), https://www.fastcompany.com/3061519/the-
ux-secret-that-will-ruin-apps-for-you [https://perma.cc/8HYY-BJY4].
197. See supra notes 87–88 and accompanying text.
198. See supra note 187 and accompanying text.
199. See Lorenz, supra note 187.
2019] REGULATING HABIT-FORMING TECHNOLOGY 159
Today, loot boxes are the testing ground for regulating pernicious design
features.
200
Regulators could conceivably go much further in the future—
“streaks,” arbitrary notifications, and even certain aesthetic elements might
come under scrutiny.
Finally, regulators might step in to limit users’ access to certain tech
products altogether. It is easy to envision the FTC limiting minors’ access to
the kind of habit-forming tech products that are today marketed to children
without so much as a warning label.
201
It is also easy to imagine many states’
self-exclusion laws, through which problem gamblers can lock themselves
out of casinos, extended to certain apps that either resemble gambling or that
already meet the state law definition of gambling.
Lawmakers cutting adults off completely from any behaviorally addictive
tech product would seem out of the question today. Not even slot machines,
long considered the “crack cocaine” of gambling, receive that kind of
regulatory treatment.
202
Over the long run, however, it is possible that we
will see tech products dangerous enough to justify this extreme measure.
4. Counter-Addictive Design
A final approach would introduce countermeasures to mitigate the habit-
forming effects of persuasive design.
203
Some anti-addictive design might
resemble the features discussed above, but without the option to turn them
off: time-outs, for instance, after a long period of use. Another approach
would allow unlimited use but subject to some degree of transactional
friction. Facebook users, for instance, might be prompted to pay some small
amount of “postage” before putting up photos or commenting, or they might
just be prompted to confirm or review their comment before doing so. Or
instead of the overconsumption-encouraging “infinite scroll” included in
most apps today, apps might—like Google’s search results—show ten or so
results at a time and require the user to “flip the page” after reaching the
bottom.
200. As of June 15, 2019, loot boxes appear to be the only area where there is serious talk
of regulation.
201. The Entertainment Software Rating Board, a trade organization, recently adopted a
vague “in-game purchases” label that covers loot boxes among various other designs. See Nick
Statt, ESRB’s Loot Box Response Is New “In-Game Purchases” Label That Applies to Almost
Every Game, V
ERGE (Feb. 27, 2018, 1:05 PM), https://www.theverge.com/
2018/2/27/17058400/esrb-in-game-purchases-label-microtransactions-loot-boxes-regulation-
oversight [https://perma.cc/F2GP-R66G].
202. See supra note 123 and accompanying text. See generally A
M. GAMING ASS’N,
R
ESPONSIBLE GAMING REGULATIONS & STATUTES (2016), https://www.americangaming.org/
wp-content/uploads/2019/01/Responsible-Gambling-Regulations-WEB.pdf
[https://perma.cc/263V-UHDP].
203. See Arielle Pardes, Quality Time, Brought to You by Big Tech, W
IRED (Dec. 31, 2018,
7:00 AM), https://www.wired.com/story/how-big-tech-co-opted-time-well-spent/ [https://
perma.cc/9A2Y-N486] (“On both iOS and Android, it’s now easier to keep track of how many
hours you spend on certain apps, and even set app time limits. But neither company has done
much to address the persuasive design of those apps.”).
160 FORDHAM LAW REVIEW [Vol. 88
In some Asian countries, some kinds of anti-addiction design are
mandatory. In addition to funding rehabilitation programs for tech addiction,
the governments of South Korea, Vietnam, and Thailand have required that
game developers and administrators implement technical features to frustrate
minors’ attempts to overuse their products.
204
Under “Cinderella” or
“Shutdown” laws, game companies must remove gamers under the age of
eighteen from their servers between the hours of midnight and 6:00 a.m.
205
China goes even further, requiring developers to incorporate “fatigue”
mechanics into their games to make them less enticing after long periods of
consecutive use.
206
After three hours of use, the value of in-game items and
rewards—such as experience points acquired for killing an enemy or
ammunition picked up off the ground—must be cut down by half.
207
After
five hours of use, the player’s character begins to take damage minute by
minute.
208
As alien as these laws may seem, they are not impossible as a technological
matter. The aversion is cultural and subject to change.
B. Tools for Regulation
1. Existing Tools for Regulation
Some regulatory authority already exists to combat addictive tech without
new legislation. Gambling law, consumer protection law, and certain
common law torts all provide avenues to regulate at least some aspects of
addictive tech.
a. Gambling Law
Certain applications do more than just borrow from casinos’ approach to
design—instead, they involve either traditional gambling or something that
resembles it closely enough to qualify as gambling under the law. So it is
fitting that gambling commissions would be the first legal authorities in the
United States and Europe to regulate addictive tech design.
Gambling in most jurisdictions, both in the United States and abroad, is
defined by three elements: first, the gambler must “stake or risk something
of value”; second, “chance is a material factor”; finally, “successful play is
rewarded by something of value.”
209
These elements can be satisfied just as
204. See Orsolya Király et al., Policy Responses to Problematic Video Game Use: A
Systematic Review of Current Measures and Future Possibilities, 7 J.
BEHAV. ADDICTIONS
503, 506 (2018) (citing Leigh Alexander, Vietnamese Gov’t Puts Curfew on Online Gaming,
G
AMASUTRA (Feb. 22, 2011), https://www.gamasutra.com/view/news/33143/
Vietnamese_Govt_Puts_Curfew_On_Online_Gaming.php) [https://perma.cc/Y2Y8-RBTJ]).
205. Ji-Young, supra note 11; Sun, supra note 11. South Korea’s online gaming platforms
must screen users by requiring them to submit a national ID number. Le, supra note 11.
206. See Király et al., supra note 204, at 508.
207. Id.
208. Id.
209. 38 A
M. JUR. 2D Gambling § 2 (2010).
2019] REGULATING HABIT-FORMING TECHNOLOGY 161
well in an online setting as in a traditional offline setting, and many
jurisdictions already regulate online gambling.
210
Now, gambling laws are being tested in two new contexts. First, plaintiffs
have brought claims against developers of “social casino” apps—free-to-play
mobile apps that contain simulated slot machines, roulette, and other casino
games that dispense virtual currency rather than actual cash. Most of these
claims have failed,
211
but in Washington State they have been successful.
212
In Kater v. Churchill Downs Inc.,
213
the Ninth Circuit held that Big Fish
Casino, a social app, counted as an “illegal gambling game” under
Washington law.
214
Under a statutory remedy, members of the plaintiff class
recovered in-game payments they had made to the game’s distributor—
payments that for some individuals amounted to thousands of dollars.
215
Second, gambling commissions around the world have considered
regulating games that contain loot boxes. Gambling commissioners in the
Netherlands
216
and Belgium
217
have classified loot boxes as gambling under
existing regulations and threatened companies that continue to market them
to children with criminal fines.
218
China and South Korea, which already
regulate video gaming heavily, now require game developers to disclose
players’ chances of winning.
219
Commissioners in the United Kingdom and
France have declined to classify loot boxes as gambling on decidedly
210. See Benjamin Miller, The Regulation of Internet Gambling in the United States: It’s
Time for the Federal Government to Deal the Cards, 34 J. NAT’L ASS’N ADMIN. L. JUDICIARY
527, 546–56 (2014).
211. See John T. Holden, Trifling and Gambling with Virtual Money, 25 UCLA
ENT. L.
REV. 41, 81–89 (2018).
212. See, e.g., Wilson v. PTT, LLC, 351 F. Supp. 3d 1325, 1326 (W.D. Wash. 2018)
(denying a motion to dismiss for failure to state a claim); Wilson v. Playtika, Ltd., 349 F.
Supp. 3d 1028, 1045 (W.D. Wash. 2018) (denying a motion to dismiss for failure to state a
claim).
213. 886 F.3d 784 (9th Cir. 2018).
214. Id. at 788.
215. Id.
216. The Netherlands’ gaming authority studied ten loot boxes and determined that four
contravened the country’s gambling law because they offered in-game rewards that could be
sold in external markets. N
ETH. GAMING AUTH., STUDY INTO LOOT BOXES: A TREASURE OR A
BURDEN? 3 (2018), https://dutchgamesassociation.nl/news/loot-boxes-netherlands-gaming-
authoritys-findings/ [https://perma.cc/3EQ3-LCY3]. The regulator found that all ten loot
boxes had the potential to be addictive, and that their addictive potential ranged from
“moderate” (comparable to bingo) to “high” (comparable to blackjack). Id. at 9.
217. The Belgian Gaming Commission has classified loot boxes generally as gambling
without regard to whether their contents can be sold for cash. B
ELG. GAMING COMM’N,
RESEARCH REPORT ON LOOT BOXES 10 (2018), https://www.gamingcommission.be/
opencms/export/sites/default/jhksweb_nl/documents/onderzoeksrapport-loot-boxen-Engels-
publicatie.pdf [https://perma.cc/Z9JG-WTZM] (“[I]t is not important if a ‘skin’ in Overwatch,
FIFA 18 or C-S: GO is merely of aesthetic value. What is important is that players attach
value to it and that this value is also emphasised by the game developers themselves.”).
218. See T. J. Hafer, The Legal Status of Loot Boxes Around the World, and What’s Next
in the Debate, PC
GAMER (Oct. 26, 2018), https://www.pcgamer.com/the-legal-status-of-loot-
boxes-around-the-world-and-whats-next/ [https://perma.cc/2XLV-FEDB].
219. See id.
162 FORDHAM LAW REVIEW [Vol. 88
technical grounds while still expressing concern.
220
In 2018, U.S. state
legislatures took up bills requiring game developers to shield minors from
loot boxes and to disclose the odds to adults.
221
Washington State considered
a bill to require its gambling commission to investigate the practice.
222
Loot boxes and social casinos clearly satisfy the “chance” element of
gambling.
223
The other two elements present more difficulty.
224
Whether the gambler has provided consideration for the opportunity to
play is an open question. For purposes of contract law, users who open loot
boxes and play social casinos apps provide consideration even absent cash
payment. In Gottlieb v. Tropicana Hotel & Casino,
225
the court held that a
gambler provided consideration by voluntarily taking a free spin on a
casino’s “Million Dollar Wheel.”
226
The court rejected the casino’s
argument that the free spin was gratuitous, pointing out that the casino had
collected valuable marketing data when the plaintiff had swiped her casino-
issued ID card.
227
Gottlieb’s reasoning maps easily onto almost all free
phone apps: the user is always providing consumer data in exchange for
some in-app experience.
228
But in the same opinion, the Gottlieb court
interpreted “consideration” to have a more constrained meaning, at least
under New Jersey law.
229
The court deferred to a formal opinion from the
Attorney General of the State of New Jersey, concluding that the “legislative
intent was to exclude from the statutory elements comprising the gambling
offense the sort of personal inconvenience which will constitute
consideration sufficient to support a contract.”
230
The “thing of value” element has become the sticking point. Courts and
gambling commissions that have declined to identify social casinos and loot
boxes as gambling have generally done so by adopting an artificially
constrained definition of “value.”
231
First, they require an aftermarket to
220. See id. The question for the European regulators comes down to the cash value of in-
game rewards. The Netherlands and Belgium pointed to the existence of real-world
aftermarkets for virtual goods. Id. The United Kingdom and France declined to recognize
these aftermarkets on the ground that many games’ terms of service prohibit aftermarket
trading. Id.
221. Jason M. Bailey, A Video Game “Loot Box” Offers Coveted Rewards, but Is It
Gambling?, N.Y.
TIMES (Apr. 24, 2018), https://www.nytimes.com/2018/04/24/business/loot-
boxes-video-games.html [https://perma.cc/2NM6-FFJ4]; see also supra note 8 and
accompanying text.
222. See S. 6266, 65th Leg., Reg. Sess. (Wash. 2018).
223. See supra note 209 and accompanying text.
224. See supra note 209 and accompanying text.
225. 109 F. Supp. 2d 324 (E.D. Pa. 2000).
226. Id. at 327–30.
227. Id.
228. See Magali Eben, Market Definition and Free Online Services: The Prospect of
Personal Data as Price, 14 I/S:
J.L. & POL’Y FOR INFO. SOC’Y 221, 240 (2018).
229. Gottlieb, 109 F. Supp. 2d at 330.
230. Id. (citing Att’y Gen. of the State of N.J., Formal Opinion No. 6—1983 (June 1,
1983)).
231. See Mason v. Mach. Zone, Inc., 851 F.3d 315, 319 (4th Cir. 2017); Phillips v. Double
Down Interactive LLC, 173 F. Supp. 3d 731, 741 (N.D. Ill. 2016); Soto v. Sky Union, LLC,
159 F. Supp. 3d 871, 87982 (N.D. Ill. 2016); Andy Chalk, French Gambling Regulator
2019] REGULATING HABIT-FORMING TECHNOLOGY 163
exist for the in-game items.
232
Second, the items must be transferable
between accounts; the fact that players can sell their account passwords does
not qualify.
233
Finally, if the game developer’s terms of service forbid users
to sell their virtual winnings for cash, then the winnings are not considered
valuable.
234
This final requirement makes no sense. The law does not normally hold
that goods lose their value if the transaction is illegitimate. Think of bribery,
for example. Bribery, too, requires that the briber offer a “thing of value” to
the bribee. An offer of unlawful narcotics would presumably qualify despite
the goods’ legal illegitimacy.
235
Why, then, should a gambling transaction
be considered valueless because it violates another contract’s terms—an
adhesion contract, no less? It appears that many gambling commissions are
simply hesitant to move into unfamiliar terrain and that this excessively
formal approach to the “thing of value” element protects them from doing so.
The effect of this final requirement is to allow video game and social casino
developers to effectively opt out of gambling regulations through contractual
boilerplate.
At some level, the whole conversation around aftermarkets comes across
as a distraction. By extending or enhancing play, virtual items obviously
deliver value to players who want them.
236
This value is what gamers pay
for when they drop quarters into arcade machines, or when they purchase
“skins” for their avatars in online social games. Unless we are to understand
these transactions as something other than contractual interactions, the
intangible or make-believe nature of the items, contrary to what at least one
court has said,
237
should make no difference. There is no substantial
difference between the purchase of an in-game item and a license to access a
non-resellable e-book for a Kindle reader; both involve value exchanges.
238
Determines That Loot Boxes Are Not Legally Gambling, PC GAMER (July 4, 2018),
https://www.pcgamer.com/french-gambling-regulator-determines-that-loot-boxes-are-not-
legally-gambling/ [https://perma.cc/KN5N-A2JC]; Loot Boxes Within Video Games,
G
AMBLING COMM’N (Nov. 24, 2017), http://www.gamblingcommission.gov.uk/news-action-
and-statistics/news/2017/Loot-boxes-within-video-games.aspx [https://perma.cc/T5DH-
7N4U].
232. Soto, 159 F. Supp. 3d at 879–80; Mason v. Mach. Zone, Inc., 140 F. Supp. 3d 457,
469 (D. Md. 2015), aff’d, 851 F.3d 315 (4th Cir. 2017); Loot Boxes Within Video Games,
supra note 231.
233. Soto, 159 F. Supp. 3d at 879–80; Mason, 140 F. Supp. 3d at 468.
234. See Mason, 851 F.3d at 320 n.3; Phillips, 173 F. Supp. 3d at 735; see also Holden,
supra note 211, at 95 (“The reliance on terms of service agreements that feature certain
components including: non-redeemability of in-game items, non-transferability of coins to
third-parties, and the revocability of license to access the account have formed the basis for
the avoidance of gambling laws by social casino operators.”).
235. See, e.g., United States v. Walker, 348 F. App’x 910, 911 (5th Cir. 2009) (affirming
defendant’s criminal conviction where he accepted OxyContin in lieu of cash).
236. See supra note 175 and accompanying text.
237. Mason, 140 F. Supp. 3d at 469 (“The laws of California and Maryland do not trifle
with play money, and so Plaintiff’s Complaint must be dismissed.”).
238. See Sarah Reis, Note, Toward a “Digital Transfer Doctrine”?: The First Sale
Doctrine in the Digital Era, 109 N
W. U. L. REV. 173, 180–83 (2014) (discussing the limitations
on Kindle e-book ownership).
164 FORDHAM LAW REVIEW [Vol. 88
Jurisdictions may differ in their approaches to these questions, to be sure.
Some jurisdictions’ gambling laws may be more accommodating to
expansive rather than restrictive definitions of “value.”
239
But absent some
clear intent to exclude nonmonetary value, a hornbook definition of gambling
would cover social casinos and loot boxes alike, and perhaps other
monetization techniques to come.
Plausible interpretations exist, then, that would give gambling
commissioners jurisdiction over essentially any online game of chance—
including games of chance where no actual cash is exchanged. That is about
as far as an ambitious gambling commissioner could go without
overreaching. But within reason, gambling commissions are already well
positioned to regulate some of the most habit-forming monetization
mechanics in gaming today.
b. Consumer Protection
The FTC has provided the country’s closest approximation of a common
law of privacy.
240
In particular, the agency has used its power under the FTC
Act to police “unfair and deceptive practices” to pursue tech firms that violate
their own advertised privacy policies.
241
Absent a specific mandate to deal
with habit-forming design techniques, the FTC may be able to adapt its
privacy approach to the problem.
Generally, the FTC has focused on deceptive practices and used its broader
power over unfair practices sparingly.
242
The FTC would likely bring that
same approach to the problem of habit-forming tech design. An unfair
practice must satisfy three criteria that probably would be difficult to fulfill:
first, the practice must cause substantial harm to consumers; second, the harm
must not be reasonably avoidable by the consumer; third, the injury caused
by the practice must not be outweighed by any countervailing benefit to
competition or consumers.
243
239. One classic question involves whether free replays on pinball and video arcade
machines constitute “things of value.” See S. R. Shapiro, Annotation, Coin-Operated Pinball
Machine or Similar Device, Played for Amusement Only for Confining Reward to Privilege of
Free Replays, as Prohibited or Permitted by Antigambling Laws, 89 A.L.R.2d 815 § 5[a]
(2019) (“The most controversial questions have been whether free replays are ‘property’ or
‘things of value.’”).
240. See Daniel J. Solove & Woodrow Hartzog, The FTC and the New Common Law of
Privacy, 114 C
OLUM. L. REV. 583, 587 (2014) (“It is fair to say that today FTC privacy
jurisprudence is the broadest and most influential regulating force on information privacy in
the United States—more so than nearly any privacy statute or common law tort. The statutory
law regulating privacy is diffuse and discordant, and common law torts fail to regulate the
majority of activities concerning privacy.”).
241. Id. at 604 (describing this power as the FTC’s “lynchpin function”).
242. See Zahr K. Said, Mandated Disclosure in Literary Hybrid Speech, 88 W
ASH. L. REV.
419, 446 (2013) (“Historically, the FTC has very rarely exercised its authority to pursue those
engaged in unfair methods of competition and instead has focused almost exclusively on
deceptive actions, statements, or practices.”).
243. 15 U.S.C. § 45(n) (2012).
2019] REGULATING HABIT-FORMING TECHNOLOGY 165
The first and third elements of unfairness pose problems for any near-term
attempt to regulate habit-forming design. The unformed state of the research
on behavioral addiction may make a determination of “substantial harm”
appear too conjectural.
244
More significantly, the close tie between addictive
design and app monetization would provide an avenue for tech companies to
argue that their techniques for “driving engagement” are what enable them
to offer software either cheaply or for free.
245
This calculus could change over the long term, however, as market
conditions change. Causal theories linking design and behavioral addiction
may gain wider acceptance. Better data analytics capabilities and a more
absorbing user experience may come to intensify the habit-forming effect.
Certain monetization strategies may come to look more like simple rent-
seeking with little to no consumer benefit at the margin. And at least where
major industry players like Google and Facebook are concerned, consumer-
risk exposure may no longer be reasonably avoidable. In that environment,
the FTC could engage in enforcement actions or rulemaking directed at habit-
forming technology as an unfair practice. One would hope that by this point
Congress would have provided the FTC or some other authority with a more
specific mandate to combat the problem.
In the near term, deceptiveness offers a more promising ground for
regulation—most likely regulation of apps marketed to children. The agency
defines deception as a “representation, omission or practice that is likely to
mislead the consumer acting reasonably in the circumstances, to the
consumer’s detriment.”
246
When it has reached settlements with tech
companies for deceptive practices, the Commission has preferred for obvious
reasons to target firms that breach express promises.
247
But in some
deception cases the Commission has also settled with developers who
breached reasonable consumer expectations rather than an express
representation.
248
Habit-forming children’s apps offer the clearest target for rulemaking or
enforcement under the Commission’s deceptive practices jurisdiction. In
Google’s Play Store and Apple’s iTunes Store, children’s apps are generally
found in the “Family” section.
249
But this reassuring label often conflicts
with the developer’s aggressive monetization model. In some cases, children
have been lured into apps that have parted them from their parents’ money at
244. See generally Letter from Francis S. Collins, Dir., Dep’t of Health & Human Servs.,
to Michael F. Bennet, U.S. Senator (June 5, 2018), https://www.scribd.com/document/
384767297/NIH-Response-to-Bennet-Letter-on-Tech-Addiction [https://perma.cc/PW3T-
WAYR] (describing the state of the research on behavioral addiction).
245. See supra Part I.
246. Cliffdale Assocs., Inc., 103 F.T.C. 110, 176 (1984).
247. Solove & Hartzog, supra note 240, at 626–34.
248. See id. at 666–72.
249. Find Family-Friendly Content in Google Play, G
OOGLE PLAY HELP,
https://support.google.com/googleplay/answer/6209531 [https://perma.cc/98H9-8Y4T] (last
visited Aug. 22, 2019); App Store > Games > Family,
ITUNES PREVIEW,
https://itunes.apple.com/us/genre/ios-games-family/id7009?mt=8 [https://perma.cc/SZ4P-
BKS3] (last visited Aug. 22, 2019).
166 FORDHAM LAW REVIEW [Vol. 88
casino-like rates of thousands of dollars per hour.
250
Even assuming that the
parents avoid these kinds of charges, it stands to reason that few parents
would expect such a manipulative presence from a game that sits in the
“Family” section of the Google Play Store.
251
Part of the appeal of the “deceptive practices” approach to enforcement is
that it would not require the FTC to take any position on the ultimate
harmfulness or efficacy of habit-forming design. All that matters is whether
certain aggressive monetization techniques conflict with the parent’s
reasonable expectation.
252
As in the FTC’s recent settlement with a firm that
misrepresented its baby mattresses as “organic,” the uncertain state of the
underlying science is simply irrelevant.
253
In the course of these enforcement efforts, the FTC can begin to develop a
“common law” of habit-forming tech in the same way that it has developed
a “common law” of online privacy.
254
Companies marketing children’s apps
might be put on guard against incorporating certain suspect features in their
products—features that may range from loot boxes to dual currency systems
to overbearing notifications.
255
In the process, regulators could catalyze the
creation of a set of industry norms that might later be incorporated into
rulemaking.
c. Common Law Torts
Tort law, in principle, might provide a means for some plaintiffs to receive
compensation for injuries that flow from compulsive tech use. But the case
law on behavioral addiction is not particularly inviting.
First, several lawsuits have alleged that casinos and video slot machine
manufacturers owe problem gamblers a duty of care; none have
250. The FTC has pursued Apple and Amazon over billing practices that allowed children
to rack up thousands of dollars in microtransactions without their parents’ consent. In Apple’s
case, the company’s billing software discreetly left users’ credit card authorizations in effect
for fifteen minutes after each payment was made. Parents authorized app purchases for their
children, and the children went on fifteen minute in-app shopping sprees without their parents’
knowledge. The FTC pursued this unannounced extension of payment authorization under its
unfairness authority. FTC v. Amazon.com, Inc., No. C14-1038-JCC, 2016 U.S. Dist. LEXIS
55569 (W.D. Wash. Apr. 26, 2016); Apple Inc., FTC File No. 112-3108, 2014 WL 1330287
(F.T.C. Mar. 25, 2014).
251. See Erik Allison, The High Cost of Free-to-Play Games: Consumer Protection in the
New Digital Playground, 70 SMU
L. REV. 449, 470 (2017) (“While a parent should be able to
tell their child ‘no,’ it is a deceptive practice on the part of the game for a game to entice
children into making all the steps for a purchase except the authorization.”).
252. “[T]he Commission will find an act or practice deceptive if, first, there is a
representation, omission, or practice that, second, is likely to mislead consumers acting
reasonably under the circumstances, and third, the representation, omission, or practice is
material.” Cliffdale Assocs., Inc., 103 F.T.C. 110, 164 (1984).
253. See Press Release, Fed. Trade Comm’n, Illinois Firm Barred from Making Misleading
Baby Mattress Claims (Sept. 28, 2017), https://www.ftc.gov/news-events/press-releases/
2017/09/illinois-firm-barred-making-misleading-baby-mattress-claims
[https://perma.cc/9PS9-T4Q4].
254. See generally Solove & Hartzog, supra note 240.
255. See supra Part II.A.
2019] REGULATING HABIT-FORMING TECHNOLOGY 167
succeeded.
256
In Stevens v. MTR Gaming Group, Inc.,
257
the widow of a
suicidal problem gambler sued a slot machine manufacturer on strict product
liability and negligent design and failure to warn theories.
258
The court held
that the state’s pervasive regulation of the gambling industry
259
precluded
any inference that the legislature would have meant to leave casinos or game
manufacturers stuck with a common law duty of care.
260
Courts in other
states have reached the same result through similar reasoning.
261
Unlike their counterparts in the gambling industry, the tech companies
who might sit as defendants in a tech-addiction tort case do not belong to
heavily regulated industries. In principle, that should make the early
gambling cases easy to distinguish; in tech, there is no regulatory apparatus
to displace the ordinary tort law duty of care.
The First Amendment, however, may still prevent plaintiffs in tech-
addiction tort cases from receiving compensation. In Watters v. TSR, Inc.,
262
the plaintiff attempted to link the publishers of Dungeons & Dragons, a
tabletop role-playing game, to the suicide of her adult son.
263
The Sixth
Circuit granted summary judgment to the publisher, reasoning that the
plaintiff had failed to establish fault or causation.
264
The court also held that
strict product liability was out of the question given the case’s First
Amendment salience.
265
The court noted that courts should “avoid applying
the common law in a way that would bring the constitutional problems to the
fore.”
266
Beyond Watters, the record for gaming addicts—or anything else that
might offer a clear precedent for product liability on account of tech
256. David M. Ranscht, Problem Gambling Is Funny, 6 UNLV GAMING L.J. 59, 74 (2015)
(“Several lawsuits have been initiated, and have been almost uniformly unsuccessful.”).
257. 788 S.E.2d 59 (W. Va. 2016).
258. See id. at 61. The plaintiff alleged that the machines “employ[ed] algorithms and other
features that deceptively cause gamblers to play longer, more quickly, and more intensely.
The machines are allegedly designed to cause physiological change in brain functioning,
which promotes the loss of willpower and curtails the capacity to make rational decisions . . . .
[B]y their construction and programming, the machines erode the players’ ability to walk away
before they have exhausted their available funds.” Id. The plaintiff also sued the casino,
alleging that it had “facilitate[d] the compulsive behavior engendered by the machines by
targeting affected patrons with marketing ploys such as offering complementary [sic] food
and lodging, and by tendering lines of credit on terms that would not otherwise be bargained
for.” Id.
259. In particular, the state had set up an administrative approval process for gambling
devices, a “self-exclusion” program whereby problem gamblers could bar themselves from
casinos and established a statewide “Compulsive Gambling Treatment Fund.” Id. at 65. The
court inferred that the legislature must have intended these administrative remedies to exclude
any tort law duty of care. See id. at 67.
260. See id. at 63.
261. See, e.g., Taveras v. Resorts Int’l Hotel, Inc., No. 07–4555, 2008 WL 4372791 (D.N.J.
Sept. 19, 2008); Caesars Riverboat Casino, LLC v. Kephart, 934 N.E.2d 1120 (Ind. 2010).
262. 904 F.2d 378 (6th Cir. 1990).
263. Id. at 379.
264. See id. at 380–84.
265. See id. at 382.
266. Id. at 383.
168 FORDHAM LAW REVIEW [Vol. 88
addiction—includes only one other case.
267
In Smallwood v. NCSoft
Corp.,
268
the U.S. District Court for the District of Hawaii allowed a similar
set of claims to survive a motion to dismiss, but the suit’s pro se plaintiff did
not proceed further.
269
Unfortunately, the underlying complaint is so crudely
drafted that it is somewhat hard to know what to take from the court’s
reasoning.
Hardly a promising record, in short. But these cases are not totally
plaintiff-preclusive either. Recall that tobacco plaintiffs had logged a
decades-long string of defeats before the first plaintiff award came in
Cipollone v. Liggett Group.
270
By the late eighties, however, an improved
medical understanding of nicotine’s addictive qualities plus an emerging
public consciousness of nicotine’s risks and the industry’s malfeasance
helped Cipollone’s plaintiffs to succeed.
271
The history of product litigation
over behavioral addictions is short, and it is too early to predict that similar
developments will not eventually come through.
2. New Tools for Regulation
The existing regulatory capacity to police habit-forming design is limited.
Social casinos, loot boxes, and similar devices that have not yet emerged can
at least in principle be regulated in the same way that gambling is regulated.
Depending on the jurisdiction’s approach to gambling, these regulations
could include licensing procedures, odds disclosure requirements, self-
exclusion programs where problem gamblers can lock themselves out of
regulated gambling establishments, and so on. Consumer protection
agencies, meanwhile, are well-positioned to police certain children’s apps
that are monetized through aggressive habit-forming design.
272
But for better
or worse, no governmental entity today appears to have authority to police
habit-forming design beyond the gaming sphere. That authority will have to
come from a new place.
267. I exclude various “violent video game” cases in which a gamer—often described as
“addicted”—allegedly picked up violent tendencies by imitating in-game events or characters.
These theories have consistently failed. See, e.g., Wilson v. Midway Games, Inc., 198 F. Supp.
2d 167, 171 (D. Conn. 2002); Sanders v. Acclaim Entm’t, Inc., 188 F. Supp. 2d 1264, 1268
(D. Colo. 2002); cf. Brown v. Entm’t Merchs. Ass’n, 564 U.S. 786, 800–01 (2011) (casting
doubt on the quality of the social science linking violent games with violent behavior).
268. 730 F. Supp. 2d 1213 (D. Haw. 2010).
269. See generally id.
270. 593 F. Supp. 1146 (D.N.J. 1984); see Peter F. Riley, Note, The Product Liability of
the Tobacco Industry: Has Cipollone v. Liggett Group Finally Pierced the Cigarette
Manufacturers’ Aura of Invincibility?, 30 B.C.
L. REV. 1103, 1117–41 (1989).
271. See Riley, supra note 270, at 1128.
272. Such a role would outgrow naturally from the FTC’s responsibility for enforcing the
Children’s Online Privacy Protection Act, which regulates commercial websites’ use of
personally-identifying information on children under the age of eighteen. See 15 U.S.C.
§ 6505(a)–(e) (2012); see also 16 C.F.R. § 312.9 (2019).
2019] REGULATING HABIT-FORMING TECHNOLOGY 169
a. Piecemeal Legislation
Over time, Congress may adopt legislation that attacks individually
offensive design techniques piecemeal. It is not too hard, for example, to
imagine a statutory scheme designed to curb intrusive and misleading push
notifications within phone apps—sort of a successor to the Controlling the
Assault of Non-Solicited Pornography and Marketing Act’s regulation of
spam email.
273
Another law might require mobile devices to come
prepackaged with software to notify users when their app usage has exceeded
reasonable limits—a utility similar to iOS’s Screen Time app but turned on
by default.
b. A General Mandate
Or Congress may grant a general mandate to an agency—most likely the
FTC but perhaps even the Consumer Product Safety Commission—to
regulate predatory design practices that tend to promote behavioral addiction.
Vague as that may sound, addiction itself is a rather elastic term even among
clinicians; it is no vaguer than similar terms, such as “unfair,” that are often
found in enabling legislation.
274
The Controlled Substances Act (CSA), in
fact, contains language to the same effect: the potential for “psychological
or physical dependence” is a criterion left to the discretion of administrative
officials.
275
Efficacy, of course, is another question. New applications and platforms
arise constantly, and those that already exist are always updating their design.
New addictive designs are always emerging, and developers can vary them
quickly in response to enforcement. But this is not a totally unique or
unprecedented concern. The problem exists in the narcotics sector as well,
where drug labs constantly tweak chemical compounds to avoid the
application of the CSA’s “Schedule I.”
276
Regulators have struggled to meet
these innovations.
277
The gambling industry, too, has its own colorful history
of cat and mouse.
278
273. Controlling the Assault of Non-Solicited Pornography and Marketing Act of 2003, 15
U.S.C. §§ 7701–7713 (2012).
274. The Federal Trade Commission Act prohibits “unfair” practices. 15 U.S.C. § 45(a)(1)
(2012).
275. See 21 U.S.C. § 812(b)(2)(C) (2012). The Attorney General, in consultation with the
Secretary of the Department of Health and Human Services, may add new substances to the
CSA’s five-category schedule of regulated and prohibited drugs. See id. The Consumer
Product Safety Act also uses vague language about “risk of injury” to instruct administrative
rulemakers. See 15 U.S.C. § 2052 (2012) (“The term ‘risk of injury’ means a risk of death,
personal injury, or serious or frequent illness.”). An addendum referring to behavioral
dependence—assuming that the language about “illness” does not already cover it—would
hardly be out of place.
276. See Timothy P. Stackhouse, Note, Regulators in Wackyland: Capturing the Last of
the Designer Drugs, 54 A
RIZ. L. REV. 1105, 1111–18 (2012).
277. See id. at 1111.
278. One machine popular in the late 1990s and early 2000s, for example, sold borderline-
worthless one-minute phone cards for a quarter a piece. See generally Hest Techs., Inc. v.
State ex rel. Perdue, 749 S.E.2d 429 (N.C. 2012). Each card entered the purchaser in a
170 FORDHAM LAW REVIEW [Vol. 88
c. Dry-Up-the-Market Approaches
Finally, legislatures could address these problems from the outside, and
not by restricting the contents of habit-forming tech products in themselves,
but by blunting or destroying the incentive to make them in the first place.
For products with advertising-based monetization models, the remedy is
clear: either restrict targeted advertising or make it less lucrative through
taxation. Within the short run, privacy regulation along the lines of the
European Union’s General Data Protection Regulation
279
may provide some
help along these lines. Over the long run, and for reasons that transcend the
addiction issue, governments should move toward a more general ban on the
behavioral advertising industry—a radical move that would clearly violate
the First Amendment and that would create a need for wider structural
regulation to fund online content.
280
Such a program is beyond the scope of
this Article, but it would surely blunt the incentive to drive engagement.
For products that are monetized through in-app purchases, there may be
less dramatic solutions. The law might, for example, set up caps on the total
amount merchants could collect from a single user within a year. Today,
many microtransaction-based games are designed to target “whales”: the
0.15 percent of users who account for half of the revenue in the average
microtransaction-financed game.
281
These “whales,” spending thousands of
dollars on average on a single game,
282
would appear to be problem users.
283
As long as they make up game developers’ biggest profit center,
284
though,
developers will likely continue to design casino-like products that are
optimized for problem use.
A new statute might allow big spenders to retrieve every dollar spent in
excess of some reasonable limit. The limit could equal a dollar figure fixed
by regulation—say, $100 per app—or it could be calculated on some floating
sweepstakes with a risk-reward spread similar to a slot machine. See generally id. Purchasers
could collect their winnings from a cashier who would scan a QR code printed on the back of
the card. See id. at 443. More recently, gambling operators have opened internet cafes where
gamblers purchase minute after minute of internet access alongside their chance to win. See
generally United States v. Davis, 690 F.3d 330 (5th Cir. 2012). Courts have never had much
trouble peering through the veil. See Hest Techs., 749 S.E.2d at 430–31 (“[N]o sooner is a
lottery defined, and the definition applied to a given state of facts, than ingenuity is at work to
evolve some scheme of evasion . . . . But, in this way, it is not possible to escape the law’s
condemnation . . . .” (quoting State v. Lipkin, 84 S.E. 340, 343 (N.C. 1915)).
279. Regulation 2016/679 of the European Parliament and of the Council of 27 April 2016
on the Protection of Natural Persons with Regard to the Processing of Personal Data and on
the Free Movement of Such Data, and Repealing Directive 95/46/EC (General Data Protection
Regulation), 2016 O.J. (L 119) 1 (EU).
280. See generally Kyle Langvardt, A New Deal for the Online Public Sphere, 26 G
EO.
MASON L. REV. (forthcoming 2019).
281. See supra note 60 and accompanying text.
282. This is a rough extrapolation from the data indicating that 50 percent of mobile gaming
revenue comes from 0.15 percent of players. See supra note 58 and accompanying text. If
that is the case, then the top 0.15 percent of players will pay at least $1000 so long as revenue
is at least $2000 and the game raises $3 per player.
283. See supra Part I.C.1.
284. See supra notes 60–61 and accompanying text.
2019] REGULATING HABIT-FORMING TECHNOLOGY 171
basis. One approach might say that for a given app, the limit should equal
the total in-app expenditure level by users at the ninety-ninth percentile.
Merchants and platforms such as Google Play and the iTunes Store could
then be made jointly liable to provide quick refunds to any user who exceeds
the limit. Ideally, this policy would enlist the major platforms as enforcers
against predatory design practices.
III.
FIRST AMENDMENT CHALLENGES
Even modest efforts to regulate addictive design will likely be challenged
as infringements on free expression. How the law should handle these
challenges, however, is unclear. Two first-order questions arise.
The first is a question of what Frederick Schauer has called “coverage.”
285
The coverage inquiry is distinct from the question of whether a type of speech
will ultimately be protected from regulation or not.
286
Instead it asks, in a
given case, whether the First Amendment has any salience at all.
287
In cases
involving speech in furtherance of a criminal conspiracy, or insider trading,
or obligations under a written contract, defendants do not even suggest that
their use of language would entitle them to First Amendment protections.
288
Gambling operations are treated as similarly “uncovered”; though litigants
have occasionally raised First Amendment challenges to gambling laws, their
arguments have never been taken seriously.
289
But the range of coverage is
always expanding,
290
and the Supreme Court has articulated no outer limit
that might help to determine whether various addictive design techniques
should be considered First Amendment subject matter at all.
291
The case law
is marked in places by language suggesting essentially total coverage for
software,
292
and tech industry lawyers have encouraged this maximal
approach.
293
But the case law is much likelier to reveal an approach that does
285. See generally Frederick Schauer, The Boundaries of the First Amendment: A
Preliminary Exploration of Constitutional Salience, 117 HARV. L. REV. 1765 (2004).
286. See id. at 1769–74.
287. See id.
288. See id. at 1783–84.
289. See, e.g., United States v. Edge Broad. Co., 509 U.S. 418, 426 (1993)
(“[G]ambling . . . implicates no constitutionally protected right; rather, it falls into a category
of ‘vice’ activity that could be, and frequently has been, banned altogether.”); Interactive
Media Entm’t & Gaming Ass’n Inc. v. Att’y Gen., 580 F.3d 113, 118 n.8 (3d Cir. 2009)
(noting that gambling “lacks any ‘communicative element’ sufficient to bring it within the
ambit of the First Amendment”) (quoting United States v. O’Brien, 391 U.S. 367, 376 (1968));
Allied Veterans of World, Inc.: Affiliate 67 v. Seminole County, 783 F. Supp. 2d 1197 (M.D.
Fla. 2011), aff’d, 468 F. App’x 922 (11th Cir. 2012) (finding that a county ordinance banning
“simulated gambling devices” regulated conduct rather than speech).
290. See generally Schauer, supra note 285.
291. See generally id.
292. Universal City Studios, Inc. v. Corley, 273 F.3d 429, 447 (2d Cir. 2001).
293. See, e.g., Matthew Panzarino, Apple Files Motion to Vacate the Court Order to Force
It to Unlock iPhone, Citing Constitutional Free Speech Rights, T
ECHCRUNCH (Feb. 25, 2016),
https://techcrunch.com/2016/02/25/apple-files-motion-to-dismiss-the-court-order-to-force-it-
to-unlock-iphone-citing-free-speech-rights/[https://perma.cc/NU74-2S86].
172 FORDHAM LAW REVIEW [Vol. 88
not regard all software design as equally expressive for purposes of the First
Amendment.
A second question has to do with regulatory means and ends and their
adequacy relative to whatever standard of heightened scrutiny might be in
play. The most obvious governmental purpose for regulating addictive
design will have to do with public health, and in particular the public health
of children who might be exposed to these products. But there are deeper
regulatory purposes at play as well—most prominently, a concern for the
health of the same public sphere that the Free Speech Clause exists to protect.
A. Coverage
Addictive design cuts across a variety of speech contexts that may be
protected by the First Amendment. At a minimum, the Ninth Circuit’s
decision in Kater has introduced gambling regulation to the world of
consumer video games, a medium that has been recognized to contain strong
expressive salience for First Amendment purposes.
294
Further regulation of
addictive tech, even if it only affects products marketed to children, could cut
closer to core constitutional territory. In particular, Justice Anthony Kennedy
recently recognized social media platforms as “the most important places . . .
for the exchange of views” in the twenty-first century.
295
Proponents of any
consumer measure to deal with addictive design—beginning with the loot
box bills now being considered in several states—should expect a First
Amendment challenge.
The outlines of such a challenge are obvious enough: video game
companies speak when they design video games. When the government
restricts loot boxes, it burdens games that contain one particular kind of
content. Facebook speaks by maintaining its service. When the government
requires Facebook to tell its users how long they have spent on the device, it
compels Facebook’s authors to speak a message that is not their own. And
so on; these arguments lend themselves to efficient mass production. After
all, if UX design is just another expression for software’s content, then
virtually any law that regulates software should count as a content-
discriminatory law of the sort that triggers strict scrutiny under today’s
interpretation of the First Amendment.
296
These arguments’ viability, however, depends heavily on the level of
generality that is chosen to define the problem. First, a software product
might be viewed holistically, as a single, unified article of speech. On this
generalistic account, laws that reach in and regulate a loot box, for example,
have the effect of burdening one aspect of the speaker’s “message.” This
way of thinking has the advantage of simplicity, and it fits well with the
current Supreme Court’s austere First Amendment style. But carried to its
logical conclusion, this approach would also require strict constitutional
294. See Kater v. Churchill Downs Inc., 886 F.3d 784, 785 (9th Cir. 2018).
295. Packingham v. North Carolina, 137 S. Ct. 1730, 1734 (2017).
296. Reed v. Town of Gilbert, 135 S. Ct. 2218, 2229–30 (2015) (finding that an ordinance
regulating “temporary directional signs” was content discriminatory and unconstitutional).
2019] REGULATING HABIT-FORMING TECHNOLOGY 173
scrutiny in any case involving the regulation of software. This holistic view
is not likely to survive sustained contact with twenty-first-century regulatory
problems involving 3D-printable consumer products, cryptocurrency
speculation, and so on.
297
Second, a piece of software might be viewed specifically, as a consumer
product that integrates speech components with nonspeech components.
298
This approach would acknowledge the expressive components in various
kinds of applications—literary or character elements in video games, shared
media content in social media, etc.—without assuming that other purely
mechanical components such as login credentialing or memory caching are
expressive by association. The question then becomes whether regulated
items such as loot boxes, or infinite scrolls, or bright red notification badges
constitute speech for First Amendment purposes on their own merits.
As the following section demonstrates, courts have not yet chosen between
the holistic and the particular approach.
1. The Holistic Approach
If laws regulating addiction mechanisms in games or social media were
challenged on First Amendment grounds, the challengers would likely cite
certain relatively recent Supreme Court opinions as broad authority for the
proposition that the medium being regulated is all but inviolable.
Challengers might cite Brown v. Entertainment Merchants Ass’n
299
for the
proposition that video games are broadly protected.
300
Challengers might
cite Packingham v. North Carolina
301
for the proposition that social media
and its platforms are essentially inviolable.
302
And challengers might cite
Sorrell v. IMS Health Inc.
303
in support of a “rule that information is
speech”—a rule that could be interpreted to cover just about everything that
happens on computers or the internet.
304
But a fair look at the case law reveals that the courts have never really said
these things, and indeed they have never extended any special constitutional
protection to tech products or the internet beyond what the First Amendment
would ordinarily furnish in a low-tech setting.
297. See Kyle Langvardt, The Doctrinal Toll of “Information as Speech,” 47 LOY. U. CHI.
L.J. 761, 769–75 (2016).
298. In the world of tangible objects, there is no such thing as pure speech; even a book
contains paper stock that is intuitively distinct from any expressive content. The perception
that software is different because of its intangible or conceptual nature is ultimately delusive.
Credit, debt, governments, corporations, family relationships, and so on are all intangible and
conceptual, and all intersect in various ways with activities that may be expressive for First
Amendment purposes. But that is no reason to say that these phenomena are somehow shot
through in every dimension with First Amendment significance.
299. 564 U.S. 786 (2011).
300. See id. at 790.
301. 137 S. Ct. 1730 (2017).
302. See id. at 1734.
303. 564 U.S. 552 (2011).
304. See id. at 571 (“The State asks for an exception to the rule that information is speech,
but there is no need to consider that request in this case.”).
174 FORDHAM LAW REVIEW [Vol. 88
a. Games
In Brown, the Supreme Court invalidated a California law that prohibited
the sale or rental of violent video games to minors and required them to be
labelled “18.”
305
Justice Antonin Scalia, writing for a seven-justice majority,
dispelled any notion that video games might deserve subordinate First
Amendment status:
Like the protected books, plays, and movies that preceded them, video
games communicate ideas—and even social messages—through many
familiar literary devices (such as characters, dialogue, plot, and music) and
through features distinctive to the medium (such as the player’s interaction
with the virtual world). That suffices to confer First Amendment
protection.
306
None of the justices seemed to disagree on this initial point.
307
From here, Justice Scalia proceeded with a conventional content-
discrimination analysis.
308
By singling out video games based on their
content, the California statute opened itself up to strict scrutiny.
309
Supposing for the sake of argument that children’s well-being, broadly
defined, constituted a compelling state interest, Justice Scalia nevertheless
found the means-end fit lacking.
310
The available social science, he wrote,
had failed to establish a strong causal connection between the video games
and any actual violence.
311
Brown never says that all video games are speech, and the Court had no
occasion to do so. Instead the case asked only whether portrayals of violence
that occur in video games are speech—for those are all that the statute
regulated. So while challengers to a loot box law would inevitably cite
305. Brown, 564 U.S. at 789.
306. Id. at 790.
307. Justice Thomas’s dissent proposed, somewhat tangentially to the other justices’
discussion, that “speech to minor children bypassing their parents” is an unprotected category
for originalist reasons. Id. at 822 (Thomas, J., dissenting). It is unclear where Justice Thomas
would come down on the question of whether video games might be speech at all. He writes:
“The Court holds that video games are speech for purposes of the First Amendment and finds
the statute facially unconstitutional. I disagree.” Id. at 838–39. But which of these two points
does Justice Thomas disagree with? That video games are speech, that the statute is facially
unconstitutional, or both? The bulk of Justice Thomas’s discussion about the history of speech
to minor children bypassing their parents would seem to assume, even if it is an unprotected
category, that some kind of speech for First Amendment purposes is going on. Even
regulations of unprotected low-value speech can offend the First Amendment if a law
discriminates within the unprotected category based on content or viewpoint. See R.A.V. v.
City of St. Paul, 505 U.S. 377 (1992). But Justice Thomas does not conduct an R.A.V. analysis
here.
308. Brown, 564 U.S. at 799–803 (majority opinion).
309. See id. at 799.
310. The social science, he reasoned, failed to support the selective application of the law:
realistically violent games like Grand Theft Auto and cartoonish games like Sonic the
Hedgehog alike were shown to rile kids up. Id. at 800–01. Second, Justice Scalia found the
law to be underbroad insofar as it permitted children to buy the games with their parents’
permission. Id. at 802. If these games were really so harmful, he reasoned, then why not bar
them to children altogether? Id.
311. See id. at 800–02.
2019] REGULATING HABIT-FORMING TECHNOLOGY 175
Brown as support for a thoroughgoing First Amendment protection for video
games, the analogy is superficial. Only three justices of the Brown Court
discuss the features that distinguish videos games from media traditionally
protected by the First Amendment—and far from suggesting that games as
such deserve total protection, they express concern about interactive video
games’ special potential for harm over and above noninteractive media.
312
A clear reading of Brown, then, leaves many aspects of gaming outside the
First Amendment’s coverage. No member of the Court addresses the
relatively questionable “speechiness” of austere, nonrepresentative video
games such as Pong or Candy Crush. Video gambling machine operators
have tried and failed in lower courts to invoke Brown in defense of
“entertaining display[s]” that simulate slot machines.
313
There is simply no
“video games as speech” doctrine that offers any shelter to the various
behavioral techniques that games and other types of “gamified” applications
use to drive engagement.
b. Social Apps and Platforms
The law extends certain extraordinary protections to online platform
owners under the Communications Decency Act of 1996
314
(CDA). Section
230 provides that online platform owners may not “be treated as the publisher
or speaker of any information provided.”
315
The CDA has indeed taken on
such a strong protective aura that tech platforms have begun to raise § 230’s
shield against liability in situations where it has no clear relevance. The
general counsel for Lyft, for example, has taken the position that the CDA
protects it from vicarious liability for the torts of its drivers.
316
Under these
312. Id. at 806 (Alito, J., concurring) (“We should not jump to the conclusion that new
technology is fundamentally the same as some older thing with which we are familiar . . . .
There are reasons to suspect that the experience of playing violent video games just might be
very different from reading a book, listening to the radio, or watching a movie or a television
show.”); id. at 855 (Breyer, J., dissenting) (“I would find sufficient grounds in these studies
and expert opinions for this Court to defer to an elected legislature’s conclusion that the video
games in question are particularly likely to harm children.”).
313. Telesweeps of Butler Valley, Inc. v. Kelly, No. 3:12-CV-1374, 2012 WL 4839010, at
*6 (M.D. Pa. Oct. 10, 2012), aff’d sub nom. Telesweeps of Butler Valley, Inc. v. Att’y Gen.,
537 F. App’x 51 (3d Cir. 2013) (“Unlike in Brown, the simulated gambling programs at issue
here do not contain plots, storylines, character development, or any elements that would
communicate ideas.”); Hest Techs., Inc. v. State ex rel. Perdue, 749 S.E.2d 429, 437 (N.C.
2012) (“While Brown confirmed that First Amendment protection extends to video games, the
Court struck down the state law at issue because it was a content-based restriction on violent
video games. Here [the video gambling ban] applies regardless of the content of the video
game.”); cf. Candy Lab Inc. v. Milwaukee County, 266 F. Supp. 3d 1139, 1146 (E.D. Wis.
2017) (“[W]hat Candy Lab’s game lacks in compelling literary tropes, it makes up for by
employing ‘features distinctive to the medium (such as the player’s interaction with the virtual
world).’”); Alfarah v. City of Soledad, No. 5:15-CV-05569-EJD, 2016 WL 3456697, at *5
(N.D. Cal. June 24, 2016) (“In general, playing or offering games is conduct, not speech.”).
314. Pub. L. No. 104-104, tit. V, 110 Stat. 56, 133–43 (codified as amended in scattered
sections of 18 and 47 U.S.C.).
315. 47 U.S.C. § 230(c)(1) (2012).
316. See Abbey Stemler, The Myth of the Sharing Economy and Its Implications for
Regulating Innovation, 67 E
MORY L.J. 197, 217 (2017).
176 FORDHAM LAW REVIEW [Vol. 88
hyperprotective conditions, there has been no need for courts to engage
seriously with the question of how much protection the First Amendment
itself might provide to online platform owners.
Even so, expectations have been set high. Online platforms today are
widely considered the First Amendment equivalent of newspaper editors.
317
Whatever the merit of this perhaps overly flattering analogy, it nevertheless
would seem to reach only a corner of a Facebook’s or a Google’s
operations—namely, the companies’ decisions to rank or censor content.
Even if Facebook really is comparable to a newspaper, a decision to regulate
addictive user design may be no more offensive than laws requiring the use
of recycled newsprint.
318
Just because an organization does a lot of editing
does not mean that this editing permeates the entire enterprise.
Then there is Justice Kennedy’s arresting remark in 2017’s Packingham v.
North Carolina that the internet, particularly social media, “is the most
important place[] . . . for the exchange of views.”
319
Tonally, Justice
Kennedy’s opinion in that case creates an unmistakable sense of anticipation
that somebody involved with social media, whether users or the platforms
themselves, will soon receive a new shelter under the First Amendment.
Some banned social media users have cited the case to imply that social
platforms are state actors.
320
Social media operators may someday cite
Kennedy’s words in support of a Lochner-like freedom of contract between
platform and user.
But the case, much like Brown, really says very little at all—only that it
was overkill for the state of North Carolina to bar a released sex offender
from most social media for life. If a similar law applied in real space—say,
one that made it unlawful for a convicted sex offender to communicate with
people in public places such as streets and parks—it would have traversed
the free speech pale. Packingham says that total exile from major social
platforms is unacceptable by the same token.
321
This obvious extension of
317. See Eugene Volokh & Donald M. Falk, Google: First Amendment Protection for
Search Engine Search Results, 8 J.L.
ECON. & POL’Y 883, 884–85, 887 (2012); see also Miami
Herald Pub. Co. v. Tornillo, 418 U.S. 241, 256–58 (1974) (striking down a Florida statute
granting a “right of reply” to political candidates personally attacked in newspaper editorial
pages). For opinions applying Tornillo to search engines, see e-ventures Worldwide, LLC v.
Google, Inc., No. 2:14-cv-646-FtM-PAM-CM, 2017 WL 2210029, at *3–4 (M.D. Fla. Feb. 8,
2017); Jian Zhang v. Baidu.com Inc., 10 F. Supp. 3d 433, 436–43 (S.D.N.Y. 2014); Langdon
v. Google, Inc., 474 F. Supp. 2d 622, 629–30 (D. Del. 2007).
318. These laws may raise First Amendment concerns, however, if they are imposed
differentially. See generally Richard Madris, Comment, Recycled Newsprint Laws and
Differential Taxation of the Press, 61 U.
CHI. L. REV. 1069 (1994) (urging that recycled
newsprint laws are unconstitutional because they affect only newspapers).
319. See Packingham v. North Carolina, 137 S. Ct. 1730, 1734 (2017).
320. See Prager Univ. v. Google LLC, No. 17-CV-06064-LHK, 2018 WL 1471939, at *8
(N.D. Cal. Mar. 26, 2018); Nyabwa v. FaceBook, No. 2:17-CV-24, 2018 WL 585467, at *1
(S.D. Tex. Jan. 26, 2018).
321. Packingham, 137 S. Ct. at 1736 (“This case is one of the first this Court has taken to
address the relationship between the First Amendment and the modern Internet. As a result,
the Court must exercise extreme caution before suggesting that the First Amendment provides
scant protection for access to vast networks in that medium.”).
2019] REGULATING HABIT-FORMING TECHNOLOGY 177
First Amendment logic hardly suggests that social platforms or applications
should be any less regulable than privately owned physical spaces. All
Packingham really means is that social platforms are not subordinate forums
within the First Amendment order.
Reno v. ACLU,
322
the 1997 case that extended robust speech protections
to the internet for the first time, is similarly modest: it raises online speech
protections up to the level of offline speech protections, but it does not
suggest that they go any higher.
323
The CDA provisions at issue in Reno
imposed a heavy age and identity verification requirement on online
publishers who distributed certain sexually explicit materials that were
defined as inappropriate for minors.
324
The law was overbroad in that it
encompassed protected nonobscene materials and would have required
consenting adults to disclose personally identifying materials before viewing
them.
325
It would have been struck down if it had been applied to brick-and-
mortar bookstores.
326
The government apparently hoped that the CDA might
get by under a lenient standard similar to the one applied to broadcast
media.
327
The Court declined the invitation. But nothing in the opinion
suggests that online publishers or merchants should enjoy protections beyond
what they would get offline.
328
c. Computer Code
When the Federal Bureau of Investigation (FBI) ordered Apple to unlock
the encrypted phone of the San Bernardino shooter, the company argued that
to do so would constitute “compelled speech.”
329
Such arguments rarely win
the day in court, though judges nevertheless take them more seriously than
they should.
The theory that computer code is a form of speech dates back to the
Northern District of California’s 1996 decision in Bernstein v. United States
Department of State,
330
where the judge reckoned that computer
programming languages should be treated as the equivalent of natural
languages such as English or German.
331
On this theory, statements
communicated in code deserve the same formidable battery of First
322. 521 U.S. 844 (1997).
323. See id. at 870 (“[O]ur cases provide no basis for qualifying the level of First
Amendment scrutiny that should be applied to [the Internet].”).
324. See id. at 857–61.
325. See id. at 881–82.
326. See id. at 864–65 (comparing the CDA unfavorably with a New York statute
restricting minors’ access to nonobscene erotic magazines).
327. See id. at 866–67 (declining to extend FCC v. Pacifica Foundation, 438 U.S. 726
(1978), which upheld FCC regulations against profanity in primetime broadcasts).
328. See generally id.
329. See Panzarino, supra note 293.
330. 922 F. Supp. 1426 (N.D. Cal. 1996).
331. See id. at 1435 (“This court can find no meaningful difference between computer
language, particularly high-level languages as defined above, and German or French. . . .
Even object code, which directly instructs the computer, operates as a ‘language.’” (quoting
Yniguez v. Arizonans for Official English, 69 F.3d 920, 934–36 (9th Cir. 1995))).
178 FORDHAM LAW REVIEW [Vol. 88
Amendment protections as statements communicated in a natural
language.
332
The Bernstein argument has had very little practical effect. Academics
who wanted to use computer code for illustrative purposes have won
victories.
333
But in cases about actual consumer-grade software, courts have
consistently used a watered-down First Amendment analysis to avoid
following Bernstein’s flawed logic through to its preposterous
conclusions.
334
2. The Particular Approach
There is a simple appeal to an approach that says “video games are
speech,” or that “social media is speech,” or that software generally is speech.
But the costs of heightened scrutiny across the board will pile up quickly.
First Amendment law will eventually have to draw some lines among the
internet’s various uses and acknowledge that most of them are
nonexpressive. It is at this point, after the generalistic approach’s easy
answers fall away, that courts might begin to evaluate specific aspects of
addictive design on their own First Amendment merits.
a. Addictive Design as Conventional Message-Bearing Speech
At least some of the design elements in question communicate some kind
of easily articulable message. When Facebook tells you that you have
twenty-nine likes, it is telling you that you have twenty-nine likes. And when
app icons use stop-sign red dots to get your attention, the message is: “Hey!
Over here!”
But messages should not be dismissed as eligible First Amendment content
just because they are austere.
335
Even simple commercial solicitations count.
Expressions Hair Design v. Schneiderman
336
frames “the communication of
prices,” apart from any surrounding advertising content, as a form of
speech.
337
A New York law that banned credit surcharges but not “cash
discounts” regulated merchants’ communicative choices in such a way as to
potentially summon First Amendment protection.
338
332. The analogy proves little, however, unless you assume that statements in English or
German qualify categorically for First Amendment protections. They do not, of course;
natural language has many strictly practical uses—in contracting, in conspiring, in insider
trading, and so on—that contain no First Amendment dimension.
333. See generally Junger v. Daley, 209 F.3d 481 (6th Cir. 2000).
334. Langvardt, supra note 297, at 772–75.
335. Cf. Hest Techs., Inc. v. State ex rel. Perdue, 749 S.E.2d 429, 437 (N.C. 2012) (finding
that a video gambling machine announcing “‘winner,’ or ‘you lose,’ or ‘good job,’ or ‘too
bad,’ or simply show the amount of money won” is not protected speech because “the
announcement is merely a necessary but incidental part of the overall noncommunicative
activity of conducting the sweepstakes”).
336. 137 S. Ct. 1144 (2017).
337. See id. at 1151.
338. Id. at 1146–47.
2019] REGULATING HABIT-FORMING TECHNOLOGY 179
Clip-art-level visual content may also incline a court toward extending
protection to certain elements of interface design. In a recent case, Candy
Lab Inc. v. Milwaukee County,
339
the Eastern District of Wisconsin extended
protection to Texas Rope ‘Em, a Pokémon Go–like scavenger hunt in which
users travel to real world locations to collect cards that the player could add
to a hand of poker.
340
Despite the game’s lack of the “plot, character or
dialogue” elements that made the difference in Brown, the court found
protected content in the “Western-themed virtual environment, complete
with a Texas-themed game title, color scheme, and graphics, allowing the
player to corral favorable playing cards using an animated lasso.”
341
Courts analyzing video gambling machines, on the other hand, have been
less generous. “[M]erely stat[ing] whether a player has won a prize by
displaying a depiction of, for instance, three cherries,” has been held
insufficient for First Amendment protection.
342
It is unclear what accounts
for the difference between the three cherries and the simple “Wild West”
images in Texas Rope ‘Em. It may have less to do with the “speechiness” of
these images as such and more to do with an unspoken decision to lay off the
First Amendment where gambling is concerned.
343
b. Addictive Design as Abstract Expression
Other addictive design techniques cannot be said to convey any message
at all. These elements—such as the “infinite scroll” of your News Feed, the
“pull to refresh” capability of your podcasting app, or the flashing lights and
sounds that intensify the ritual of opening a loot box—fit even less easily into
the First Amendment domain than the clip-art-level expression of Texas Rope
‘Em. But the Supreme Court has stressed that a clear and particularized
message is not necessary to invoke constitutional protection.
In Hurley v. Irish-American Gay, Lesbian & Bisexual Group of Boston,
Inc.,
344
a touchstone in discussions about abstract expression, the Court
upheld a parade organizer’s right to exclude a gay and lesbian organization
from marching in a St. Patrick’s Day festival.
345
The state’s public
accommodation law generally prohibited discrimination on the basis of
sexual orientation—conduct that in most instances is not considered
expressive for First Amendment purposes—and it was unclear what message,
if any, the parade organizer intended to send. But Justice David Souter wrote
for the Court that “a narrow, succinctly articulable message is not a condition
339. 266 F. Supp. 3d 1139 (E.D. Wis. 2017).
340. See id. at 1141.
341. See id. at 1146.
342. Telesweeps of Butler Valley, Inc. v. Kelly, No. 3:12-CV-1374, 2012 WL 4839010, at
*5 (M.D. Pa. Oct. 10, 2012), aff’d sub nom. Telesweeps of Butler Valley, Inc. v. Att’y Gen.,
537 F. App’x 51 (3d Cir. 2013).
343. See generally Amanda Shanor, First Amendment Coverage, 93 N.Y.U.
L. REV. 318
(2018) (arguing that the First Amendment is less likely to cover situations, such as sexual
harassment, where strong social norms mandate clear consequences for the speech).
344. 515 U.S. 557 (1995).
345. See id. at 580–81.
180 FORDHAM LAW REVIEW [Vol. 88
of constitutional protection, which if confined to expressions conveying a
‘particularized message,’ would never reach the unquestionably shielded
painting of Jackson Pollock, music of Arnold Schoenberg, or Jabberwocky
verse of Lewis Carroll.”
346
Developers may argue that much of what makes addictive design work
rests within this zone of sensory abstraction. Notification badges are more
effective in bold colors—red in particular—and in certain organizations of
physical space.
347
Many apps and games put on little productions for the
purpose of building suspense before user events. When a Twitter user “pulls
down” to see recent tweets, the top of the screen shows a short animation.
Loot boxes in video games put on a more elaborate show. The box typically
sits isolated in an austere, empty space and begins to quake. It then explodes
open with a blinding flash accompanied by noises and music. Stars or
confetti fly into the air, and finally the user’s new in-game items are
revealed.
348
What these phenomena lack in an articulable message, they
possess in terms of sheer aesthetic force.
Such arguments can easily be carried too far, though. It is significant that
Justice Souter chose three artists to illustrate his point and that Hurley itself
arose in the context of a parade.
349
Art and parades, however inscrutable
their meaning, are almost purely expressive phenomena by definition. Even
if the expression defies articulation, it is there. But that is not obviously the
case for abstract design choices—say, stripes on wallpaper—that do not
occur within the context of some kind of expressive work. To extend
Hurley’s protection to images generally, sounds generally, or UX design
generally, would disregard the possibility of nonrepresentational meaning—
a critical distinguishing feature of art itself.
One reason to protect Pollock’s work despite its nonrepresentational nature
would be to say that the choice not to represent any particular subject matter
is itself meaningful. In this respect, any public decision to withhold
protections from work that “doesn’t depict anything” or “that my three-year-
old could draw” would discriminate not only in terms of content, but along
even more troubling lines of taste and culture.
350
346. Id. at 569 (citation omitted).
347. Paul Lewis, “Our Minds Can Be Hijacked”: The Tech Insiders Who Fear a
Smartphone Dystopia, G
UARDIAN (Oct. 6, 2017, 1:00 AM), https://www.theguardian.com/
technology/2017/oct/05/smartphone-addiction-silicon-valley-dystopia [https://perma.cc/
7ZWD-PYXW] (“When smartphone users glance at their phones, dozens or hundreds of times
a day, they are confronted with small red dots beside their apps, pleading to be tapped. ‘Red
is a trigger colour,’ [former Google ethicist Tristan] Harris says. ‘That’s why it is used as an
alarm signal.’”).
348. Bailey, supra note 221 (“When an elite soccer player is opened in the game FIFA 18,
fireworks explode and confetti falls. Overwatch’s items soar into view to a triumphant tune.”).
349. Hurley v. Irish-Am. Gay, Lesbian & Bisexual Grp. of Bos., Inc., 515 U.S. 557, 559,
569 (1995).
350. Cf. Nat’l Endowment for the Arts v. Finley, 524 U.S. 569, 576 (1998) (upholding
requirement that the National Endowment for the Arts “tak[e] into consideration general
standards of decency and respect for the diverse beliefs and values of the American public”
when awarding artistic merit grants).
2019] REGULATING HABIT-FORMING TECHNOLOGY 181
As for instrumental music, it may often carry clear national, ethnic, or
cultural messaging that no liberal society has any business prohibiting. And
where the message is harder to discern—say in the work of John Cage or the
noise artist Merzbow, any attempt to suppress that kind of material would
reflect a troubling authoritarian concern with suppressing cultural
subversion.
351
Red badges, infinite scrolls, and other nonrepresentational features of the
consumer interface lie far outside these concerns, which makes it hard to say
that elements of UI design should command any degree of First Amendment
coverage on their own intrinsic merits. A more persuasive theory would hold
that these aspects of design receive protection because of their attention-
grabbing or attention-holding role—their monetizing role—in relation to
products that are expressive for other reasons.
352
c. Addictive Design as an Auxiliary to Speech
The instinct that habit-forming design techniques intersect with First
Amendment coverage is understandable. Yet it is hard to give any persuasive
account for why most gamification tricks—badges, loot boxes, infinite
scrolls, streaks, and so on—should count either as speech or as the sort of
content that cannot be burdened without triggering strict scrutiny. There is
no good reason to say that either software, the internet, social platforms,
video games, or other “gamified” applications should be treated as pure
speech. And even if some design techniques—the two-second pyrotechnics
display that occurs when a loot box opens, for instance—might be viewed as
speech standing alone, techniques such as YouTube’s autoplay function or
Twitter’s pull-to-refresh design are hard to rationalize as First Amendment
material. The best that can be said for them is that they sit adjacent to the
real expression—the “literary and dramatic elements” in video games, the
conversations about life, news and culture within social media, and so on.
This is not to say that adjacent is a bad place to be. First, the addictive UX
in social media and video games helps to monetize the content.
353
If
engagement suffers on Facebook, Facebook’s ability to deliver core speech
suffers as well. And the same is true for video games that are financed largely
or entirely through advertising or microtransactions. Any restriction against
the use of some addictive design technique—say, autoplay—would
presumably burden speech financially. If that burden applies on a content-
discriminatory basis—as if educational video platforms were permitted to
use autoplay but others were not—then strict scrutiny should apply.
354
But
351. See Alan K. Chen, Instrumental Music and the First Amendment, 66 HASTINGS L.J.
381, 425–31 (2015).
352. See Disc. Tobacco City & Lottery, Inc. v. United States, 674 F.3d 509, 547 (6th Cir.
2012) (invalidating as overbroad a ban on all use of color and imagery in tobacco ads displayed
outside certain designated locations).
353. See supra Part I.A.
354. It would be another matter, of course, if some apps were prohibited from trafficking
in addictive UX and others were not. In Sorrell v. IMS Health Inc., the Supreme Court struck
182 FORDHAM LAW REVIEW [Vol. 88
as long as a law regulated autoplay across the board, without respect to the
content of the video or the identity or message or viewpoint of the platform,
then a law regulating autoplay should probably be considered akin to a time,
place, and manner restriction calling for relatively deferential review.
Second, and more broadly, there might be prophylactic reasons not to
allow regulation of design components that are bound up so closely with
expressive subject matter. Loot boxes sit alongside fully protected literary
and dramatic content that should not be disturbed. Autoplay is sewn into
YouTube and Netflix, the new century’s answer to broadcast media.
355
Even
if certain design elements are only doubtfully expressive, courts may choose
to err on the side of caution in light of the design elements’ close association
with material that is clearly expressive. But practically speaking, the
protective buffer can only extend so far if courts are to avoid the Lochner-
like absurdities that would follow from wholesale constitutional protection
for tech.
B. Degrees of Scrutiny
So far, this Article has addressed only the basic question of whether the
First Amendment is in play at all when the government regulates addictive
design. But in those cases where the First Amendment does come into play,
it will become necessary to determine how far the Constitution’s protections
extend. Here lie the usual black-letter questions about the appropriate level
of scrutiny, the adequacy of the means-end fit, and so on.
This Article discusses a couple of doctrinal details that can be made out
today, but the likely state of the doctrine over the next decade or so is too
fluid to speak with precision. The Supreme Court’s almost limitless view of
the reach of “speech,” together with its determination to apply strict scrutiny
to every form of content regulation, is plainly unsustainable within an
economy and society that is increasingly built around information both as an
economic resource and as an apparatus of industrial control.
356
To date,
courts have not formulated a clear and workable theory to describe software’s
down a law against the sale of consumer data to name-brand pharmaceutical retailers. 564
U.S. 552, 580 (2011). There was no need for the Court to say definitively that the data was
actually speech, though the Court hinted strongly at it. See generally id. Instead, it was enough
for the Court to identify the pharmaceutical data as an important input in the expressive
business of marketing and to say that the law discriminated among marketers by denying them
that input on a content- and viewpoint-selective basis. See id. at 571.
355. See Kevin Roose, The Making of a YouTube Radical, N.Y.
TIMES (June 8, 2019),
https://www.nytimes.com/interactive/2019/06/08/technology/youtube-radical.html
[https://perma.cc/TP49-UPRC]; Tufekci, supra note 140; Chris Welch, Netflix Now Lets You
Disable Post-Play to Avoid Binge Watching Entire TV Seasons, V
ERGE (Jan. 27, 2014, 4:13
PM), https://www.theverge.com/2014/1/27/5351268/netflix-now-lets-you-disable-post-play-
avoid-binge-watching-tv [https://perma.cc/RA7A-C3PC].
356. See Ashutosh Bhagwat, When Speech Is Not “Speech,” 78
OHIO ST. L.J. 839, 843–50
(2017) (describing a “coverage crisis” in which the Supreme Court increasingly views all
forms of communication and information sharing as speech and applies strict scrutiny to
content regulation of all kinds).
2019] REGULATING HABIT-FORMING TECHNOLOGY 183
place in the universe of First Amendment speech, let alone worked out a
doctrinal framework.
One point is relatively clear today, and that is that certain modest efforts
to regulate habit-forming apps can probably slide in under rational basis
review. This is for two reasons. First, the First Amendment’s protections for
commercial speech do not extend to deceptive advertising.
357
Second, the
Court has allowed the government to require product labeling where doing
so is reasonably related to the goal of preventing consumer deception.
358
When the government requires advertising content to consist of “purely
factual and uncontroversial information,” the Court has held that those
requirements are not “unjustified or unduly burdensome.”
359
Above, this Article suggested that near-term regulatory efforts may
include a crackdown on aggressive monetization in children’s products.
360
Under their authority to police deceptive practices, the FTC or similar state-
level agencies could push developers either to remove certain habit-forming
features from children’s products or to attach warning labels for parents.
361
At least some of these measures should be able to avoid heightened scrutiny
altogether even assuming that the First Amendment covers the app in
question.
Beyond this point, however, the degree of First Amendment protection, if
any, is wide open. One aspect of this contingency has to do with the basic
question discussed above: whether the design techniques under regulatory
scrutiny constitute speech for First Amendment purposes and, if they do, on
what basis? The answers in this area will inform the question not only of
whether the regulation of addictive UX design brings the First Amendment
into play at all, but also of whether that regulation should count as truly
content discriminatory or not. If a loot box is considered “speech,” then a
loot box ban is content discriminatory. If a loot box is not considered
“speech,” on the other hand, then a ban on loot boxes may be considered a
content-neutral burden on video games that profit from loot boxes.
And it may well turn out that the Court determines the level of scrutiny
based on broader equities rather than the formal divide between content-
based and content-neutral regulation. Most of the Supreme Court had an easy
time applying the formal approach, and ultimately strict scrutiny, in a low-
stakes case involving the ill counsels of violent video games.
362
But in cases
involving international terrorism or even “speech advocating drug use,” even
357. Cent. Hudson Gas & Elec. Corp. v. Pub. Serv. Comm’n, 447 U.S. 557, 563 (1980)
(“[T]here can be no constitutional objection to the suppression of commercial messages that
do not accurately inform the public about lawful activity. The government may ban forms of
communication more likely to deceive the public than to inform it.”).
358. Zauderer v. Office of Disciplinary Counsel, 471 U.S. 626, 628 (1985) (“[A]n
advertiser’s rights are adequately protected as long as disclosure requirements are reasonably
related to the State’s interest in preventing deception of consumers.”).
359. Id. at 651.
360. See supra Part II.A.
361. See supra Part II.A.
362. Brown v. Entm’t Merchs. Ass’n, 564 U.S. 786, 800 (2011).
184 FORDHAM LAW REVIEW [Vol. 88
today’s highly speech-protective Court has shown itself startlingly capable
of inventing new and more deferential tests.
363
If a strong consensus emerges
that behavioral manipulation by big tech constitutes a serious threat to the
public, then the Court’s First Amendment doctrine may adapt to that
consensus in unpredictable ways.
The level of scrutiny will depend still more broadly on the long-term status
of tech platforms and computing within the constitutional order. The issues
here are important enough to public policy and challenging enough to
existing doctrine that it would not be surprising to witness a period of deep
doctrinal modification at the Supreme Court over the coming decades.
Today, tech platforms call on the First Amendment and its statutory
adjunct, § 230 of the CDA, as a regulatory shelter.
364
It seems clear that this
shelter is beginning to weaken. In 2018, Congress qualified § 230 for the
first time in the Stop Enabling Sex Traffickers Act,
365
a law that holds
platforms liable for knowingly assisting, facilitating, or supporting sex
trafficking.
366
And on the First Amendment front, it becomes harder every
year to maintain the posture that computing and the internet consist either
entirely or even mostly of expressive activity. It will not be long before
courts are forced to mark off certain nonexpressive uses of information as
either unprotected or less protected.
It is even possible that some app developers and online platforms may face
a total reversal of fortune with respect to free speech law and norms. If the
legal and policy communities come to view the largest online platforms as
de facto regulators of speech and society, then courts may constitutionalize
their activities through some modification of the state action doctrine. Or, as
this Article’s author has urged in a previous paper, courts may allow
government extra room to ensure that these private regulators protect civil
rights and liberties.
367
The effect in either case would be to put big tech on
the back foot and to give government a free hand.
C
ONCLUSION
This Article has focused mainly on developers’ efforts, through applied
behavioral science, to maximize time on device by encouraging users to form
habits. But note that habit-forming design represents only a crude application
of a wider technology of behavioral manipulation that grows more
sophisticated every year.
It is too indirect to say that Facebook and Google monetize their products
by “placing ads.” It is better to say that these firms’ actual “product” consists
363. See Holder v. Humanitarian Law Project, 561 U.S. 1, 2 (2010) (holding that the law
may prohibit giving “material support” for lawful activities of designated foreign terrorist
organizations); Morse v. Frederick, 551 U.S. 393, 406 (2007) (holding that schools may
prohibit speech, here “BONG HiTS 4 JESUS,” that can be reasonably interpreted as
advocating drug use).
364. See supra notes 315–16 and accompanying text.
365. See 47 U.S.C. § 230(e)(5) (2012).
366. See id.
367. See generally Langvardt, supra note 280.
2019] REGULATING HABIT-FORMING TECHNOLOGY 185
in the ability to raise the odds that a targeted consumer will perform a desired
action following a behavioral cue. At some point, through pervasive
surveillance and conditioning and visual stimuli embedded in users’ contact
lenses,
368
some tech developers may become so good at raising the odds of a
purchase that probability approaches certainty and persuasion approaches
control. As the always PR-challenged Boundless Mind (formerly Dopamine
Labs) boasts in their own promotional materials, “we’ve built a Skinner box
more powerful than anything B.F. [Skinner] himself could have imagined in
his wildest dreams.”
369
Even now, after all of the criticism that has been leveled against big tech,
the basic threat to freedom and dignity has received relatively little public
attention.
370
But the issue is simply too immense to ignore forever. There
will eventually be calls to impose some restraints on overreaching platforms,
and to extend some shelter to objectors who wish to disconnect from the
behaviorist machine. Doing so will involve deep political, practical, and
most likely constitutional difficulty. By taking habit-forming design
seriously, as a legitimate subject matter for public concern and later for public
regulation, we can begin to develop the social capacity to confront those
challenges skillfully.
368. See Doug Bolton, Samsung Patents Design for “Smart” Augmented Reality Contact
Lenses, I
NDEPENDENT (Apr. 6, 2016), https://www.independent.co.uk/life-style/gadgets-and-
tech/news/samsung-smart-contact-lenses-patent-a6971766.html [https://perma.cc/KCV8-
B5KL].
369. See Matt Mayberry, How a 1930’s Harvard Student Laid the Groundwork for
Facebook Likes, B
OUNDLESS MIND (Mar. 22, 2018), https://www.boundless.ai/blog/how-a-
1930s-harvard-student-laid-the-groundwork-for-modern-phone-addiction/
[https://perma.cc/WCH7-XXTP].
370. Some notable books over the past year or so, however, have taken the free will
problem on directly. See, e.g., B
RETT FRISCHMANN & EVAN SELINGER, RE-ENGINEERING
HUMANITY (2018); ZUBOFF, supra note 159.
