AI 3.0（畅销书《复杂》作者、复杂系统前沿科学家梅拉妮·米歇尔，历经10年思考，厘清人工智能与人类智能的全新力作，全景式地展现了人工智能的...

27 W. Knight,“How Long Before AI Systems Are Hacked in Creative New Ways?”, MIT Technology Review , Dec. 15, 2016.

28 J. Clune,“How Much Do Deep Neural Networks Understand About the Images They Recognize?”, lecture slides (2016), accessed Dec. 7, 2018.

07 确保价值观一致_，构建值得信赖、有道德的人工智能

1 D. Palmer,“AI Could Help Solve Humanity's Biggest Issues by Taking Over from Scientists, Says DeepMind CEO”, Computing , May 26, 2015.

2 S. Lynch,“Andrew Ng: Why AI Is the New Electricity”, Insights by Stanford Business , March 11, 2017.

3 J. Anderson, L. Rainie, and A. Luchsinger, “Artificial Intelligence and the Future of Humans”, Pew Research Center, Dec. 10, 2018.

4 关于人工智能和大数据的道德问题的两种最新处理方法参见C. O'Neil, Weapons of Math Destruction:

5 C. Domonoske,“Facebook Expands Use of Facial Recognition to ID Users in Photos”, National Public Radio, Dec. 19, 2017.

7 J. Snow,“Amazon's Face Recognition Falsely Matched 28 Members of Congress with Mugshots”, Free Future (blog), ACLU, July 26, 2018.

8 B. Brackeen,“Facial Recognition Software Is Not Ready for Use by Law Enforcement”, Tech Crunch , June 25, 2018.

17 同上, 192。

08 强化学习_，最重要的是学会给机器人奖励

2 更确切地说，这是一种被称为“值函数学习”（value learning）的强化学习方法，但并不是唯一可行的 方 法 。 还 有 一 种 被 称 为 “ 策 略 学 习 ” （ policylearning）的强化学习方法，其目标是直接学习一个给定状态下要执行的动作，而非首先学习如何获得更优动作的值。

3 C. J. Watkins & P. Dayan,“Q-Learning”, Machine Learning 8, nos. 3-4 (1992): 279-292.

与强化学习有关的详细技术性介绍，参阅R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction , 2nd ed. (Cambridge, Mass.: MIT Press, 2017)。

5 参考示例，请查阅如下文章：P. Christiano et al.,“Transfer from Simulation to Real World Through Learning Deep Inverse Dynamics Model”, arXiv: 1610.03518(2016); J. P. Hanna and P. Stone, “Grounded Action Transformation for Robot Learning in Simulation”, in Proceedings of the Conference of the American Association for Artificial Intelligence (2017), 3834-3840; A. A. Rusu et al., “Sim-to-Real Robot Learning from Pixels with Progressive Nets”, in Proceedings of the First Annual Conference on Robot Learning , CoRL (2017); S. James, A. J. Davison, and E. Johns, “Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-stage Task”, in Proceedings of the First Annual Conference on Robot Learning , CoRL (2017); M. Cutler, T. J. Walsh, and J. P. How,“Real-World Reinforcement Learning via Multifidelity Simulators”, IEEE Transactions on Robotics 31, no. 3 (2015): 655- 671。

09 学会玩游戏_，智能究竟从何而来

1 P. Iwaniuk,“A Conversation with Demis Hassabis, the Bullfrog AI Prodigy Now Finding Solutions to the World's Big Problems”, PCGamesN , accessed Dec. 7, 2018.

2 “From Not Working to Neural Networking”, Economist , June 25, 2016.

3 M. G. Bellemare et al.,“The Arcade Learning Environment: An Evaluation Platform for General Agents”, Journal of Artificial Intelligence Research 47(2013): 253-279.

这里更深层的技术性原理是：DeepMind的程序使用了一种被称为“epsilon-greedy”的方法来选择每个时步的动作。程序以epsilon 的概率随机选择一个动作；以1-epsilon 的概率选择具有最高值的动作。epsilon 是一个介于0和1之间的数值，其初始值被设置为接近1，经多个片段训练后逐渐减小。

5 R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction , 2nd ed. (Cambridge, Mass.: MIT Press, 2017), 124.

更 多 细 节 参 见 V. Mnihet al.,“Human-Level Control Through Deep Reinforcement Learning”, Nature 518, no. 7540 (2015): 529。

7 V. Mnih et al.,“Playing Atari with Deep Reinforcement Learning”, Proceedings of the Neural Information Processing Systems (NIPS) Conference, Deep Learning Workshop (2013).

8 “Arthur Samuel”, History of Computer website.

9 塞缪尔的程序使用了可变数量的层，具体选择哪种，取决于行棋动作。

10 塞缪尔的程序在每一轮还使用了一种被称为“alpha-beta剪枝”的方法来确定博弈树中的哪些节点不需要被评估。alpha-beta剪枝也是IBM的国际象棋程序深蓝的重要组成部分。

12 同上。