人工智能與深度學習

1Artificial

Intelligence

and

DeepLearningSargur

N.

Sriharisrihari@TopicsArtificial

Intelligence

HistoryAI paradigm

shiftsDeep

LearningDisruption

caused

by

AISoftware

developmentSocietalIndia

and

AI2History

of

AI3Greek

MythologyPygmalion

of

CyprusSculpts

marble

Galatea

that

came

to

life

(falls in

love)GBS’

Pygmalion:

Higgins

teaches

Eliza

to

speak

Queen’s

EnglishEliza

is

first

AI

program(Weizenbaum,

MIT)Indian

MythologyGanesa

(Buddhipriya)Parvati

forms

sandalwood

Ganesa,

whose

head

istransplanted

from

an

elephantToday

AI

is

ubiquitousAutomate

routine

laborSearchUnderstand

speechSIRI,

AlexaAutonomous

Vehicles4Handwriting

as

“Fruit

Fly”

of

AINYU,

Toronto,

Montreal,

Baidu5Technology

Ages

(50-year

spans)1771

Industrial

RevolutionArkwright’s

mill

in

Cramford1826

Age

of

Steam

and

Railways“Rocket

steam”

engine

for

Manchester

railway1875

Age

of

Steel,

Electricity

&

Heavy

Eng.Carnegie

Bessemer

steel

plant

in

Pittsburgh1908

Age

of

Oil,

Automobile,

Mass

ProductionFirst

model

T

comes

out

in

Detroit1971

Age

of

Information

&

TelecommunicationsIntel

microprocessor

announced

in

Santa

Clara2017

Age

of

Artificial

IntelligenceMachines

and

people

connectedData

is

new

oil,goldDisruptions

due

to

AI742%

of

US

jobs

at

riskover

two

decades.AI

will

outperform

humans

in*:Translating

languages

2024Driving

a

truck

2027Working

in

retail

2031Working

as

a

surgeon

2053Outperform

humans

in

all:

50%

chance

in

45

yearsAutomating

all

human

jobs

120

years*NIPS/ICML

SurveyInternational

(Rich

and

mid-income):14%

of

jobs

in

32

countries

have

70%

riskFor

32%

of

jobs,

that

probability

is

60%Thus

210m

jobs

at

risk

in

those

countries

alone.Rich-countries

less

at

risk

than

mid-income

statesNational

AI

Initiatives8“Artificial

Intelligence

is

poised

to

disrupt

the

world”NITI

AayogAI

ParadoxHard

problems

for

people

are

easy

for

AIEasy

problems

are

hard

for

AI–

Narrow

IntelligenceGeneral

IntelligencePeople

easy

tasks:9What

tasks

require

intelligence?Reasoning–

Puzzles,

JudgmentsPlanningAction

sequencesLearningImprove

with

dataNatural

languageIntegrating

skillsAbilities

to

sense,

act10Everyday

life

needs

knowledgeKnowledge

is

intuitive

and

subjectiveKey

challenge

of

AI

is

how

to

get

this

informalknowledge

into

a

computerKnowledge-based

ApproachHard-code

knowledge

in

a

formal

languageComputer

can

reason

about

statements

in

theselanguages

using

inference

rules11Knowledge-Based

AIInputHand-designedprogramOutputRule-based

SystemDisadvantage:

Unwieldy

processTime

of

human

expertsPeople

struggle

to

formalize

rules

with

enough

complexity

to

describe

the

worldThe

Machine

LearningapproachDifficulties

of

hard-coded

approach

suggests:–

Allow

computers

to

learn

from

experienceDetermine

what feature

representations

to

useMap

the

features

to

outputsDecide

whether

email

is

spam13Two

paradigms

in

AIInputHand-designedprogramOutputInputHand-designedFeaturesMapping

fromfeaturesOutputRule-based

SystemClassic

MachinelearningShaded

boxes

indicate

components

that

can

learn

from

dataDesigning

right

set

of

featuresSimple

Machine

Learning

depends

heavily

onrepresentation

of

given

dataFor

detecting

a

car

in

photographs–

Tire

shape

difficult

in

terms

of

pixel

values–

Shadows,

glare,

occlusion15Representation

LearningSolution:

use

ML

to

not

only

learn

mapping

fromrepresentation

to

output

but

representation

itselfBetter

results

than

hand-coded

representationsAllows

AI

systems

to

rapidly

adapt

to

new

tasksDesigning

features

can

take

great

human

effortCan

take

decades

for

a

community

of

researchersDoes

not

need

programmer

to

have

deepknowledge

of

the

problem

domain16Deep

LearningUnderstand

the

world

as

hierarchy

of

conceptsHow

these

concepts

are

built

on

top

of

each

other

isdeep,

with

many

layersWeights

learnt

by

gradient

descentxtxt

1

xt

f

(xt

)UnsupervisedRepresentationLearningAutoencoderEncoder:Converts

input

into

arepresentationDecoder:Converts

representationback

to

input18New

designs

from

representationDeep

StyleLanguage

Processed

as

NumbersTraining

DataWord-to-vecOne-hot

vector

mappedto

vector

of

300Word

embeddingSimilar

words

are

closetogether19InputAdditional

layersof

more

abstractfeaturesMapping

fromfeaturesOutputSimple

featuresThree

Paradigms

of

AIInputHand-designedprogramOutputInputHand-designedFeaturesMapping

fromfeaturesOutputRule-based

SystemClassic

MachinelearningInputFeaturesMapping

fromfeaturesOutputRepresentation

LearningDeep

LearningShaded

boxes

indicate

components

that

can

learn

from

dataDisruption

in

Software

DevelopmentDeep

Learning

is

not

just

another

toolIt

is

a

fundamental

shift

in

how

software

is

writtenSoftware

1.0

(Classical

“stack”)It

is

code

we

writee.g.,

LAMP(Linux,

Apache,

MySQL,

Python/Perl)Software

2.0

(Code

written

by

Optimizer)It

is

in

a

user

unfriendly

languageThere

are

millions

of

weightsNo

human

involved

in

coding21Software

1.0

and

2.0:

FizzbuzzPrint

i=

1

to

100,

except:if

divisible

by

3

print

fizz,if

divisible

by

5

print

buzz,if

divisible

by

both

3

and

5

print

fizzbuzzTwo

approaches:– Software

1.0:

C++ Software

2.0:

Python/Tensorflow122def

model(X,

w_h,

w_o):h

=

tf.nn.relu(tf.matmul(X,

w_h))return

tf.matmul(h,

w_o)Program

Space:

Software

1.0

vs

2.0*A.

Karpathy,

Tesla23Software

1.0By

writing

each

line

of

code,

programmer

identifies

a

point

inprogram

space

with

some

desirable

behaviorSoftware

2.0Restrict

search

space

to

continuous

subset

of

program

spaceSearch

is

made

efficient

by

using

stochastic

gradient

descentBenefits

of

Software

2.0Computationally

homogeneousSandwich

of

two

operations:

matrix

multiply,

RELUSimple

to

bake

into

siliconSmall

instruction

setConstant

run

timeEvery

iteration

of

forward

pass

has

same

FLOPSConstant

memory

useHighly

portable:

sequence

of

matrix

multiplies

is

easierVery

agileC++

is

hard

to

speed-up,

instead

remove

half

of

channelsCan

meld

into

optimal

wholeSoftware

often

has

modules,

can

jointly

optimize–

It

is

better

than

you24Limitations

of

Software

2.0stackAfter

training

we

have

large

networks

that

workvery

well,

but

hard

to

tell

how90%

accurate

model

we

understand99%

accurate

model

we

don’tCan

fail

unintuitively

(Adversarial

examples)Correct

steering

With

darker

imagey

=“panda”

y

=“gibbon”with

58%

with

99%confidence

confidenceHow

Deep

Learning

disrupts

CSSoftware

DeveloperPresent:

Write

and

maintain

layers

of

tangled

codeFuture:

A

teacher–

curate

training

data

&

analyze

resultsMathematicsPresent: Logic,

Discrete

mathematicsFuture:

Probability

Theory,

Calculus,

Linear

AlgebraProgramming

EnvironmentsPresent:

C++,

JavaFuture:

Tensorflow/Pytorch/Gluon/…–

In

ten

years

most

software

jobs

won’t

involve

programmingHardwarePresent:

CPUsFuture:

GPUs26The

transitionConventional

software

can

be

replaced

with

adeep

learning

solution

with

improvement,

e.g.*,Upgrading

search

rankingData

center

energy

usageLanguage

translationSolving

G