What Is Deep Learning (The Rise of Artificial Intelligence)
To support the production of more
high-quality content
consider supporting us on
Patreon or
YouTube membership.
Additionally, consider visiting our
parent company, EarthOne.
For sustainable living made simple!
Deep learning is a sub-field of AI
that has taken the world by storm,
in large part, since the start of this decade.
In this sixth video in our artificial intelligence series
and as for the purpose of this deep learning series,
we'll explore why it has exploded in popularity,
how deep learning systems work
and their future applications.
So sit back, relax and join me on an exploration
into the field of deep learning.
As I hope I've gotten across
in past videos in this AI series,
artificial intelligence refers to any model that can mimic,
develop or demonstrate human thinking,
perception or actions.
In the case of AI that we all refer to today
it is about computational intelligence,
that being intelligence
demonstrated by computing technology.
Machine learning then is the ability
for computers to learn without being explicitly programmed.
In other words,
having the ability to derive structure from data,
that being the features of a data set,
to be able to predict future outputs,
whether that output be a value or a category.
We discussed both supervised
and unsupervised learning techniques
quite in-depth in our machine learning series
and going a step further
also discussed the 5 tribes of machine learning,
highlighting the different thought philosophies
in giving computers learning capabilities.
Over the past few years,
the rise in artificial intelligence in mainstream popularity
has been attributed to the successes
of the connectionist tribe of machine learning,
spawning a new field, deep learning.
The goal of connectionism
is to digitally re-engineer the brain
and all of its connections.
The title given to this digital reconstruction
of the connections in a brain is a neural network.
Deep learning then is the ability
to learn the underlying features in a data set
using neural networks.
By this meaning, the ability to learn features
directly from data rather than being hand-engineered.
As quoted from a seminal paper on deep learning
from the godfathers of the deep learning revolution,
Geoffrey Hinton, Yoshua Bengio and Yann LeCun:
Conventional machine-learning techniques
were limited in their ability
to process natural data in their raw form.
For decades, constructing a pattern-recognition
or machine-learning system required careful engineering
and considerable domain expertise
to design a feature extractor tht transformed the raw data,
such as the pixel values of an image,
into a suitable internal representation
or feature vector from which the learning subsystem,
often a classifier, could detect
or classify patterns in the input.
Representation learning is a set of methods
that allows a machine to be feed with raw data
and to automatically discover the representations needed
for detection or classification.
Deep-learning methods are representation-learning methods
with multiple levels of representation,
obtained by composing simple but non-linear modules
that each transform the representation at one level,
starting with the raw input,
into a representation at a higher,
slightly more abstract level.
With the composition of enough such transformations,
very complex functions can be learned.
For classification tasks,
higher layers of representation amplify aspects of the input
that are important for discrimination
and suppress irrelevant variations.
Let's now take this definition
of representation-based learning and apply it to an example,
say of a neural net detecting a face in an input image.
First off, this image is actually composed
of an array of pixel values as the input to our system.
Following that, the first few layers of representation
learn low-level features like the presence
or absence of lines and edges at particular orientations
and locations in the image. The next layers of
representation begin discerning mid-level features
based off the low-level features previously found.
This involves finding particular arrangements
of edges and lines and producing features like eyes,
noses and ears.
Finally, the last layers of representation
discover high-level features,
that being a combination of mid-level features,
to identify different facial structures.
The key aspect of deep learning
is that these layers of features
are not chosen by human engineers,
they are learned from data
using a type of general-purpose learning procedure.
This as opposed to the hand-engineered features
of traditional machine learning algorithms,
which is time consuming, costly,
and overall not scalable for more complex applications.
The true birth of deep learning,
that being when the mainstream got wind of it,
and where an algorithm publicly demonstrated
this feature detection capability came back in 2012
at the ImageNet competition.
Since 2010,
the annual ImageNet Large Scale Visual Recognition Challenge
is a competition where research teams
evaluate their algorithms on image data sets.
These image data sets are comprised of millions of images
in approximately 1000 image categories.
Along with a variety of changing completion milestones
the general year to year goal
is to get as low as a possible error rate
in the classification of images in the data set.
In 2010, the inception of the challenge,
the error rate was 28%.
Progressing forward, in 2011,
this error rate for the winning team dropped to 26%.
However, as you can see there was quite a bit of spread
in between the teams.
In 2012, most teams began converging
around the 25 to 30% range in error rate
through the use of traditional image recognition,
in other words classification machine learning algorithms,
like those we discussed in our machine learning series.
There was one major outlier in 2012 however,
the winner, an algorithm dubbed SuperVision,
with an error rate of 16%,
nearly 10% better than its closest competitor.
SuperVision, more academically referred to as AlexNet,
is a deep convolutional neural network with eight layers,
terms we will dissect later in this series.
It was a network designed
by University of Toronto professor Geoffrey Hinton
and his graduate students.
By the next year, 2013,
nearly every top five team
had switched to a deep learning approach,
with the winning team having an error rate of 12%
and eight layers.
Progressing forward,
in 2014 the winner had 22 layers and an error rate of 7%
and by the next year it was the first time
that the human record error rate on the dataset of 4.94%
was breached by a network dubbed ResNet,
comprised of 152 layers, and having an error rate of 4%.
As an additional note,
2016's winner had an error rate of 3% and 2017's of 2.25%.
Had it not been for AlexNet in 2012,
the error rate may have lingered around 20%
for another few years.
This because while most machine learning algorithms
that have been in use for decades
work great for smaller, structured data sets,
when it comes to more real-word datasets,
large and unstructured, the performance
of machine learning algorithms falls drastically.
Deep learning acts like a turbocharger of sorts
for machine learning algorithms.
As stated earlier, instead of manual feature extraction,
the neural network acquires its own internal representations
of the data, that being, the relevant features
that describe it.
To add to this, these networks can be trained
in an unsupervised or supervised manner
for both unsupervised and supervised tasks.
As quoted from the paper discussing
this breakthrough in 2012 titled,
"ImageNet Classification
"With Deep Convolutional Neural Networks":
Our results show that a large,
deep convolutional neural network
is capable of achieving record-breaking results
on a highly challenging dataset
using purely supervised learning.
To simplify our experiments,
we did not use any unsupervised pre-training
even though we expect that it will help,
especially if we obtain enough computational power
to significantly increase the size of the network
without obtaining a corresponding increase
in the amount of labeled data.
Our results have improved
as we've made our network larger and trained it longer.
So, to reiterate, the AlexNet victory in 2012
was a neural net trained in a supervised manner
executing on an unsupervised dataset.
As I'm sure is clear now,
these networks and the results they produced
translated to nearly every task
machine learning algorithm's tackled
beyond just image recognition.
Neural nets in principle have existed for decades
as we saw in the Birth of AI video,
so why their sudden rise in effectiveness
and popularity now?
Well, as we discussed in the last video
in this series on machine learning,
some major factors include the pervasiveness of big data
and increases in computing storage and power.
Other major factors include
increasingly parallelizable applications,
streamlined software interfaces like TensorFlow,
the list can go on and on.
As stated earlier,
conventional machine-learning techniques
were limited in their ability
to process natural data in its raw form,
whereas deep learning benefits
from vast amounts of training data to produce its models.
In other words, deep learning is able to leverage big data.
Also, deep learning, unlike traditional machine learning,
excels at processing unstructured data,
as the network's underlying structure
derives the representations and features of a data set.
Additionally, the composition of the networks,
that being the mathematics these neural nets operate on,
matrix operations, benefit greatly from parallel operation
and therefore increase the computing performance.
While this is a lot of terminology all at once,
I promise this will all make more sense
by the end of this deep learning series
after we've discussed the structure of neural
networks and how they work.
Coming back on topic, as I hope you can see now,
deep learning really is the culmination of advances
in many fields and the hard work and dedication
of various groups and individuals.
When most people in the mainstream say AI now-a-days
or when businesses use the term to hype a product,
they are really referring to machine learning
and when they say machine learning
they really mean deep learning.
Just like Matryoshka dolls,
they are further subsets of one another.
As I hope you took away from our machine learning series,
machine learning on a high-level
is essentially pattern recognition,
a subset of intelligence.
Deep learning then is turbo-charged pattern recognition.
Over the past videos
we have seen the field of connectionism evolve
from the single-layer perceptron nets of the '60s,
to the shallow multi-layer networks of the '80s,
and many other innovations
leading to the development of modern deep networks
with tens to hundreds of layers,
each building layers of representation
and abstraction on top of one another.
Now while it may take a lot of work
to build a deep learning system
as we'll see in the next videos in this series,
If you want to learn more about
deep learning, and I mean
really learn about the field,
from how these artificial learning algorithms
were inspired from the brain
to their foundational building blocks,
the perceptron. Scaling up to
multi-layer networks, different types
of networks such as
convolutional networks and recurrent
networks and much more,
then Brilliant.org is the place
for you to go!
Now what we love about how
the topics in these courses are
presented is that first an
intuitive explanation is given and then
you are taken through related problems.
If you get a problem wrong
you can see an explanation for where you
went wrong, and how to rectify that flaw!
In a world were automation through
algorithms will increasingly replace more jobs,
it is up to us as individuals to keep our
brains sharp and think of creative solutions
to multi-disciplinary problems!
To support Futurology and learn more
about Brilliant, go to:
Brilliant.org Futurology
and sign up for free!
Additionally, the first 200 people that
go to that link will get 20% off
their annual premium subscription!
(bright music)
At this point the video has concluded,
we'd like to thank you for taking
the time to watch it!
If you enjoyed it, consider
supporting us on Patreon or
YouTube membership to keep
this brand growing!
And if you have any topic
suggestions, please leave them in
the comments below.
Consider subscribing for more content
and check out our website and
parent company, EarthOne, for
more information!
This has been Ankur,
you've been watching Futurology
and we'll see you again soon!
(upbeat percussive music)