Convolutional neural network

Yeah.

They used the metadata from those photos, the titles, descriptions, and hashtags to train convolutional neural networks, or CNNs, to predict words.

Okay, 100 million photos sounds like the massive scale we're looking for.

Were they predicting full natural language sentences?

Pooling is how the network shrinks the resolution of the image as it processes deeper layers, kind of summarizing the features.

But what is the anti-aliased blur part actually doing?

It solves a classic fragility in CNNs.

In a standard convolutional network, when you reduce the resolution of an image, when you downsample it, the network can lose what's called spatial translation invariance.

How do I do that? Okay, how about something with a few more lines of context and a few more lines required to complete the objective? In this case, we'll start with some common TensorFlow and Keras imports, and then we'll load in some data set that we have.

And then again, we'll use a comment to suggest the following code should be a three layer ConvNet with a 64 by 64 imagery with five classes.

I tried to pick sizes and classes that are a little more rare, so something like 28x28 and 10 classes is going to be super common, for example.

64x64 might be common, I don't know.

So you can use that in your projects down the road.

Cool, so just to kind of blaze past things, I kind of started by talking through where things were pre-2017 when the Transformer paper came out on both the RNN and the CNN side, and kind of tied a lot of the innovation around the transformer to how modern convolutional neural nets, specifically residual nets, were working.

And kind of the connections there were closer than the connections to RNNs.

Kind of walk through how we got to the self-attention block with this fancy code, which is basically just saying like you're splitting your heads and you can kind of think of your heads in a self-attention block as the different kind of kernels or filters in a CNN layer.

And then kind of closing with this full self-attention block, where we're actually doing the RNN-style attention.

And then this question of this non-linearity that we're adding at the end.

Because without this non-linearity and this sort of MLP that we're adding to the end of each transformer block, we're really just doing weighted averages of linear transforms of values.

OK, so if we kind of take this as ground truth, starting point for what a transformer block looks like, very much inspired by the ideas of CNNs and RNNs with attention at the time, we have this residual connection here, which is kind of just adding x over and over again as we stack more and more layers together.

We leave that on the soft core arm that we have lying around there on FPGA.

CNNs can work very well on FPGAs.

So we try also with hardware.

I think eventually, because putting good hardware or special hardware for a case is one thing that you can do.

So on-prem versions of a factory, co-location, the idea of a neocloud or an NCP, and of course, public cloud.

I mean, the large pharmas of the world, certainly on the cutting edge, are you thinking that some of those other types of models, the CNNs, the world models, things like that, those are going to be eventually used in all enterprises across the board?

Yeah, I think it's a matter of time.

Right now, you're starting to see more like SLM, TLM, small language model, tiny language models.

And that lack of context causes endless false alarms.

Intelligent security systems discard simple motion detection in favor of convolutional neural networks or CNNs.

What do CNNs do?

They process high definition video feeds in real time.

A CNN doesn't just look for changing pixels.