Root mean square

So in the original transformer, here we're normalizing across all the activation by first recentering X and then having a, a term gamma and then, uh, so some bias beta, which helps with training stability and convergence.

But people have tried with other normalization methods, and in particular, they have found that RMSNorm, RMS stands for root mean square norm, actually does the job just as well.

And here you don't need to recenter X.

You just need to rescale it and to have just one factor that you need to learn, and that is the reason why people tend to use it.

And I think this was all the tricks I had.

So just as a summary, before, on the normalization front, people used to do PostNorm and Layer Norm, and nowadays people do PreNorm and RMSNorm.

So long story short, nowadays, uh, RoPE, local attention, GQA, PreNorm, and RMSNorm.

And with that, I'll give it to Shervin.

So what are the actual practical rules? If someone walks into a shop, what do they buy?

The standard guidance from the wire guides, 18 gauge for home or car speakers up to about 25 feet at 50 watts RMS or less, 16 gauge for longer runs or if you're pushing 75 to 100 watts, 14 gauge for runs of 100 feet or more or for high-power 2 and 4-ohm subwoofers.

Another way of putting it, 18 gauge is fine for about 50 watts into a 4-ohm load or about 100 watts into 8 ohms at 50 feet and 100 feet respectively.

And peak power versus RMS matters here, doesn't it?

Hugely.

Speaker and amplifier marketing loves peak power, 2,000 watts on the box.

Peak is a momentary figure.

It's meaningless for sizing.

By quadratic mean is denoted the square root of the mean of the squares of all the values between the limits considered.

Other names for the quadratic mean of any quantity are its virtual value or its RMS, meaning root mean square value.

The French term is valeur efficace.

If y is the function under consideration and the quadratic mean is to be taken between the limits of x equals zero and x equals l, then the quadratic mean is expressed as the square root of the quantity one over l integral from zero to l of y squared dx.