LLM Primer

Channels

Channels are parallel grids of values.

An RGB image has three input channels: red, green, and blue. A hidden convolutional layer may have many channels, each holding a feature map.

channels in, feature maps outRGBedgetexture
Kernels read input channels and produce output feature maps.

A kernel for a multi-channel input usually reads across all input channels. If the input has 3 channels and the kernel is 3 x 3, then one output feature map uses:

3 * 3 * 3 = 27

weights, plus usually one bias.

Output channels are the number of feature maps produced by the layer.

DL-C14-T05-001Exercise: Kernel weights with channels

A kernel is 3 x 3 and reads 4 input channels. How many weights does it have?

Compute it first, then check your number.

DL-C14-T05-002Exercise: Output channels

A layer has 16 kernels. How many output channels does it produce?

Compute it first, then check your number.