Skip to content

4. LOADING MODEL ARGS

loadModelArgs is called if includeTensors is true.

In this phase, the model configuration file "params.json" is parsed as ModelArgs object via JSON parser.

Then, loadModelArgs function takes its result and assigns it to result object's ModelArgs property: model.ModelArgs.

from src/model/loader.go

func LoadModelEx(modelDir string, includeTensors bool, includeVocab bool) (*Model, error) {
    if includeTensors {
        ...
        err = loadModelArgs(modelDir, model)
        if err != nil {
            return nil, err
        }
    }
    ...
}

Now, we have model.ModelArgs object with similar values:

Dim: 4096
N_Layers: 32
N_Heads: 32
N_KVHeads: -1 // if -1, it will be equal to N_Heads
VocabSize: -1 // if -1, it will be size of tokenizer.model
MultipleOf: 256
FFNDimMultiplier: -1 // will be calculated further
NormEpsilon: 0.000001
MaxSequenceLength: 4096
N_Rep: 0 // will be calculated further
HeadDim: 0 // will be calculated further