GPT-4 is a new AI model with an unprecedented 1.8 trillion parameters, dwarfing its predecessor GPT-3 by tenfold.

The model incorporates 16 specialized experts, each with a substantial 111 billion parameters focused on multi-layer perceptrons (MLPs).

GPT-4's training involved 13 trillion tokens, spanning both text and code, and cost an astounding $63 million.

Operating GPT-4 requires considerable resources, including 128 GPUs and advanced parallelism techniques, and it's three times costlier to run than its predecessor.

The model features a breakthrough vision encoder, enhancing its ability to interpret visual data from web pages, images, and videos.

Looking ahead, GPT-5 promises to exponentially increase complexity, suggesting ongoing, rapid advancement in AI capabilities.