Context model
A context model defines how context data are structured and maintained. It aims to produce a formal or semi-formal description of the context information that is present in a context-aware system. In other words, the context is the surrounding element for the system, and a model provides the mathematical interface and a behavioral description of the surrounding environment.
It is used to represent the reusable context information of the components.
A key role of context model is to simplify and introduce greater structure into the task of developing context-aware applications. Prompt engineering is equivalent to context engineering, where a Large language model prompt establishes a context for a development task.
Examples of context models
The Unified Modeling Language as used in systems engineering defines a context model as the physical scope of the system being designed, which could include the user as well as the environment and other actors. A system context diagram represents the context graphically..Several examples of context models occur under other domains.
- In the situation of parsing a grammar, a context model defines the surrounding text of a lexical element. This enables a context sensitive grammar that can have deterministic or stochastic rules. In the latter case, a hidden Markov model can provide the probabilities for the surrounding context.
- A context model can also apply to the surrounding elements in a gene sequence. Like the context rules of a grammar disambiguating a lexical element, this helps to disambiguate the role of the gene.
- Within an ontology, a context model provides disambiguation of a subject via semantic analysis of information related to the subject.
- In terms of a physical environment, a context model defines the external interfaces that a system will interact with. This type of context model has been used to create models for virtual environments such as the Adaptive Vehicle Make program. A context model used during design defines land, aquatic, or atmospheric characteristics that the eventual product will face in the real environment.
- In the context of large language models, a context model refers to a component or aspect of the language model that focuses on understanding and incorporating contextual information from the input text. The main purpose of a context model is to provide the language model with a better understanding of the context surrounding words, phrases, or sentences, so that it can generate more coherent and contextually appropriate responses. In deep learning-based language models like GPT-4 or BERT, the context model is an inherent part of the architecture. These models use mechanisms such as attention mechanisms and multi-layered transformer (machine learning) architectures to capture contextual information from the input sequence. The context model takes into account the relationships between words and their surrounding text, helping the language model understand the meaning of a word in a specific context, handle ambiguities, and generate more accurate and coherent responses.
- Examples of AI-based weather forecasting systems that apply context models include Google DeepMind's GraphCast, Huawei's PanguWeather, and NVIDIA's FourCastNet, drawing from historical and re-analysis context data. In general, the approach is to match up current conditions using past data as context and then apply a mix of physics and historical outcomes to form a projection.