Calibration and Validation of Models | BSc.CSIT | Simulation and Modeling | 5th Sem

Download our Android App from Google Play Store and start reading Reference Notes Offline.

calibration and validationCalibration and Validation of Models,
Simulation and Modeling Reference Notes
Fifth Semester | Third year
BSc.CSIT | Tribhuvan University (TU)

Calibration and Validation of Models
Calibration is the iterative process of comparing the model to the real system, making adjustments to the model, comparing again and so on. The comparison of the model to reality is carried out by variety of test.

calibration of modelTests are subjective and objective. Subjective test usually involve people, who are knowledgeable about one or more aspects of the system, making judgments about the model and its output. Objective tests always require data on the system’s behavior plus the corresponding data produced by the model.

A possible criticism of the calibration phase, were it to stop at point, i.e., the model has been validated only for the one data set used; that is, the model has been “fit” to one data set.

Validation Process
As an aid in the validation process, Naylor and Finger [1967] formulated a three step approach which has been widely followed:-

  1. Build a model that has high face validity.
  2. Validate model assumptions.
  3. Compare the model input-output transformations to corresponding input-output transformations for the real system.

1. Face Validity:-

  • The first goal of the simulation modeler is to construct a model that appears reasonable on its face to model users and others who are knowledgeable about the real system being simulated.
  • The users of a model should be involved in model construction from its conceptualization to its implementation to ensure that a high degree of realism is built into the model through reasonable assumptions regarding system structure, and reliable data.
  • Another advantage of user involvement is the increase in the models perceived validity or credibility without which manager will not be willing to trust simulation results as the basis for decision making.
  • Sensitivity analysis can also be used to check model’s face validity.
  • The model user is asked if the model behaves in the expected way when one or more input variables is changed.
  • Based on experience and observations on the real system the model user and model builder would probably have some notion at least of the direction of change in model output when an input variable is increased or decreased.
  • The model builder must attempt to choose the most critical input variables for testing if it is too expensive or time consuming to: vary all input variables.

2. Validation of Model Assumptions:-

Model assumptions fall into two general classes: structural assumptions and data assumptions.

  1. Structural assumptions involve questions of how the system operates and usually involve simplification and abstractions of reality.
  2. Data assumptions should be based on the collection of reliable data and correct statistical analysis of the data.

The procedure for analyzing input data consist of three steps:-

  1. Identifying the appropriate probability distribution.
  2. Estimating the parameters of the hypothesized distribution.
  3. Validating the assumed statistical model by goodness – of – fit test such as the chi-square test, K-S test and by graphical methods.

3. Validating Input-Output Transformation:-

  • In this phase of validation process the model is viewed as input – output transformation. i.e; the model accepts the values of input parameters and transforms these inputs into output measure of performance. It is this correspondence that is being validated.
  • Instead of validating the model input-output transformation by predicting the future ,the modeler may use past historical data which has been served for validation purposes that is, if one set has been used to develop calibrate the model, its recommended that a separate data test be used as final validation test.
  • Thus accurate “prediction of the past” may replace prediction of the future for purpose of validating the future.
  • A necessary condition for input-output transformation is that some version of the system under study exists so that the system data under at least one set of input condition can be collected to compare to model prediction.
  • If the system is in planning stage and no system operating data can be collected, complete input-output validation is not possible.
  • Validation increases modeler’s confidence that the model of existing system is accurate.
(Visited 13 times, 1 visits today)

Posted By : Digvijay | Comment RSS | Category : Fifth Semester
Tag :

Post a Comment

Your email is never published nor shared. Required fields are marked *


Wordpress DMCA
Community | Toolbar | Android App | Founder/Developer : Hari Prasad Chaudhary | CSIT Portal Manager : Digvijay Chaudhary